18 results about "Active vision" patented technology

The invention discloses a robot grabbing system based on visual perception, and relates to the field of robot grabbing systems, and the robot grabbing system comprises a multi-mode active data perception module, a scene dynamic pre-judgment and decision module, a flexible self-adaptive grabbing execution module and a cross-module cooperative control module. According to the robot grabbing system based on visual perception, the multi-mode active visual perception module effectively solves the problem of inaccurate perception in backlight, shielding, transparent and reflective target scenes through active view angle adjustment and a multi-mode image fusion technology, the target recognition accuracy can be effectively improved, and the robot grabbing system based on visual perception has the advantages that the robot grabbing efficiency is improved. The system can adapt to various complex industrial and life scenes, the cross-module cooperative control module reduces the response delay of the system to a millisecond level through high-precision time sequence synchronization and distributed data interaction, meanwhile, the system has an abnormal self-adaptive adjustment capability, the cooperative lagging and single-point fault risk of an existing linear architecture is avoided, and the system reliability is improved. And the operation stability and reliability of the system are improved.

This invention discloses a navigation-following unmanned surface vessel (USV) formation control method based on active vision, comprising the following steps: constructing a kinematic model of the USV based on a geodetic coordinate system; establishing a gimbal monocular camera model for the gimbal camera mounted on the USV; defining servo tracking errors for the navigation-following USV formation system by combining the USV kinematic model and the gimbal monocular camera model; defining image coordinate errors based on the viewing angle constraints of the gimbal monocular camera; designing a motion controller and adaptive parameter update law for the USV using the Lyapunov synthesis method; and designing a gimbal camera steering controller and adaptive parameter update law using the Lyapunov synthesis method. This method ensures the smooth implementation of visual servo control of the USV by defining servo tracking errors and image coordinate errors, and then incorporating these two errors into the Lyapunov synthesis method to design the USV motion controller, the gimbal camera steering controller, and the adaptive parameter update law.

The present application belongs to the technical field of intelligent welding, and discloses a multi-layer multi-pass weld tracking method and system based on active and passive vision. The system of the present application comprises an active and passive vision weld tracking sensor, a welding robot and an industrial computer. The active and passive vision weld tracking sensor comprises an active vision device, a passive vision device and a linear laser emitter. The method of the present application generates an original groove model by detecting groove information with a structured light. Then, the active and passive vision are combined to detect the weld surface shape and the weld corner point position during welding, and a groove model of the current weld is established. The groove model of the current weld is compared with the original groove model to determine whether the weld is offset and measure the offset, so as to determine the position of the next weld and plan and generate a preset welding gun posture, thereby avoiding problems caused by heat loss estimation failure, weld offset and poor welding gun posture.

The invention discloses an active visual guidance ton bag sealing and shaping method and device. The device comprises a visual acquisition module, double mechanical arms and a main control unit. According to the method, RGB and depth data of a ton bag sealing area are collected and preprocessed through the camera module, after inner bag sealing line extraction, operation point screening and feasibility inspection are completed through the detection module, the decision control module generates an outer bag adjusting / shaping instruction according to a detection result, and after coordinate mapping, the double mechanical arms are controlled to cooperatively execute operation. And an active visual closed loop of visual acquisition, detection positioning, decision control and operation execution is formed. According to the scheme, the problems of inaccurate inner bag positioning, difficult outer bag posture adjustment and easy collision in existing ton bag sealing and shaping are solved, the shaping precision and the operation stability are improved, and the device is suitable for automatic packaging production lines in the fields of chemical engineering, grains and the like.

The application discloses a fruit autonomous picking method and system based on active perception and visual arm cooperative optimization, and relates to the technical field of robot control. The method comprises the following steps: acquiring a color image and a depth image of a working environment, and performing semantic segmentation on the color image by using a preset zero-sample learning model; performing semantic point cloud extraction on the depth image based on a mask corresponding to a target category, and constructing a fruit tree environment map containing spatial information and semantic information; performing candidate target screening and extraction according to the fruit tree environment map, and generating a candidate grasping pose according to the screened candidate target; performing kinematic reachability analysis on all candidate targets and candidate grasping poses, and calculating a joint score; determining a next grasping object according to the joint score; and picking the next grasping object according to a preset trajectory. The application deeply couples active visual perception and mechanical arm grasping action, and realizes efficient fruit autonomous picking of the robot.

This invention relates to the field of metallurgical equipment condition monitoring, and discloses a method and apparatus for detecting and judging cracks in the refractory layer of a full-size molten iron ladle before it receives hot iron. The method includes: acquiring an image of the inner wall of the molten iron ladle; performing a first-stage processing on the image to quickly locate one or more suspected crack areas using a convolutional neural network; performing a second-stage processing on each suspected crack area by actively projecting an optical pattern and analyzing its geometric deformation to calculate the three-dimensional physical parameters of the crack, such as its physical width and depth; and comparing the calculated physical parameters with preset safety standards to automatically judge the molten iron ladle. This invention, through a two-stage processing strategy, combines the rapid recognition capability of deep learning with the precise quantification capability of active visual three-dimensional measurement, achieving accurate acquisition of the physical dimensions of cracks. This provides reliable data support for automated and objective ladle judgment decisions, significantly improving the accuracy and reliability of detection.

The application provides a welding defect detection method and a robot welding defect detection system, the method comprising the steps of: acquiring a structured light image of a weld surface structure by shooting, processing the structured light image, extracting defects, and identifying and classifying the defects. The robot welding defect detection system comprises a visual detection device and a computer device for processing data of images collected by the visual detection device to determine welding defects. The visual detection device comprises an active visual detection device, which comprises a laser emitter for emitting structured light to a weld and a first camera for acquiring a structured light image of the weld by high-frequency shooting. The computer device is connected to the first camera and configured to process the structured light image of the weld acquired by the first camera through high-frequency shooting to identify defects. In the application, the structured light image of the weld surface is shot and processed to detect various defects such as surface concave-convex, cracks and pores.

This invention relates to the field of computer vision technology, and more particularly to a cross-modal image retrieval method based on a multimodal large language model (VLM) agent. The method includes: S1, initial candidate retrieval using the CLIP model; S2, attribute-structured decomposition and difficulty determination of the query text; S3, identity clustering of candidate images by actively calling DBSCAN clustering, and attribute inference verification of the clustered multi-view images by calling the multimodal large language model; S4, resolving ambiguity by using fine-grained location scoring or direct visual inspection when competing candidates exist; and S5, outputting structured retrieval results. This invention compresses redundant candidate space by introducing DBSCAN unsupervised clustering technology, enhances the inference confidence of the VLM through multi-view features, and accurately distinguishes between multiple highly similar candidates through fine-grained location scoring and active visual inspection.

The application belongs to the technical field of unmanned aerial vehicle cooperative positioning, and provides a kind of unmanned aerial vehicle cluster distributed cooperative positioning method based on active vision, including distributed state propagation and cross covariance maintenance, active relative observation target selection, active relative observation target selection proposed in the application, so that each unmanned aerial vehicle can autonomously decide optimal observation target based on local information, through the introduction of flight safety constraints, while ensuring flight safety, the observation information gain is maximized, unlike the passive evaluation mechanism of traditional single machine navigation error boundary calculation and alarm, the application can not only perceive the uncertainty of relative measurement online through active vision mechanism and ERMU effectiveness check, but also actively plan maneuvering observation to obtain high-quality observation data, realize active inhibition of cumulative drift and high-precision relative state estimation in GNSS-free environment.

This invention proposes a mobile robot active recognition method and apparatus, belonging to the field of target recognition technology. The method includes: having the mobile robot acquire an RGB image and a depth image of its current position and orientation within a workspace containing a pre-defined target object; inputting the RGB image and depth image into a pre-defined robot active recognition network, which outputs the mobile robot's action at the next moment, including the direction and distance of the action; the mobile robot moving accordingly according to the network's output until it reaches the position closest to the pre-defined target object, at which point recognition is complete. This invention, based on reinforcement learning technology and combining depth and target object information, predicts the optimal field of view using an active vision algorithm. This overcomes the problem of insufficient target object information in the camera's field of view and significantly improves recognition reliability caused by target object occlusion or excessive distance.

An exemplary system and method are disclosed for providing visual communication between agents in a multi-agent system. In some implementations, the exemplary system and method are configured to (i) receive and decode a data packet or message visually received from other communication targets and (ii) visually transmit a reply data packet or message, in response to the communication targets, using a digital or electromechanical display.

The application discloses a kind of based on active vision's round tube class curved surface weld reinforcement measurement method, belong to welding quality detection technical field.The method includes: by line laser scanning obtains workpiece point cloud and reconstructs three-dimensional model;After pre-processing to model, it is converted into the point cloud to be measured;Point cloud is input into the trained weld area extraction model CSWeldNet to obtain segmentation result, the model is with PointNet++ as backbone, fused fast point feature histogram FPFH descriptor, and integrated geometry-semantic interaction GSI module and maximum-average pooling aggregation MAA module;Based on segmentation result, weld point cloud is projected to the plane perpendicular to workpiece axis and the azimuth angle is calculated, according to target azimuth angle, extract slice point cloud, fit local base material reference line in corresponding slice plane and calculate reinforcement.The application effectively solves the problems of missing reference surface, cross-section distortion and segmentation difficulty in round tube curved surface weld measurement, realizes the automatic, high-precision and robust measurement of weld reinforcement.

A multimodal active vision-based apple detection method employs a six-DOF robotic arm equipped with an RGB-D camera to simultaneously acquire RGB, depth, and infrared images and generate point cloud data. Preprocessing is performed on both the images and the point clouds. A three-way parallel YOLOv11 and PointNet multimodal feature extraction network is constructed to extract multi-scale features and 3D global geometric features from the images. Multimodal feature differentiation enhancement is achieved through a hierarchical selective feature fusion (SFF) module, followed by PAN-FPN cross-scale enhancement, and the detection head outputs recognition and segmentation results. Occlusion is perceived based on a semantic octree, and a single-fruit 3D bounding box is obtained through OPTICS clustering. The occluded target's viewing angle is sampled within the robotic arm's working space, and the optimal pose is selected via ray tracing, driving the robotic arm to actively perform supplementary measurements, forming a closed-loop iterative optimization. This invention effectively suppresses interference from foliage occlusion, improves the accuracy, completeness, and robustness of apple detection in complex orchard environments, and achieves automated, occlusion-free active visual detection across the entire domain.

This application provides an embedded active visual servo control method, system, and defect identification method, applied in the field of robotics. The steps are as follows: acquiring real-time image data of the manipulated object; obtaining the intensity values ​​of pixels in the real-time image based on the real-time image data; obtaining an error value based on the intensity values ​​of pixels in the real-time image and the intensity values ​​of pixels in a preset image; obtaining the input control quantities of the robot's joint actuators based on the error value, the pseudo-inverse matrix of the robotic arm's Jacobian matrix, and the gain factor. By extracting the intensity values ​​of pixels from the real-time image data of the manipulated object and using these intensity values ​​to obtain the input control quantities of the visual servo controller, the control of the robotic arm is achieved, reducing computation time and computational load. Furthermore, this process is implemented in an embedded system, further improving the real-time performance of the entire control. Simultaneously, based on the intensity values, defect identification of the manipulated object's surface is also achieved.

The application provides a kind of method and system for pipeline intelligent welding based on data decision, it is related to the technical field of intelligent pipeline welding, including in response to welding request, the groove image of the pipeline to be welded is acquired by active vision sensor, based on the groove image, the groove topographic feature parameter of the pipeline to be welded is determined, based on the groove topographic feature parameter, the corresponding basic layer arrangement rule and reference process parameter are determined from the preset layer arrangement database, and combined with the groove topographic feature parameter, the welding process scheme matched with the current groove is generated, based on the welding process scheme, the welding operation is executed in the groove of the pipeline to be welded, solve the technical problems of global planning ability loss, local lag of correction strategy and lack of precise control in the whole process in the prior art, with the effect of improving the stability and reliability of pipeline welding quality.

The invention discloses an unmanned aerial vehicle indoor and outdoor seamless connection system based on external active visual guidance and a cooperative control method, and solves the problems of hard light failure, coordinate switching jump and unreliable communication in the prior art. The system comprises an airborne subsystem and a ground guide subsystem, an airborne end is provided with a specific wavelength active cursor and a dual-mode receiving module, and a ground end comprises a visual capture unit with a narrow-band optical filter, a dual-mode transmitting unit and a central processing unit. According to the control method, through outdoor hovering, visual capturing and filtering, coordinate handshaking and dual-mode link grading guiding, stable positioning in a strong light environment, smooth coordinate switching and millimeter-level connection precision are achieved, and indoor and outdoor seamless safe connection of the unmanned aerial vehicle is guaranteed.

This application provides a laser welding quality control method and system based on active vision sensing. The method includes: preprocessing the original weld seam image sequence by using median filtering to remove welding spatter and uneven illumination interference, resulting in a denoised weld seam image sequence; extracting features from the denoised weld seam image sequence by using an edge detection algorithm to identify the laser beam centerline and weld seam edge line, resulting in feature line pairs; calculating the deviation through the feature line pairs, quantizing the lateral and longitudinal offsets using a geometric transformation method, resulting in a two-dimensional deviation vector; performing spatial transformation based on the three-dimensional deviation parameters by using homogeneous coordinate transformation to map the vision sensor coordinate system to the robot motion coordinate system, resulting in robot compensation commands; adjusting the welding torch posture and position using the robot compensation commands, and using Kalman filtering to fuse real-time feedback data to determine the final trajectory correction path.