9 results about "Active vision" patented technology

The application discloses a fruit autonomous picking method and system based on active perception and visual arm cooperative optimization, and relates to the technical field of robot control. The method comprises the following steps: acquiring a color image and a depth image of a working environment, and performing semantic segmentation on the color image by using a preset zero-sample learning model; performing semantic point cloud extraction on the depth image based on a mask corresponding to a target category, and constructing a fruit tree environment map containing spatial information and semantic information; performing candidate target screening and extraction according to the fruit tree environment map, and generating a candidate grasping pose according to the screened candidate target; performing kinematic reachability analysis on all candidate targets and candidate grasping poses, and calculating a joint score; determining a next grasping object according to the joint score; and picking the next grasping object according to a preset trajectory. The application deeply couples active visual perception and mechanical arm grasping action, and realizes efficient fruit autonomous picking of the robot.

This invention relates to the field of metallurgical equipment condition monitoring, and discloses a method and apparatus for detecting and judging cracks in the refractory layer of a full-size molten iron ladle before it receives hot iron. The method includes: acquiring an image of the inner wall of the molten iron ladle; performing a first-stage processing on the image to quickly locate one or more suspected crack areas using a convolutional neural network; performing a second-stage processing on each suspected crack area by actively projecting an optical pattern and analyzing its geometric deformation to calculate the three-dimensional physical parameters of the crack, such as its physical width and depth; and comparing the calculated physical parameters with preset safety standards to automatically judge the molten iron ladle. This invention, through a two-stage processing strategy, combines the rapid recognition capability of deep learning with the precise quantification capability of active visual three-dimensional measurement, achieving accurate acquisition of the physical dimensions of cracks. This provides reliable data support for automated and objective ladle judgment decisions, significantly improving the accuracy and reliability of detection.

The application provides a welding defect detection method and a robot welding defect detection system, the method comprising the steps of: acquiring a structured light image of a weld surface structure by shooting, processing the structured light image, extracting defects, and identifying and classifying the defects. The robot welding defect detection system comprises a visual detection device and a computer device for processing data of images collected by the visual detection device to determine welding defects. The visual detection device comprises an active visual detection device, which comprises a laser emitter for emitting structured light to a weld and a first camera for acquiring a structured light image of the weld by high-frequency shooting. The computer device is connected to the first camera and configured to process the structured light image of the weld acquired by the first camera through high-frequency shooting to identify defects. In the application, the structured light image of the weld surface is shot and processed to detect various defects such as surface concave-convex, cracks and pores.

This invention relates to the field of computer vision technology, and more particularly to a cross-modal image retrieval method based on a multimodal large language model (VLM) agent. The method includes: S1, initial candidate retrieval using the CLIP model; S2, attribute-structured decomposition and difficulty determination of the query text; S3, identity clustering of candidate images by actively calling DBSCAN clustering, and attribute inference verification of the clustered multi-view images by calling the multimodal large language model; S4, resolving ambiguity by using fine-grained location scoring or direct visual inspection when competing candidates exist; and S5, outputting structured retrieval results. This invention compresses redundant candidate space by introducing DBSCAN unsupervised clustering technology, enhances the inference confidence of the VLM through multi-view features, and accurately distinguishes between multiple highly similar candidates through fine-grained location scoring and active visual inspection.

The application belongs to the technical field of unmanned aerial vehicle cooperative positioning, and provides a kind of unmanned aerial vehicle cluster distributed cooperative positioning method based on active vision, including distributed state propagation and cross covariance maintenance, active relative observation target selection, active relative observation target selection proposed in the application, so that each unmanned aerial vehicle can autonomously decide optimal observation target based on local information, through the introduction of flight safety constraints, while ensuring flight safety, the observation information gain is maximized, unlike the passive evaluation mechanism of traditional single machine navigation error boundary calculation and alarm, the application can not only perceive the uncertainty of relative measurement online through active vision mechanism and ERMU effectiveness check, but also actively plan maneuvering observation to obtain high-quality observation data, realize active inhibition of cumulative drift and high-precision relative state estimation in GNSS-free environment.

The application discloses a kind of based on active vision's round tube class curved surface weld reinforcement measurement method, belong to welding quality detection technical field.The method includes: by line laser scanning obtains workpiece point cloud and reconstructs three-dimensional model;After pre-processing to model, it is converted into the point cloud to be measured;Point cloud is input into the trained weld area extraction model CSWeldNet to obtain segmentation result, the model is with PointNet++ as backbone, fused fast point feature histogram FPFH descriptor, and integrated geometry-semantic interaction GSI module and maximum-average pooling aggregation MAA module;Based on segmentation result, weld point cloud is projected to the plane perpendicular to workpiece axis and the azimuth angle is calculated, according to target azimuth angle, extract slice point cloud, fit local base material reference line in corresponding slice plane and calculate reinforcement.The application effectively solves the problems of missing reference surface, cross-section distortion and segmentation difficulty in round tube curved surface weld measurement, realizes the automatic, high-precision and robust measurement of weld reinforcement.

A multimodal active vision-based apple detection method employs a six-DOF robotic arm equipped with an RGB-D camera to simultaneously acquire RGB, depth, and infrared images and generate point cloud data. Preprocessing is performed on both the images and the point clouds. A three-way parallel YOLOv11 and PointNet multimodal feature extraction network is constructed to extract multi-scale features and 3D global geometric features from the images. Multimodal feature differentiation enhancement is achieved through a hierarchical selective feature fusion (SFF) module, followed by PAN-FPN cross-scale enhancement, and the detection head outputs recognition and segmentation results. Occlusion is perceived based on a semantic octree, and a single-fruit 3D bounding box is obtained through OPTICS clustering. The occluded target's viewing angle is sampled within the robotic arm's working space, and the optimal pose is selected via ray tracing, driving the robotic arm to actively perform supplementary measurements, forming a closed-loop iterative optimization. This invention effectively suppresses interference from foliage occlusion, improves the accuracy, completeness, and robustness of apple detection in complex orchard environments, and achieves automated, occlusion-free active visual detection across the entire domain.

This application provides a laser welding quality control method and system based on active vision sensing. The method includes: preprocessing the original weld seam image sequence by using median filtering to remove welding spatter and uneven illumination interference, resulting in a denoised weld seam image sequence; extracting features from the denoised weld seam image sequence by using an edge detection algorithm to identify the laser beam centerline and weld seam edge line, resulting in feature line pairs; calculating the deviation through the feature line pairs, quantizing the lateral and longitudinal offsets using a geometric transformation method, resulting in a two-dimensional deviation vector; performing spatial transformation based on the three-dimensional deviation parameters by using homogeneous coordinate transformation to map the vision sensor coordinate system to the robot motion coordinate system, resulting in robot compensation commands; adjusting the welding torch posture and position using the robot compensation commands, and using Kalman filtering to fuse real-time feedback data to determine the final trajectory correction path.