1110 results about "Visual guidance" patented technology

Aspects of the present disclosure relate to systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using a head mounted display with systems, techniques and methods for improving the accuracy of an augmented reality display and / or for enhancing the 3D visualization of an augmented reality display.

The invention belongs to the technical field of mechanical arm automatic grabbing, and particularly relates to a 3D visual guidance mechanical arm grabbing method for irregular objects. Comprising the following steps that (1) a 3D visual guidance mechanical arm grabbing system is built, specifically, a 3D camera based on line structured light is adopted to guide a mechanical arm, the mechanical arm is a visual flexible six-axis mechanical arm, and after the mechanical arm completes accurate grabbing of all objects and carries out placement according to requirements, it is regarded that the grabbing task is successfully completed; and the camera in the 3D camera is calibrated, and the calibration of the camera is realized by calling a CalibrateCamera function in the OpenCV library. Through mutual cooperation of the 3D camera and the mechanical arm, intelligent grabbing of irregular objects such as industrial parts is achieved, more accurate grabbing operation is carried out according to the actual application situation, and the robot does not depend on training of three-dimensional point cloud samples. And the 3D camera can accurately obtain external information, and the grabbing requirement of the mechanical arm is met.

The invention discloses a grabbing control system for robot visual guidance, and belongs to the technical field of grabbing control systems.The grabbing control system comprises a visual perception module used for collecting workpiece images, extracting workpiece feature information and calculating accurate coordinates of workpieces in space; the robot main body module is used for receiving a system instruction to drive a carrier to move to a target position; the grabbing execution module is used for being in contact with the workpiece, grabbing the workpiece stably and transferring the workpiece to a designated position to be placed without damage; the system integration module is used for guaranteeing the safety, operability and stability of the feeding and discharging process; according to the system, workpiece information can be accurately collected, the posture of the grabbing execution end is analyzed based on the workpiece position, the grabbing posture is corrected in real time according to the grabbing dynamic state, and the workpiece grabbing precision and stability are improved.

The invention discloses a 500kV standardized switching operation method based on a VR virtual reality technology, and aims to solve the problems of unreal environment, incomplete evaluation and low efficiency in the switching operation training of an existing 500kV transformer substation. The method comprises the following steps: constructing a full-immersive virtual reality scene, collecting actual environment data of a transformer substation, and constructing and optimizing a high-precision model; designing multi-modal interaction of the standardized switching operation process, wherein the multi-modal interaction comprises visual guidance, a multi-modal interaction interface and an intelligent sensing function; dynamic action scoring and intelligent behavior evaluation are realized, actions are captured in real time by using high-precision action capture equipment, an instant score is generated by comparing with a standard, and dynamic feedback is provided; and real-time optimization and learning migration of the virtual character model are realized, user operation habits are captured, virtual character actions are optimized, and multi-scene migration and adaptation are realized. The method is suitable for VR-based 500kV standardized switching operation work, and the training authenticity and efficiency are improved.

The invention relates to an unmanned aerial vehicle visual guidance precise landing method, and belongs to the technical field of machine vision and unmanned aerial vehicle control. The method comprises the following steps: capturing surrounding environment information through a camera of an unmanned aerial vehicle to obtain operation environment data, and constructing a flight control model through construction parameters of the unmanned aerial vehicle; setting an expected landing point, extracting a target image from the corresponding operation environment data according to the expected landing point, and establishing visual mapping between the unmanned aerial vehicle and the expected landing point based on the target image; the relative position data and the to-be-flied distance of the unmanned aerial vehicle are calculated according to the visual mapping, and a predicted moving trajectory based on the relative position data and the to-be-flied distance is obtained through a flight control model; and an expected landing point is automatically captured and tracked in the landing process, relative pose information between the unmanned aerial vehicle and the expected landing point is solved in real time, and flight parameters of the unmanned aerial vehicle are adjusted in real time according to the relative pose information and the predicted moving trajectory. Visual guidance precise landing of the unmanned aerial vehicle is realized.

The invention discloses a printed circuit board defect detection method and system based on cross-modal prompt learning and visual guidance, and the method comprises the following steps: obtaining a defect-free PCB image sample, and obtaining a training data set; constructing an initial printed circuit board detection model; training the initial printed circuit board detection model by using the training data set to obtain a printed circuit board detection model; and completing the defect detection of the printed circuit board by using the printed circuit board detection model. According to the method, a synthetic anomaly generation strategy based on a PCB structure and physical prior is provided, a two-channel storage library mechanism is introduced, and a synthetic anomaly sample most valuable for model training is screened out through a difficult case mining strategy, so that visual anomaly mode guidance is provided. According to the method, a double-constraint alignment optimization strategy combining comparison loss and triple loss is adopted, the mapping relation between vision and text features is adjusted, and cross-modal alignment and clear normal / abnormal boundaries are achieved. According to the method provided by the invention, the accuracy, the robustness and the generalization capability of PCB defect detection under the data sparse condition can be remarkably improved.

The invention relates to a tunnel illumination control method considering visual adaptability of a driver, and belongs to the technical field of tunnel illumination control, and the method comprises the following steps: S1, employing a volume scattering point location method to obtain visual adaptability parameters of visual adaptation states of all segments in a tunnel; s2, dynamically constructing a predictive visual adaptation model of the driver group based on the visual adaptation parameters, and outputting expected visual states of the drivers according to the predictive visual adaptation model; s3, according to the expected visual state, a parameter control mechanism is adopted to control target illumination parameters of all the sections of the tunnel; s4, according to the target lighting parameters, the working state of lighting equipment in a corresponding section in the tunnel is regulated and controlled, so that the light environment in the tunnel is matched with the dynamic visual adaptability of a driver, and safe visual guidance is achieved; the method has the beneficial effects that the target illumination parameters of all the sections of the tunnel are controlled by adopting a parameter control mechanism according to the expected visual state according to the target illumination parameters.

The invention discloses an enamel product and an intelligent spraying system thereof, and belongs to the technical field of enamel product spraying, and the enamel product comprises a high-precision visual guidance module, an environment intelligent control module and an auxiliary function module, the intelligent environment control module is used for monitoring and dynamically regulating and controlling parameters such as temperature and humidity and dust concentration of a spraying environment in real time, ensuring stable coating quality and reducing energy consumption, and the auxiliary function module is used for realizing accurate spraying track planning and quality control through process connection, safety guarantee and data collaboration. And high efficiency and reliability of the spraying process are guaranteed. On the basis that spraying of the enamel product is achieved, the quality of the whole process is improved, and comprehensive benefits can be optimized.

The invention provides a dexterous hand based on multi-modal fusion control, and relates to the technical field of automatic control, the dexterous hand comprises a palm module, a palm center vision sensing module, a finger module, a fingertip force sensing module and a multi-modal control module; the palm center visual sensing module is used for acquiring visual information of the dexterous hand in a working area; the finger module comprises at least two fingers; the fingertip force sensing module is used for detecting information of contact force applied to the fingertip position; and the multi-mode control module is used for controlling the dexterous hand to grab a target object based on the visual information and the contact force information. According to the dexterous hand based on multi-mode fusion control, data of two modes of visual information and contact force information are fused, so that the dexterous hand can adapt to different grabbing environments, intelligent closed-loop control of visual guidance and force sense regulation and control is realized, and the environmental adaptability and flexibility of the dexterous hand are improved.

The invention provides a movable flexible docking platform, method and equipment based on visual guidance and a storage medium, and relates to the field of automatic auxiliary assembly. The docking platform comprises an AGV trolley, a six-degree-of-freedom platform erected in the middle of the upper portion of the AGV trolley, and a binocular vision system erected on the rear side of the upper portion of the AGV trolley. The binocular vision system provides real-time vision guidance in the advancing process of the AGV; a butt joint workpiece A is placed on a six-degree-of-freedom platform, and a butt joint workpiece B which has an assembly relation with the butt joint workpiece A is placed on a working table at a target position; and the AGV transports the butt joint workpiece A to the target position, and butt joint of the butt joint workpiece A and the butt joint workpiece B is completed. According to the invention, the six-degree-of-freedom parallel platform is arranged on the AGV trolley, so that the pose fine adjustment of the assembled workpiece is realized, the pose of the workpiece can be continuously adjusted in the transportation process to keep balance, and the workpiece transportation and high-precision butt joint can be realized.

The invention belongs to the technical field of medical image three-dimensional reconstruction, and discloses an abdominal cavity reconstruction and focus positioning method based on a binocular endoscope, which comprises the following steps of: calibrating and correcting internal reference of the binocular endoscope; inputting the trained depth estimation network to obtain a disparity map and processing the disparity map into a picture according to frames; the luminosity error is minimized through a coarse-to-fine pyramid optimization method, and a transformation matrix of the endoscope pose between two continuous frames is calculated through nonlinear iterative optimization; light beam adjustment optimization is carried out on the motion trail of the endoscope, a depth map is superposed, outliers are removed, and three-dimensional reconstruction of the abdominal cavity tissue is achieved; recognizing a focus, extracting three-dimensional information according to the disparity map, and visually guiding the operation arm to move to a focus point in combination with a calibrated hand-eye matrix so as to realize positioning. According to the method, rapid and dense three-dimensional reconstruction of the peritoneal environment before the operation and accurate focus positioning based on visual guidance during the operation are realized, the accuracy of motion track estimation of the endoscope is improved, and autonomous development of the surgical robot is expected to be promoted.

Devices and methods for performing an interventional vascular procedure with visual guidance using one or more optical head mounted displays are disclosed. Devices and methods for compensating the display of an optical head mounted display for cardiac and / or respiratory motion are disclosed.

The invention relates to the technical field of electromyographic signal monitoring, and discloses a guided monitoring method and system based on a wireless surface electromyographic analysis feedback instrument. The method comprises the following steps: acquiring electromyographic signal time sequence data and limb movement track space coordinate information when a target muscle group moves through a multi-channel wireless surface electromyographic sensor; muscle activation intensity and fatigue change features are extracted, a joint activity angle change sequence is calculated, and a muscle-joint collaborative activity feature matrix is generated through space-time alignment fusion; analyzing signal transduction delay and amplitude matching degree in the matrix, and evaluating the nerve driving coordination level; according to the deviation from the preset standard mode, generating a real-time biological feedback signal containing an action correction instruction and a muscle activation intensity adjustment parameter; and converting into a tactile vibration mode and a visual guidance mark to be output. According to the method, synchronous acquisition and fusion analysis of multi-dimensional motion information are realized, and visual guidance is provided through multi-modal feedback.

The invention relates to the technical field of robot visual guidance, in particular to an out-of-order target recognition and positioning method based on binocular laser three-dimensional scanning imaging, which comprises the following steps of: performing laser scanning on an object in a target area, and synchronously acquiring laser stripe images by adopting a binocular camera to obtain a target area image sequence; extracting a laser stripe center line of the target area image sequence to obtain auxiliary features; performing three-dimensional point cloud reconstruction on the auxiliary features to obtain original point cloud data; compensating the original point cloud data to obtain fused point cloud data; on the basis of a spherical multi-view sampling and positioning algorithm framework, off-line model point cloud features are constructed, and object recognition and pose matching are carried out on fused point cloud data; and visualizing an object identification and pose matching result. The method has the characteristics of stable identification and positioning results, high applicability in complex scenes, high identification accuracy and the like.

A non-transitory machine-readable media may store instructions that, when run by one or more processors, cause the one or more processors to generate stereo endoscopic image data of a medical field and define a medical field volume from the stereo endoscopic image data. The processors may also project a 3D representation of the medical field volume to an extended-reality display device in a remote volume. The 3D representation may include a stereoscopic image and a 3D scene generated from the stereo endoscopic image data. The processors may also generate visual guidance in the remote volume to augment the stereo endoscopic image, map the visual guidance from the remote volume to the medical field volume to generate an augmented image of the medical field volume, and project the augmented image of the medical field volume to a display device viewed by an operator of instruments in the medical field.

The invention provides a visual guidance type automatic disassembly method and system for a photovoltaic module, and relates to the technical field of module disassembly, and the method comprises the steps: obtaining real-time production data, constructing a multi-objective optimization decision model, and solving through an adaptive inertia weight particle swarm algorithm; verifying and dynamically optimizing the optimization scheme by adopting a deep reinforcement learning method in combination with a digital twinning environment; production line optimization control is realized and operation data is fed back to form a closed loop. According to the invention, the disassembly efficiency is improved, the energy consumption is reduced, the production quality is optimized, and intelligent automatic disassembly is realized.

Devices and methods for performing an interventional vascular procedure with visual guidance using one or more optical head mounted displays are disclosed. Devices and methods for compensating the display of an optical head mounted display for cardiac and / or respiratory motion are disclosed.

The invention discloses an unmanned aerial vehicle visual guidance obstacle avoidance system and method based on polarization information, and relates to the technical field of unmanned aerial vehicle navigation and obstacle avoidance. Polarization image information in a flight scene is collected in real time through a polarization sensor, and the perception capability of a complex environment is enhanced. Polarization parameters are extracted through a compression observation method, the efficiency and accuracy of feature extraction are improved, a visual attention mechanism is combined, an obstacle background area is accurately recognized, a self-adaptive decision fusion model is constructed, the feature weight is optimized, and the reliability of target contour features is enhanced. Preliminary path search is carried out based on target contour features, and a centralized optimization sampling method is adopted, so that the probability of generating an optimal flight path in a complex environment is effectively improved. And the obstacle avoidance flight module is used for smoothing the flight path, so that the smoothness and controllability of the flight path are ensured, and the navigation and obstacle avoidance capabilities of the unmanned aerial vehicle in a complex environment are effectively improved.

The invention belongs to the technical field of ablation puncture navigation, and particularly relates to an ablation puncture surgery navigation system under AI visual guidance. Through cooperative work of the phase monitoring module, the channel construction module, the needle moving control module, the displacement prediction module and the feedback control module, an intelligent surgical navigation system is constructed, breathing phase information and multi-modal medical image data of a patient can be obtained in real time, and therefore a safe and optimized needle moving path is accurately planned, and the operation accuracy is improved. Meanwhile, through real-time monitoring and quantitative evaluation of parameters such as tissue deformation and pressure distribution in the needle moving process, potential operation risks can be found and corrected in time, smooth operation is ensured, in addition, the system can dynamically adjust the needle moving path and the puncture angle according to the real-time breathing state of a patient, and the operation safety is improved. The accuracy and the safety of the operation are further improved, so that intelligent management and personalized treatment in the ablation puncture operation process are realized.

The invention belongs to the technical field of steel structure damage repair. The invention provides a large steel structure corrosion damage integrated repairing robot and a control method. The robot comprises a robot body, and a force control servo motor, an automatic polishing module, a damage identification camera, an air compression pump, a paint storage box, a control terminal, a magnetic adsorption module, a pipeline climbing module, a paint film thickness measuring module and an automatic paint spraying module which are arranged on the robot body. A robot compliance control strategy integrating force feedback, visual guidance and other modes is provided, dynamic parameter adjustment is combined, efficient and compliance control over the rust removal and spraying process of the repairing robot is achieved, the high efficiency and stability of the corrosion damage repairing process are ensured, and the corrosion damage repairing efficiency is improved. And flexible movement and efficient operation of the integrated repairing robot in a complex space structure are achieved.

The invention discloses a towel rack laser welding control method and welding device based on AI visual guidance, and relates to the field of laser welding. Time-space synchronization constraint conditions of the visual six-axis module and the welding six-axis module are established based on the welding seam point set, and a double-station alternating path is generated in combination with a path planning algorithm; the double-station alternating path is converted into a control instruction to be issued to a visual six-axis module and a welding six-axis module, and the welding six-axis module executes a welding task according to the control instruction; and laser welding characteristics are obtained in real time when the welding six-axis module executes a welding task, and a composite control model is constructed based on the laser welding characteristics so as to dynamically adjust welding parameters. According to the method, collaborative path generation, collision pre-judgment and local path optimization between the double modules are achieved, and the intelligent collaborative efficiency of the welding task, the performability of path planning and the stability of the welding process are effectively improved.

The invention belongs to the technical field of machine vision positioning and guiding, and discloses a nanocrystalline magnetism isolating sheet vision guiding laser deburring device and application thereof, and the nanocrystalline magnetism isolating sheet vision guiding laser deburring device comprises an image acquisition module, an image preprocessing module, a central control module, a laser module, a positioning module, a collection module, a data analysis module and a display module; according to the visual guidance technology, an advanced visual recognition system is introduced, the burr position is accurately positioned by monitoring the surface state of a workpiece in real time, and therefore the more intelligent laser deburring process is achieved. And laser parameter optimization: parameters such as wavelength, power and pulse frequency of laser are optimized according to different materials and burr types, so that the optimal deburring effect is achieved, and meanwhile, the completeness of the workpiece is ensured. And in the laser deburring process, environment-friendly materials and technologies are adopted, the influence on the environment is reduced, and the safety in the operation process is enhanced.

The invention discloses a tomato stem picking point attitude estimation method, and relates to a computer vision processing technology. The method comprises the following steps: detecting a long-range directional frame target, acquiring a color flow by a depth camera, and detecting position information and shielding conditions of fruits and fruit stems through a directional frame; visual guidance: matching fruit stems based on tomato growth constraint conditions, and visually guiding the mobile platform and the mechanical arm to move in a coupling manner until the fruit stems are observed when the fruit stems are not matched; detecting close-range key points, regulating and controlling a mechanical arm to a close-range observation position, identifying two points of a main stem and three points of a fruit stem through a key point detection model, and extracting point clouds; performing point cloud data processing, removing point cloud noise through point cloud filtering, and performing adaptive compensation on key points with depth anomaly by using a neighborhood mean value; calculating picking poses, and establishing the picking poses of the fruit stem shear points according to the three-dimensional information of the five key points; according to the method, the problem of tomato shielding in a greenhouse complex scene can be effectively solved, and the fruit stem shearing point pose is obtained.

The invention relates to a text knowledge enhanced scene graph generation method, and belongs to the field of computational vision. The method comprises the following steps: extracting text features of text information; extracting visual features and entity labels of the scene image, and combining the extracted visual features into paired entity pair features; introducing entity pair feature guidance to generate visual guidance text feature representation, so that the extracted text features are associated with visual entities; obtaining refined entity enhancement features and entity-to-visual predicate features based on the visual features and the entity tags; performing cross-modal alignment on the visual guidance text features and the entity pair visual predicate features to obtain graphical multi-modal feature representation; and inputting the multi-modal feature representation and the entity enhancement feature into a multi-layer perceptron for prediction to obtain prediction output of the entity pair and the entity pair relationship. According to the invention, the accuracy of entity object prediction and entity relationship prediction in the scene graph generation process can be improved.

The invention provides a circuit board component accurate positioning and mounting method and system based on visual guidance, and relates to the technical field of mounting, and the method comprises the steps: obtaining the multi-angle image information of a to-be-mounted component, determining a three-dimensional point cloud model and a depth map sequence, and determining the type information; segmenting the point cloud model through a region growing algorithm to extract features for pose estimation; the mounting parameters are called and combined with the posture deviation information to control a mechanical arm; and motion parameters are optimized based on a self-adaptive optical flow algorithm until the precision requirement is met. According to the invention, the component mounting precision and efficiency are improved, and the mounting failure rate is reduced.

The invention discloses an anesthesia nerve precise positioning method and system based on ultrasonic image auxiliary processing, and the method comprises the following steps: obtaining a real-time dynamic ultrasonic image sequence of a target region through an ultrasonic image data collection device, carrying out the multi-modal data fusion processing, and generating an enhanced ultrasonic image data set; performing three-dimensional semantic segmentation on the enhanced ultrasonic image data set based on a preset anesthesia nerve region segmentation model, and extracting morphological characteristics and spatial distribution information of a target nerve tissue; generating a three-dimensional navigation path grid based on the extracted morphological characteristics and spatial distribution information, and constructing a real-time visual guide interface; and dynamically optimizing a navigation path and a risk early warning strategy based on real-time operation feedback. The method has the following advantages and effects: a precise positioning method capable of fusing multi-scale neural features and dynamically optimizing a navigation path is provided, so that the key technical bottleneck of nerve ending visualization and real-time obstacle avoidance is solved.

The invention discloses a modular visual guidance robot feeding and discharging device, and relates to the technical field of robot-assisted feeding and discharging equipment, and the modular visual guidance robot feeding and discharging device comprises a visual module which is installed on a production line and is used for identifying the position of a workpiece and outputting coordinate information; the manipulator module is mounted on the production line, and a clamp connecting mechanism and an end effector are arranged at the tail end of the manipulator module; the clamp module is installed at the tail end of the mechanical arm module through a clamp connecting mechanism, and the clamp module is electrically connected with the tail end executor and used for grabbing a workpiece; and the control system module is electrically connected with the vision module, the manipulator module and the end effector and is used for controlling the manipulator module to act according to the coordinate information output by the vision module and controlling the clamp module to act through the end effector. The production requirements of diversified products on different production lines can be met, and the universality and adaptability of equipment are improved.

The invention discloses an apple recognition positioning and obstacle avoidance method based on zero samples and visual guidance. An improved YOLOV8 model is used for recognizing unknown category objects in a picking scene; semantic embedding of apples and branches is constructed by combining an external knowledge base Word2Vec, and unknown category objects in a picking scene are identified by using a zero sample learning method; constructing three-dimensional point cloud data of a fruit tree where an identification result is located, carrying out iterative convergence point cloud denoising by adopting a convex set constrained by an alpha norm, and updating point cloud coordinates; performing point cloud segmentation by using a potential energy function segmentation model; obtaining three-dimensional information of the apple and the obstacle through three-dimensional reconstruction of the point cloud feature information; extracting three-dimensional space position data of the branches at different time points to form a time sequence data set, and predicting the motion trail of the branches by using LSTM (Long Short Term Memory); through branch coordinate data of the point cloud and the LSTM, predicting a motion track of the branch, and utilizing a geometric collision detection method to evaluate and predict a collision risk between a robot end effector and the branch; and planning an optimal obstacle avoidance path by using an A * algorithm.

The invention relates to a method for clamping a cabin section A to complete butt-joint assembly with a cabin section B by a visual guidance robot. The mode that six degrees of freedom x, y, z, rx, ry and rz are measured and matched separately is adopted, measurement and matching of a circumferential rolling pin groove angle rx are completed through monocular vision, fixed alignment and matching of a pitching angle ry and a yaw angle rz are completed through a clamping tool and a fixing workpiece, distance adjustment in the y direction and the z direction in a butt joint plane is completed through monocular vision, and matching is completed. Adjustment of the distance in the x direction is completed through teaching to complete matching, finally six-degree-of-freedom adjustment and matching of the butt joint cabin sections A and B are achieved, and efficient and high-precision automatic assembly of the small-caliber and thin-wall cabin sections A and B is achieved.

The invention relates to the technical field of artificial intelligence, and discloses a vision-guided automatic sorting and deviation rectifying method and system.The vision-guided automatic sorting and deviation rectifying system comprises a vision parameter processing module, a sorting strategy decision module and a deviation rectifying execution control module, a real-time judgment mechanism of dynamic and static resolution differences is established by setting a resolution collaborative correction end, and the deviation rectifying execution control module is established; a self-adaptive interpolation algorithm is adopted to adjust visual feature data, so that the system can eliminate imaging resolution deviation caused by ambient light and transmission vibration, the acquisition accuracy of material surface texture features is ensured, sorting identification errors caused by projection chromatic aberration are avoided, the environmental adaptability of the visual guidance system is improved, and the visual guidance system is more accurate in image quality. The position and posture coupling analysis end is arranged, the dynamic matching degree of the position and posture offset of the materials and the motion trail of the mechanical arm is monitored in real time, the grabbing angle offset is automatically corrected through a space coordinate compensation mechanism, and it is guaranteed that the relative space position of an end effector and the materials is in the optimal grabbing state all the time.