961 results about "Visual guidance" patented technology

The invention discloses a grabbing control system for robot visual guidance, and belongs to the technical field of grabbing control systems.The grabbing control system comprises a visual perception module used for collecting workpiece images, extracting workpiece feature information and calculating accurate coordinates of workpieces in space; the robot main body module is used for receiving a system instruction to drive a carrier to move to a target position; the grabbing execution module is used for being in contact with the workpiece, grabbing the workpiece stably and transferring the workpiece to a designated position to be placed without damage; the system integration module is used for guaranteeing the safety, operability and stability of the feeding and discharging process; according to the system, workpiece information can be accurately collected, the posture of the grabbing execution end is analyzed based on the workpiece position, the grabbing posture is corrected in real time according to the grabbing dynamic state, and the workpiece grabbing precision and stability are improved.

The invention discloses a 500kV standardized switching operation method based on a VR virtual reality technology, and aims to solve the problems of unreal environment, incomplete evaluation and low efficiency in the switching operation training of an existing 500kV transformer substation. The method comprises the following steps: constructing a full-immersive virtual reality scene, collecting actual environment data of a transformer substation, and constructing and optimizing a high-precision model; designing multi-modal interaction of the standardized switching operation process, wherein the multi-modal interaction comprises visual guidance, a multi-modal interaction interface and an intelligent sensing function; dynamic action scoring and intelligent behavior evaluation are realized, actions are captured in real time by using high-precision action capture equipment, an instant score is generated by comparing with a standard, and dynamic feedback is provided; and real-time optimization and learning migration of the virtual character model are realized, user operation habits are captured, virtual character actions are optimized, and multi-scene migration and adaptation are realized. The method is suitable for VR-based 500kV standardized switching operation work, and the training authenticity and efficiency are improved.

The invention discloses a printed circuit board defect detection method and system based on cross-modal prompt learning and visual guidance, and the method comprises the following steps: obtaining a defect-free PCB image sample, and obtaining a training data set; constructing an initial printed circuit board detection model; training the initial printed circuit board detection model by using the training data set to obtain a printed circuit board detection model; and completing the defect detection of the printed circuit board by using the printed circuit board detection model. According to the method, a synthetic anomaly generation strategy based on a PCB structure and physical prior is provided, a two-channel storage library mechanism is introduced, and a synthetic anomaly sample most valuable for model training is screened out through a difficult case mining strategy, so that visual anomaly mode guidance is provided. According to the method, a double-constraint alignment optimization strategy combining comparison loss and triple loss is adopted, the mapping relation between vision and text features is adjusted, and cross-modal alignment and clear normal / abnormal boundaries are achieved. According to the method provided by the invention, the accuracy, the robustness and the generalization capability of PCB defect detection under the data sparse condition can be remarkably improved.

The invention relates to a tunnel illumination control method considering visual adaptability of a driver, and belongs to the technical field of tunnel illumination control, and the method comprises the following steps: S1, employing a volume scattering point location method to obtain visual adaptability parameters of visual adaptation states of all segments in a tunnel; s2, dynamically constructing a predictive visual adaptation model of the driver group based on the visual adaptation parameters, and outputting expected visual states of the drivers according to the predictive visual adaptation model; s3, according to the expected visual state, a parameter control mechanism is adopted to control target illumination parameters of all the sections of the tunnel; s4, according to the target lighting parameters, the working state of lighting equipment in a corresponding section in the tunnel is regulated and controlled, so that the light environment in the tunnel is matched with the dynamic visual adaptability of a driver, and safe visual guidance is achieved; the method has the beneficial effects that the target illumination parameters of all the sections of the tunnel are controlled by adopting a parameter control mechanism according to the expected visual state according to the target illumination parameters.

The invention discloses an enamel product and an intelligent spraying system thereof, and belongs to the technical field of enamel product spraying, and the enamel product comprises a high-precision visual guidance module, an environment intelligent control module and an auxiliary function module, the intelligent environment control module is used for monitoring and dynamically regulating and controlling parameters such as temperature and humidity and dust concentration of a spraying environment in real time, ensuring stable coating quality and reducing energy consumption, and the auxiliary function module is used for realizing accurate spraying track planning and quality control through process connection, safety guarantee and data collaboration. And high efficiency and reliability of the spraying process are guaranteed. On the basis that spraying of the enamel product is achieved, the quality of the whole process is improved, and comprehensive benefits can be optimized.

The invention provides a dexterous hand based on multi-modal fusion control, and relates to the technical field of automatic control, the dexterous hand comprises a palm module, a palm center vision sensing module, a finger module, a fingertip force sensing module and a multi-modal control module; the palm center visual sensing module is used for acquiring visual information of the dexterous hand in a working area; the finger module comprises at least two fingers; the fingertip force sensing module is used for detecting information of contact force applied to the fingertip position; and the multi-mode control module is used for controlling the dexterous hand to grab a target object based on the visual information and the contact force information. According to the dexterous hand based on multi-mode fusion control, data of two modes of visual information and contact force information are fused, so that the dexterous hand can adapt to different grabbing environments, intelligent closed-loop control of visual guidance and force sense regulation and control is realized, and the environmental adaptability and flexibility of the dexterous hand are improved.

Devices and methods for performing an interventional vascular procedure with visual guidance using one or more optical head mounted displays are disclosed. Devices and methods for compensating the display of an optical head mounted display for cardiac and / or respiratory motion are disclosed.

The invention relates to the technical field of electromyographic signal monitoring, and discloses a guided monitoring method and system based on a wireless surface electromyographic analysis feedback instrument. The method comprises the following steps: acquiring electromyographic signal time sequence data and limb movement track space coordinate information when a target muscle group moves through a multi-channel wireless surface electromyographic sensor; muscle activation intensity and fatigue change features are extracted, a joint activity angle change sequence is calculated, and a muscle-joint collaborative activity feature matrix is generated through space-time alignment fusion; analyzing signal transduction delay and amplitude matching degree in the matrix, and evaluating the nerve driving coordination level; according to the deviation from the preset standard mode, generating a real-time biological feedback signal containing an action correction instruction and a muscle activation intensity adjustment parameter; and converting into a tactile vibration mode and a visual guidance mark to be output. According to the method, synchronous acquisition and fusion analysis of multi-dimensional motion information are realized, and visual guidance is provided through multi-modal feedback.

The invention relates to the technical field of robot visual guidance, in particular to an out-of-order target recognition and positioning method based on binocular laser three-dimensional scanning imaging, which comprises the following steps of: performing laser scanning on an object in a target area, and synchronously acquiring laser stripe images by adopting a binocular camera to obtain a target area image sequence; extracting a laser stripe center line of the target area image sequence to obtain auxiliary features; performing three-dimensional point cloud reconstruction on the auxiliary features to obtain original point cloud data; compensating the original point cloud data to obtain fused point cloud data; on the basis of a spherical multi-view sampling and positioning algorithm framework, off-line model point cloud features are constructed, and object recognition and pose matching are carried out on fused point cloud data; and visualizing an object identification and pose matching result. The method has the characteristics of stable identification and positioning results, high applicability in complex scenes, high identification accuracy and the like.

A non-transitory machine-readable media may store instructions that, when run by one or more processors, cause the one or more processors to generate stereo endoscopic image data of a medical field and define a medical field volume from the stereo endoscopic image data. The processors may also project a 3D representation of the medical field volume to an extended-reality display device in a remote volume. The 3D representation may include a stereoscopic image and a 3D scene generated from the stereo endoscopic image data. The processors may also generate visual guidance in the remote volume to augment the stereo endoscopic image, map the visual guidance from the remote volume to the medical field volume to generate an augmented image of the medical field volume, and project the augmented image of the medical field volume to a display device viewed by an operator of instruments in the medical field.

The invention provides a visual guidance type automatic disassembly method and system for a photovoltaic module, and relates to the technical field of module disassembly, and the method comprises the steps: obtaining real-time production data, constructing a multi-objective optimization decision model, and solving through an adaptive inertia weight particle swarm algorithm; verifying and dynamically optimizing the optimization scheme by adopting a deep reinforcement learning method in combination with a digital twinning environment; production line optimization control is realized and operation data is fed back to form a closed loop. According to the invention, the disassembly efficiency is improved, the energy consumption is reduced, the production quality is optimized, and intelligent automatic disassembly is realized.

The invention belongs to the technical field of ablation puncture navigation, and particularly relates to an ablation puncture surgery navigation system under AI visual guidance. Through cooperative work of the phase monitoring module, the channel construction module, the needle moving control module, the displacement prediction module and the feedback control module, an intelligent surgical navigation system is constructed, breathing phase information and multi-modal medical image data of a patient can be obtained in real time, and therefore a safe and optimized needle moving path is accurately planned, and the operation accuracy is improved. Meanwhile, through real-time monitoring and quantitative evaluation of parameters such as tissue deformation and pressure distribution in the needle moving process, potential operation risks can be found and corrected in time, smooth operation is ensured, in addition, the system can dynamically adjust the needle moving path and the puncture angle according to the real-time breathing state of a patient, and the operation safety is improved. The accuracy and the safety of the operation are further improved, so that intelligent management and personalized treatment in the ablation puncture operation process are realized.

The invention discloses a towel rack laser welding control method and welding device based on AI visual guidance, and relates to the field of laser welding. Time-space synchronization constraint conditions of the visual six-axis module and the welding six-axis module are established based on the welding seam point set, and a double-station alternating path is generated in combination with a path planning algorithm; the double-station alternating path is converted into a control instruction to be issued to a visual six-axis module and a welding six-axis module, and the welding six-axis module executes a welding task according to the control instruction; and laser welding characteristics are obtained in real time when the welding six-axis module executes a welding task, and a composite control model is constructed based on the laser welding characteristics so as to dynamically adjust welding parameters. According to the method, collaborative path generation, collision pre-judgment and local path optimization between the double modules are achieved, and the intelligent collaborative efficiency of the welding task, the performability of path planning and the stability of the welding process are effectively improved.

The invention belongs to the technical field of machine vision positioning and guiding, and discloses a nanocrystalline magnetism isolating sheet vision guiding laser deburring device and application thereof, and the nanocrystalline magnetism isolating sheet vision guiding laser deburring device comprises an image acquisition module, an image preprocessing module, a central control module, a laser module, a positioning module, a collection module, a data analysis module and a display module; according to the visual guidance technology, an advanced visual recognition system is introduced, the burr position is accurately positioned by monitoring the surface state of a workpiece in real time, and therefore the more intelligent laser deburring process is achieved. And laser parameter optimization: parameters such as wavelength, power and pulse frequency of laser are optimized according to different materials and burr types, so that the optimal deburring effect is achieved, and meanwhile, the completeness of the workpiece is ensured. And in the laser deburring process, environment-friendly materials and technologies are adopted, the influence on the environment is reduced, and the safety in the operation process is enhanced.

The invention discloses a tomato stem picking point attitude estimation method, and relates to a computer vision processing technology. The method comprises the following steps: detecting a long-range directional frame target, acquiring a color flow by a depth camera, and detecting position information and shielding conditions of fruits and fruit stems through a directional frame; visual guidance: matching fruit stems based on tomato growth constraint conditions, and visually guiding the mobile platform and the mechanical arm to move in a coupling manner until the fruit stems are observed when the fruit stems are not matched; detecting close-range key points, regulating and controlling a mechanical arm to a close-range observation position, identifying two points of a main stem and three points of a fruit stem through a key point detection model, and extracting point clouds; performing point cloud data processing, removing point cloud noise through point cloud filtering, and performing adaptive compensation on key points with depth anomaly by using a neighborhood mean value; calculating picking poses, and establishing the picking poses of the fruit stem shear points according to the three-dimensional information of the five key points; according to the method, the problem of tomato shielding in a greenhouse complex scene can be effectively solved, and the fruit stem shearing point pose is obtained.

The invention provides a circuit board component accurate positioning and mounting method and system based on visual guidance, and relates to the technical field of mounting, and the method comprises the steps: obtaining the multi-angle image information of a to-be-mounted component, determining a three-dimensional point cloud model and a depth map sequence, and determining the type information; segmenting the point cloud model through a region growing algorithm to extract features for pose estimation; the mounting parameters are called and combined with the posture deviation information to control a mechanical arm; and motion parameters are optimized based on a self-adaptive optical flow algorithm until the precision requirement is met. According to the invention, the component mounting precision and efficiency are improved, and the mounting failure rate is reduced.

The invention discloses an anesthesia nerve precise positioning method and system based on ultrasonic image auxiliary processing, and the method comprises the following steps: obtaining a real-time dynamic ultrasonic image sequence of a target region through an ultrasonic image data collection device, carrying out the multi-modal data fusion processing, and generating an enhanced ultrasonic image data set; performing three-dimensional semantic segmentation on the enhanced ultrasonic image data set based on a preset anesthesia nerve region segmentation model, and extracting morphological characteristics and spatial distribution information of a target nerve tissue; generating a three-dimensional navigation path grid based on the extracted morphological characteristics and spatial distribution information, and constructing a real-time visual guide interface; and dynamically optimizing a navigation path and a risk early warning strategy based on real-time operation feedback. The method has the following advantages and effects: a precise positioning method capable of fusing multi-scale neural features and dynamically optimizing a navigation path is provided, so that the key technical bottleneck of nerve ending visualization and real-time obstacle avoidance is solved.

The invention discloses a modular visual guidance robot feeding and discharging device, and relates to the technical field of robot-assisted feeding and discharging equipment, and the modular visual guidance robot feeding and discharging device comprises a visual module which is installed on a production line and is used for identifying the position of a workpiece and outputting coordinate information; the manipulator module is mounted on the production line, and a clamp connecting mechanism and an end effector are arranged at the tail end of the manipulator module; the clamp module is installed at the tail end of the mechanical arm module through a clamp connecting mechanism, and the clamp module is electrically connected with the tail end executor and used for grabbing a workpiece; and the control system module is electrically connected with the vision module, the manipulator module and the end effector and is used for controlling the manipulator module to act according to the coordinate information output by the vision module and controlling the clamp module to act through the end effector. The production requirements of diversified products on different production lines can be met, and the universality and adaptability of equipment are improved.

The invention discloses an apple recognition positioning and obstacle avoidance method based on zero samples and visual guidance. An improved YOLOV8 model is used for recognizing unknown category objects in a picking scene; semantic embedding of apples and branches is constructed by combining an external knowledge base Word2Vec, and unknown category objects in a picking scene are identified by using a zero sample learning method; constructing three-dimensional point cloud data of a fruit tree where an identification result is located, carrying out iterative convergence point cloud denoising by adopting a convex set constrained by an alpha norm, and updating point cloud coordinates; performing point cloud segmentation by using a potential energy function segmentation model; obtaining three-dimensional information of the apple and the obstacle through three-dimensional reconstruction of the point cloud feature information; extracting three-dimensional space position data of the branches at different time points to form a time sequence data set, and predicting the motion trail of the branches by using LSTM (Long Short Term Memory); through branch coordinate data of the point cloud and the LSTM, predicting a motion track of the branch, and utilizing a geometric collision detection method to evaluate and predict a collision risk between a robot end effector and the branch; and planning an optimal obstacle avoidance path by using an A * algorithm.

The invention relates to the technical field of artificial intelligence, and discloses a vision-guided automatic sorting and deviation rectifying method and system.The vision-guided automatic sorting and deviation rectifying system comprises a vision parameter processing module, a sorting strategy decision module and a deviation rectifying execution control module, a real-time judgment mechanism of dynamic and static resolution differences is established by setting a resolution collaborative correction end, and the deviation rectifying execution control module is established; a self-adaptive interpolation algorithm is adopted to adjust visual feature data, so that the system can eliminate imaging resolution deviation caused by ambient light and transmission vibration, the acquisition accuracy of material surface texture features is ensured, sorting identification errors caused by projection chromatic aberration are avoided, the environmental adaptability of the visual guidance system is improved, and the visual guidance system is more accurate in image quality. The position and posture coupling analysis end is arranged, the dynamic matching degree of the position and posture offset of the materials and the motion trail of the mechanical arm is monitored in real time, the grabbing angle offset is automatically corrected through a space coordinate compensation mechanism, and it is guaranteed that the relative space position of an end effector and the materials is in the optimal grabbing state all the time.

The invention discloses a visual guidance positioning system and method in the machine tool flexible shaft assembling process, and relates to the technical field of flexible shaft assembling, a visual system obtains three-dimensional pose data of the tail end of a flexible shaft and a target connector, a six-dimensional force sensor is combined to monitor contact force / torque in real time, the limitation of a single sensor is broken through, and the blind area of the single sensor is solved; visual and force sense coordinate systems are aligned in real time, and coordinate drifting caused by movement of the mechanical arm is eliminated; generating an approximation path based on the pose difference, introducing a compensation amount formula, inhibiting oscillation in combination with an attenuation coefficient, and improving the path smoothness; when the contact force exceeds the dynamic threshold value or the pose difference converges, path correction is automatically triggered, overload damage is avoided, and the assembly damage rate is effectively reduced. The method breaks through the limitation of traditional static path planning, has high precision, high reliability and complex working condition adaptability, and is especially suitable for precise instrument and heavy load machinery assembly scenes.

The invention discloses a pitaya robot system based on intelligent treatment and recuperation and a control method, and the system comprises a high-precision image collection and human body image collection and three-dimensional reconstruction module which is used for precisely recognizing and positioning a treatment region and acupoints; the core physician manipulation digitization module is used for finely capturing professional treatment manipulations and standardized recuperating manipulations of deep physicians for operating pitayas through a multi-dimensional sensor array and forming a manipulation model library; the intelligent strategy generation module based on artificial intelligence can call, fuse or optimize and generate an optimal execution scheme from a corresponding model library according to a mode selected by a user; and the high-precision and compliant force control robot execution module reproduces the digitized technique of the pitaya in a high-fidelity manner under the real-time closed-loop control of visual guidance and force sense feedback. Through the double-track parallel strategy, quantification, inheritance and popularization of the high-level pitaya therapy are achieved, and application of the high-level pitaya therapy is pushed to the wide field.

The invention relates to a visual guidance robot automatic control method, system and device and a medium. The method comprises the following steps: firstly, acquiring a target image sequence and extracting initial visual features, and processing the initial visual features through a deep learning algorithm to output a high-resolution image; calculating a distance switching value based on the high-resolution image, and if the distance switching value exceeds a preset threshold value, performing scale distortion correction on the high-resolution image to obtain a target image; multiple frames of time sequence data are fused through a Kalman filtering algorithm, image distortion real-time compensation and feature point track stability judgment are synchronously completed, and a target positioning precision estimated value is output; and finally, according to the target positioning precision estimated value, determining the position of a grabbing point through a posture mapping rule of mapping preset grabbing feature points in a target coordinate system to a robot-based coordinate system, and based on the position of the grabbing point, optimizing and fusing track parameters through a Kalman filtering algorithm to generate adaptive track parameters. The method improves the positioning precision and track following effect of automatic control of the robot.

The invention discloses a mechanical arm autonomous grabbing system and method based on visual guidance and visual touch fusion. The mechanical arm autonomous grabbing system at least comprises a visual semantic guidance and grabbing pose generation module, a smooth grabbing control module based on visual touch fusion and a dynamic planning module based on model prediction path integration. The visual module efficiently generates a six-degree-of-freedom grabbing pose from the point cloud data by using a deep learning network; the smooth grabbing control module monitors object slippage in real time and adaptively adjusts clamping force by fusing visual and tactile information, and stable grabbing of fragile or slippery objects is achieved. And the dynamic planning module adopts a model prediction path integral algorithm MPPI to realize collision-free path planning of the mechanical arm in a complex dynamic environment. The problems that an existing mechanical arm is insufficient in sensing capacity, poor in grabbing robustness and low in path planning efficiency in an unstructured scene are solved, and the autonomous operation capacity and safety of the mechanical arm in the scenes of accompanying, intelligent sorting and the like are remarkably improved.

The invention discloses a position and posture compensation method for guiding installation of a contact network cantilever based on 3D vision and an automatic installation system. The position and posture compensation method comprises the steps that the spatial transformation relation between a 3D vision sensor coordinate system and a robot coordinate system is determined; the robot is controlled to move, and the three-dimensional point cloud data of the to-be-installed cantilever component and the three-dimensional point cloud data of the field installation foundation are collected; the collected three-dimensional point cloud data is registered with a corresponding preset CAD model, and a first real-time pose of the cantilever component in a robot coordinate system and a second real-time pose of the installation foundation in the robot coordinate system are obtained through calculation; calculating a pose compensation transformation quantity based on a pre-stored theoretical mounting pose of the cantilever, the first real-time pose and the second real-time pose; and according to the pose compensation transformation amount, planning and executing a motion track of the robot, and completing butt joint installation. The method is high in compensation precision, robustness and adaptability and high in construction efficiency, and the problem of uncertainty in automatic installation is fundamentally solved.

The invention relates to an underwater image restoration method based on multi-modal visual guidance and characteristic decomposition, which is used for solving the problems in the prior art that color distortion and contrast reduction of an image are caused by light absorption due to gradual attenuation of light with different wavelengths in water, and the light is scattered when encountering particles in the water, so that the image cannot be restored. And the problems of image blurring and detail loss are solved. According to the scheme, multi-mode prompt learning is adopted, complementarity among multiple modes is utilized, and features from different modes are integrated into prompt information so as to support color and contrast recovery of underwater images; and extracting high-frequency information from the RGB modal features by using learnable wavelets, and focusing on fine and difficult-to-perceive detail areas in the high-frequency features by using a high-frequency mixed attention module so as to recover the definition of textures in the underwater image.

The invention is suitable for the field of data processing, and discloses a visual guidance method and device for a storage location, terminal equipment and a storage medium. The visual guidance method for the storage location comprises the following steps: when an access instruction is detected, positioning a target storage location according to a product identification code carried by the access instruction; according to the target storage location, unique indicator light control parameters are generated, and the indicator light control parameters comprise color parameters, brightness parameters and / or flicker frequency; according to the indicator light control parameters, control operation of the LED lamp is executed, and visual guidance is provided for the target position where the target storage location is located. By generating a unique indicator lamp control parameter (including a color parameter, a brightness parameter and / or a flicker frequency) for each access instruction, each operator can clearly identify an LED lamp that guides himself / herself. The detection of the access instruction, the positioning of the target storage location and the generation of the indicator light control parameters make the storage process more efficient.

The invention discloses a robot bolt tightening method and system based on 3D visual guidance, and relates to the technical field of intelligent manufacturing, and the method comprises the following steps: S1, constructing a minimum photographing frequency model according to known camera and frame data, and solving an optimal photographing path; s2, acquiring an image and point cloud information of the frame; s3, inputting the image into an intelligent bolt detection model to obtain size and position data of the bolt; s4, bolt surface point cloud information is cut out according to the data of the bolt, and the posture and the specification size of the bolt are calculated through fitting of the bolt surface point cloud information; s5, calculating the transformation relation of the pose of the tightening gun under the image coordinate system and the frame coordinate system; and S6, the robot is guided to conduct tightening operation according to the transformation relation and the posture, specification and size of the bolt.

The invention discloses a 3D visual guidance steel box girder pushing and position adjusting method based on Beidou positioning. The method comprises the steps that a Beidou positioning device and an inclinometer are installed at a positioning datum point and a horizontal datum point of a steel box girder respectively; feature identifiers are installed on the box girder segments, and visual devices are correspondingly installed on the outer sides of the box girder segments and are used for monitoring three-dimensional displacement of the feature identifiers in the pushing process in real time; monitoring data of the Beidou positioning device, the inclinometer and the visual device are collected in real time, a dynamic three-dimensional model of the steel box girder is established, and the pushing posture and the pushing stroke of the steel box girder are adjusted on the basis of the dynamic three-dimensional model till the current box girder section reaches the designed mileage. According to the method, the dynamic characteristics of the steel box girder are perfected by taking the Beidou positioning data as the reference and utilizing the visual monitoring data and the inclination angle monitoring data, and the steel box girder pushing dynamic three-dimensional model is established, so that the pushing posture and the pushing stroke of the steel box girder can be automatically and accurately adjusted in the pushing process, and the efficiency and the safety of pushing construction are ensured.