792 results about "Visually guided" patented technology

The invention discloses a grabbing control system for robot visual guidance, and belongs to the technical field of grabbing control systems.The grabbing control system comprises a visual perception module used for collecting workpiece images, extracting workpiece feature information and calculating accurate coordinates of workpieces in space; the robot main body module is used for receiving a system instruction to drive a carrier to move to a target position; the grabbing execution module is used for being in contact with the workpiece, grabbing the workpiece stably and transferring the workpiece to a designated position to be placed without damage; the system integration module is used for guaranteeing the safety, operability and stability of the feeding and discharging process; according to the system, workpiece information can be accurately collected, the posture of the grabbing execution end is analyzed based on the workpiece position, the grabbing posture is corrected in real time according to the grabbing dynamic state, and the workpiece grabbing precision and stability are improved.

The invention discloses a printed circuit board defect detection method and system based on cross-modal prompt learning and visual guidance, and the method comprises the following steps: obtaining a defect-free PCB image sample, and obtaining a training data set; constructing an initial printed circuit board detection model; training the initial printed circuit board detection model by using the training data set to obtain a printed circuit board detection model; and completing the defect detection of the printed circuit board by using the printed circuit board detection model. According to the method, a synthetic anomaly generation strategy based on a PCB structure and physical prior is provided, a two-channel storage library mechanism is introduced, and a synthetic anomaly sample most valuable for model training is screened out through a difficult case mining strategy, so that visual anomaly mode guidance is provided. According to the method, a double-constraint alignment optimization strategy combining comparison loss and triple loss is adopted, the mapping relation between vision and text features is adjusted, and cross-modal alignment and clear normal / abnormal boundaries are achieved. According to the method provided by the invention, the accuracy, the robustness and the generalization capability of PCB defect detection under the data sparse condition can be remarkably improved.

Systems and methods are provided for facilitating an interactive artificial-reality activity. A method includes, after a user of a head-wearable device has opted-in to using an artificial-reality application to facilitate connecting with other participating users, determining, based on user-specific suggestion criteria, that a suggested user is located in an approved common space with the user and has opted-in to use the artificial-reality application. The method includes causing the head-wearable device to present an user interface (UI) element for linking the suggested user and the user. Upon the user selecting the UI element for linking the suggested user with the user, the method includes automatically causing the head-wearable device to provide visual-guidance UI elements to navigate the user to an interactive-activity location where the user and the suggested user will perform an artificial-reality activity while also displaying information about the artificial-reality activity to be performed.

The invention relates to a tunnel illumination control method considering visual adaptability of a driver, and belongs to the technical field of tunnel illumination control, and the method comprises the following steps: S1, employing a volume scattering point location method to obtain visual adaptability parameters of visual adaptation states of all segments in a tunnel; s2, dynamically constructing a predictive visual adaptation model of the driver group based on the visual adaptation parameters, and outputting expected visual states of the drivers according to the predictive visual adaptation model; s3, according to the expected visual state, a parameter control mechanism is adopted to control target illumination parameters of all the sections of the tunnel; s4, according to the target lighting parameters, the working state of lighting equipment in a corresponding section in the tunnel is regulated and controlled, so that the light environment in the tunnel is matched with the dynamic visual adaptability of a driver, and safe visual guidance is achieved; the method has the beneficial effects that the target illumination parameters of all the sections of the tunnel are controlled by adopting a parameter control mechanism according to the expected visual state according to the target illumination parameters.

The invention relates to the technical field of image analysis, in particular to an automatic marking robot calibration method based on visual guidance, which comprises the following steps of: establishing an image set comprising different resolution levels based on operation site image data acquired by a marking robot, and performing corner detection and straight line segment detection on each level image in parallel. According to the method, through a parallel detection mechanism of a multi-resolution hierarchical image set, angular point and straight line segment features under different scales are synchronously extracted, and a multi-scale feature point set with high robustness is constructed in combination with cross-hierarchical coordinate stability measurement and response intensity quantification. And performing dynamic screening and grouping association on the feature points based on a preset geometric constraint condition, eliminating noise interference and false detection features, and generating a candidate calibration structure set with spatial consistency. Weighted contribution value fitting is adopted, cross-scale stability and detection confidence of feature points are integrated, and geometric accuracy and anti-interference capability of calibration reference point coordinates are improved.

The invention relates to the technical field of electromyographic signal monitoring, and discloses a guided monitoring method and system based on a wireless surface electromyographic analysis feedback instrument. The method comprises the following steps: acquiring electromyographic signal time sequence data and limb movement track space coordinate information when a target muscle group moves through a multi-channel wireless surface electromyographic sensor; muscle activation intensity and fatigue change features are extracted, a joint activity angle change sequence is calculated, and a muscle-joint collaborative activity feature matrix is generated through space-time alignment fusion; analyzing signal transduction delay and amplitude matching degree in the matrix, and evaluating the nerve driving coordination level; according to the deviation from the preset standard mode, generating a real-time biological feedback signal containing an action correction instruction and a muscle activation intensity adjustment parameter; and converting into a tactile vibration mode and a visual guidance mark to be output. According to the method, synchronous acquisition and fusion analysis of multi-dimensional motion information are realized, and visual guidance is provided through multi-modal feedback.

The invention belongs to the technical field of ablation puncture navigation, and particularly relates to an ablation puncture surgery navigation system under AI visual guidance. Through cooperative work of the phase monitoring module, the channel construction module, the needle moving control module, the displacement prediction module and the feedback control module, an intelligent surgical navigation system is constructed, breathing phase information and multi-modal medical image data of a patient can be obtained in real time, and therefore a safe and optimized needle moving path is accurately planned, and the operation accuracy is improved. Meanwhile, through real-time monitoring and quantitative evaluation of parameters such as tissue deformation and pressure distribution in the needle moving process, potential operation risks can be found and corrected in time, smooth operation is ensured, in addition, the system can dynamically adjust the needle moving path and the puncture angle according to the real-time breathing state of a patient, and the operation safety is improved. The accuracy and the safety of the operation are further improved, so that intelligent management and personalized treatment in the ablation puncture operation process are realized.

The invention belongs to the technical field of machine vision positioning and guiding, and discloses a nanocrystalline magnetism isolating sheet vision guiding laser deburring device and application thereof, and the nanocrystalline magnetism isolating sheet vision guiding laser deburring device comprises an image acquisition module, an image preprocessing module, a central control module, a laser module, a positioning module, a collection module, a data analysis module and a display module; according to the visual guidance technology, an advanced visual recognition system is introduced, the burr position is accurately positioned by monitoring the surface state of a workpiece in real time, and therefore the more intelligent laser deburring process is achieved. And laser parameter optimization: parameters such as wavelength, power and pulse frequency of laser are optimized according to different materials and burr types, so that the optimal deburring effect is achieved, and meanwhile, the completeness of the workpiece is ensured. And in the laser deburring process, environment-friendly materials and technologies are adopted, the influence on the environment is reduced, and the safety in the operation process is enhanced.

The invention discloses a tomato stem picking point attitude estimation method, and relates to a computer vision processing technology. The method comprises the following steps: detecting a long-range directional frame target, acquiring a color flow by a depth camera, and detecting position information and shielding conditions of fruits and fruit stems through a directional frame; visual guidance: matching fruit stems based on tomato growth constraint conditions, and visually guiding the mobile platform and the mechanical arm to move in a coupling manner until the fruit stems are observed when the fruit stems are not matched; detecting close-range key points, regulating and controlling a mechanical arm to a close-range observation position, identifying two points of a main stem and three points of a fruit stem through a key point detection model, and extracting point clouds; performing point cloud data processing, removing point cloud noise through point cloud filtering, and performing adaptive compensation on key points with depth anomaly by using a neighborhood mean value; calculating picking poses, and establishing the picking poses of the fruit stem shear points according to the three-dimensional information of the five key points; according to the method, the problem of tomato shielding in a greenhouse complex scene can be effectively solved, and the fruit stem shearing point pose is obtained.

The invention discloses a modular visual guidance robot feeding and discharging device, and relates to the technical field of robot-assisted feeding and discharging equipment, and the modular visual guidance robot feeding and discharging device comprises a visual module which is installed on a production line and is used for identifying the position of a workpiece and outputting coordinate information; the manipulator module is mounted on the production line, and a clamp connecting mechanism and an end effector are arranged at the tail end of the manipulator module; the clamp module is installed at the tail end of the mechanical arm module through a clamp connecting mechanism, and the clamp module is electrically connected with the tail end executor and used for grabbing a workpiece; and the control system module is electrically connected with the vision module, the manipulator module and the end effector and is used for controlling the manipulator module to act according to the coordinate information output by the vision module and controlling the clamp module to act through the end effector. The production requirements of diversified products on different production lines can be met, and the universality and adaptability of equipment are improved.

The invention relates to the technical field of home appliance maintenance, in particular to a cross-brand home appliance maintenance knowledge sharing system and method based on federated learning, a multi-mode fault data collection and intelligent maintenance system integrates image, text, fault phenomena and multi-mode instruction data collection, and after pre-processing, the multi-mode fault data collection and intelligent maintenance system is established. A feature decoupling model is established through a maintenance knowledge analysis module, a cross-brand maintenance knowledge base stores general and specific fault information and assists knowledge migration, a federal learning training module adopts differential privacy homomorphic encryption, data privacy is protected, gradient aggregation cooperative training is achieved, a reality maintenance guidance terminal is augmented to visually guide maintenance, the efficiency is improved, and the maintenance efficiency is improved. The preventive maintenance module calculates the equipment maintenance cycle, generates a prevention scheme, and reduces the failure rate and cost. The system significantly improves the maintenance accuracy, rapidly fuses into a new brand, realizes knowledge sharing and utilization, and brings revolutionary revolutions to the household appliance maintenance industry.

The invention discloses an apple recognition positioning and obstacle avoidance method based on zero samples and visual guidance. An improved YOLOV8 model is used for recognizing unknown category objects in a picking scene; semantic embedding of apples and branches is constructed by combining an external knowledge base Word2Vec, and unknown category objects in a picking scene are identified by using a zero sample learning method; constructing three-dimensional point cloud data of a fruit tree where an identification result is located, carrying out iterative convergence point cloud denoising by adopting a convex set constrained by an alpha norm, and updating point cloud coordinates; performing point cloud segmentation by using a potential energy function segmentation model; obtaining three-dimensional information of the apple and the obstacle through three-dimensional reconstruction of the point cloud feature information; extracting three-dimensional space position data of the branches at different time points to form a time sequence data set, and predicting the motion trail of the branches by using LSTM (Long Short Term Memory); through branch coordinate data of the point cloud and the LSTM, predicting a motion track of the branch, and utilizing a geometric collision detection method to evaluate and predict a collision risk between a robot end effector and the branch; and planning an optimal obstacle avoidance path by using an A * algorithm.

The invention relates to the technical field of artificial intelligence, and discloses a vision-guided automatic sorting and deviation rectifying method and system.The vision-guided automatic sorting and deviation rectifying system comprises a vision parameter processing module, a sorting strategy decision module and a deviation rectifying execution control module, a real-time judgment mechanism of dynamic and static resolution differences is established by setting a resolution collaborative correction end, and the deviation rectifying execution control module is established; a self-adaptive interpolation algorithm is adopted to adjust visual feature data, so that the system can eliminate imaging resolution deviation caused by ambient light and transmission vibration, the acquisition accuracy of material surface texture features is ensured, sorting identification errors caused by projection chromatic aberration are avoided, the environmental adaptability of the visual guidance system is improved, and the visual guidance system is more accurate in image quality. The position and posture coupling analysis end is arranged, the dynamic matching degree of the position and posture offset of the materials and the motion trail of the mechanical arm is monitored in real time, the grabbing angle offset is automatically corrected through a space coordinate compensation mechanism, and it is guaranteed that the relative space position of an end effector and the materials is in the optimal grabbing state all the time.

The invention discloses a visual guidance positioning system and method in the machine tool flexible shaft assembling process, and relates to the technical field of flexible shaft assembling, a visual system obtains three-dimensional pose data of the tail end of a flexible shaft and a target connector, a six-dimensional force sensor is combined to monitor contact force / torque in real time, the limitation of a single sensor is broken through, and the blind area of the single sensor is solved; visual and force sense coordinate systems are aligned in real time, and coordinate drifting caused by movement of the mechanical arm is eliminated; generating an approximation path based on the pose difference, introducing a compensation amount formula, inhibiting oscillation in combination with an attenuation coefficient, and improving the path smoothness; when the contact force exceeds the dynamic threshold value or the pose difference converges, path correction is automatically triggered, overload damage is avoided, and the assembly damage rate is effectively reduced. The method breaks through the limitation of traditional static path planning, has high precision, high reliability and complex working condition adaptability, and is especially suitable for precise instrument and heavy load machinery assembly scenes.

The invention discloses a control method and system based on a self-positioning helicopter field navigation aid lamp and a medium, and relates to the technical field of aircraft landing auxiliary devices.The method comprises the steps that environment information, unmanned aerial vehicle state information and navigation aid lamp layout information are obtained, and initial guiding parameters are determined; calculating a visual guidance signal parameter based on the position condition of the unmanned aerial vehicle relative to each navigation aid lamp and the initial guidance parameter; determining a navigation aid lamp control scheme according to the visual guidance signal parameters and the initial guidance parameters, and sending a lighting starting instruction to a plurality of corresponding navigation aid lamps; acquiring environment feedback information from each navigation aid lamp area, and calculating an environment difference value; if the environment difference value exceeds a preset environment change threshold range, triggering an environment adaptability adjustment instruction; and calculating corresponding navigation-aid lamp adjusting parameters and sending an illumination adjusting instruction to the corresponding navigation-aid lamp area. According to the invention, the safety and reliability of the unmanned aerial vehicle under a complex or emergency landing condition are improved.

The invention discloses a pitaya robot system based on intelligent treatment and recuperation and a control method, and the system comprises a high-precision image collection and human body image collection and three-dimensional reconstruction module which is used for precisely recognizing and positioning a treatment region and acupoints; the core physician manipulation digitization module is used for finely capturing professional treatment manipulations and standardized recuperating manipulations of deep physicians for operating pitayas through a multi-dimensional sensor array and forming a manipulation model library; the intelligent strategy generation module based on artificial intelligence can call, fuse or optimize and generate an optimal execution scheme from a corresponding model library according to a mode selected by a user; and the high-precision and compliant force control robot execution module reproduces the digitized technique of the pitaya in a high-fidelity manner under the real-time closed-loop control of visual guidance and force sense feedback. Through the double-track parallel strategy, quantification, inheritance and popularization of the high-level pitaya therapy are achieved, and application of the high-level pitaya therapy is pushed to the wide field.

The invention relates to a visual guidance robot automatic control method, system and device and a medium. The method comprises the following steps: firstly, acquiring a target image sequence and extracting initial visual features, and processing the initial visual features through a deep learning algorithm to output a high-resolution image; calculating a distance switching value based on the high-resolution image, and if the distance switching value exceeds a preset threshold value, performing scale distortion correction on the high-resolution image to obtain a target image; multiple frames of time sequence data are fused through a Kalman filtering algorithm, image distortion real-time compensation and feature point track stability judgment are synchronously completed, and a target positioning precision estimated value is output; and finally, according to the target positioning precision estimated value, determining the position of a grabbing point through a posture mapping rule of mapping preset grabbing feature points in a target coordinate system to a robot-based coordinate system, and based on the position of the grabbing point, optimizing and fusing track parameters through a Kalman filtering algorithm to generate adaptive track parameters. The method improves the positioning precision and track following effect of automatic control of the robot.

The invention discloses a mechanical arm autonomous grabbing system and method based on visual guidance and visual touch fusion. The mechanical arm autonomous grabbing system at least comprises a visual semantic guidance and grabbing pose generation module, a smooth grabbing control module based on visual touch fusion and a dynamic planning module based on model prediction path integration. The visual module efficiently generates a six-degree-of-freedom grabbing pose from the point cloud data by using a deep learning network; the smooth grabbing control module monitors object slippage in real time and adaptively adjusts clamping force by fusing visual and tactile information, and stable grabbing of fragile or slippery objects is achieved. And the dynamic planning module adopts a model prediction path integral algorithm MPPI to realize collision-free path planning of the mechanical arm in a complex dynamic environment. The problems that an existing mechanical arm is insufficient in sensing capacity, poor in grabbing robustness and low in path planning efficiency in an unstructured scene are solved, and the autonomous operation capacity and safety of the mechanical arm in the scenes of accompanying, intelligent sorting and the like are remarkably improved.

The invention discloses a position and posture compensation method for guiding installation of a contact network cantilever based on 3D vision and an automatic installation system. The position and posture compensation method comprises the steps that the spatial transformation relation between a 3D vision sensor coordinate system and a robot coordinate system is determined; the robot is controlled to move, and the three-dimensional point cloud data of the to-be-installed cantilever component and the three-dimensional point cloud data of the field installation foundation are collected; the collected three-dimensional point cloud data is registered with a corresponding preset CAD model, and a first real-time pose of the cantilever component in a robot coordinate system and a second real-time pose of the installation foundation in the robot coordinate system are obtained through calculation; calculating a pose compensation transformation quantity based on a pre-stored theoretical mounting pose of the cantilever, the first real-time pose and the second real-time pose; and according to the pose compensation transformation amount, planning and executing a motion track of the robot, and completing butt joint installation. The method is high in compensation precision, robustness and adaptability and high in construction efficiency, and the problem of uncertainty in automatic installation is fundamentally solved.

The invention provides a space-time causal intelligent driving method based on an eVTOL aircraft, and the method comprises the steps: obtaining route information, and determining a take-off and landing place according to a terrain selection decision algorithm; according to a flight route preprocessing result, a preliminary route is preliminarily planned for the selected take-off and landing place; real-time modeling is carried out in the take-off and landing stages of the aircraft, real-time three-dimensional models near the take-off and landing places are constructed, and the aircraft is helped to achieve autonomous take-off and landing; using a space-time causal model to predict the occurrence probability and path of a dynamic obstacle in the route, and dynamically optimizing the initial route; when an air emergency occurs, a route is re-planned or a landing place is found nearby, and emergency landing is guided visually; and after the voyage is finished, the route information of this time is stored in the database in time, so that the accuracy of the next time-space prediction is improved. The method meets the operation requirements of complex and changeable urban air traffic and high flexibility.

The invention is suitable for the field of data processing, and discloses a visual guidance method and device for a storage location, terminal equipment and a storage medium. The visual guidance method for the storage location comprises the following steps: when an access instruction is detected, positioning a target storage location according to a product identification code carried by the access instruction; according to the target storage location, unique indicator light control parameters are generated, and the indicator light control parameters comprise color parameters, brightness parameters and / or flicker frequency; according to the indicator light control parameters, control operation of the LED lamp is executed, and visual guidance is provided for the target position where the target storage location is located. By generating a unique indicator lamp control parameter (including a color parameter, a brightness parameter and / or a flicker frequency) for each access instruction, each operator can clearly identify an LED lamp that guides himself / herself. The detection of the access instruction, the positioning of the target storage location and the generation of the indicator light control parameters make the storage process more efficient.

The invention discloses a robot bolt tightening method and system based on 3D visual guidance, and relates to the technical field of intelligent manufacturing, and the method comprises the following steps: S1, constructing a minimum photographing frequency model according to known camera and frame data, and solving an optimal photographing path; s2, acquiring an image and point cloud information of the frame; s3, inputting the image into an intelligent bolt detection model to obtain size and position data of the bolt; s4, bolt surface point cloud information is cut out according to the data of the bolt, and the posture and the specification size of the bolt are calculated through fitting of the bolt surface point cloud information; s5, calculating the transformation relation of the pose of the tightening gun under the image coordinate system and the frame coordinate system; and S6, the robot is guided to conduct tightening operation according to the transformation relation and the posture, specification and size of the bolt.

The invention discloses a numerical control machine tool autonomous tool setting method based on 3D visual guidance. The method comprises the following steps: firstly, acquiring three-dimensional point cloud data of a cutter and a workpiece by using a 3D visual sensor and preprocessing; then, segmenting the point clouds through a region growing algorithm, realizing the separation of the tool point clouds and the workpiece point clouds, and extracting the point clouds of each cylindrical section of the workpiece; thirdly, based on a cylinder fitting algorithm, cylinder model fitting is conducted on the segmented tool and workpiece point cloud, and geometric parameters such as the axis direction, the radius and the center coordinate are obtained; calculating a relative position relation between the tool and the workpiece according to the geometric parameters, generating a tool setting path and converting the tool setting path into an NC code; and finally, the NC code is transmitted to a grinding machine control system through a grinding machine communication module, and tool setting operation is completed. The automatic tool setting device has the advantages that the relative position relation between the workpiece and the tool can be automatically measured in the machining process, automatic and rapid tool setting is completed, and the machining quality and efficiency are guaranteed.

The invention discloses a 3D visual guidance steel box girder pushing and position adjusting method based on Beidou positioning. The method comprises the steps that a Beidou positioning device and an inclinometer are installed at a positioning datum point and a horizontal datum point of a steel box girder respectively; feature identifiers are installed on the box girder segments, and visual devices are correspondingly installed on the outer sides of the box girder segments and are used for monitoring three-dimensional displacement of the feature identifiers in the pushing process in real time; monitoring data of the Beidou positioning device, the inclinometer and the visual device are collected in real time, a dynamic three-dimensional model of the steel box girder is established, and the pushing posture and the pushing stroke of the steel box girder are adjusted on the basis of the dynamic three-dimensional model till the current box girder section reaches the designed mileage. According to the method, the dynamic characteristics of the steel box girder are perfected by taking the Beidou positioning data as the reference and utilizing the visual monitoring data and the inclination angle monitoring data, and the steel box girder pushing dynamic three-dimensional model is established, so that the pushing posture and the pushing stroke of the steel box girder can be automatically and accurately adjusted in the pushing process, and the efficiency and the safety of pushing construction are ensured.

The invention provides a visual guidance numerical control grinding and polishing surface quality real-time regulation and control method and system, and the method comprises the following steps: starting numerical control grinding and polishing equipment, checking a power supply, air pressure and the operation state of a robot, switching the equipment to an automatic operation mode, carrying out the registration of an industrial camera and a robot coordinate system through a calibration plate, and carrying out the real-time regulation and control of the robot. And establishing a mapping relationship between the image and the physical size. Workpiece surface images are collected in real time through linkage of a visual sensor and robot IO signals, characteristic parameters are analyzed through a convolutional neural network model, quality scores are generated, grinding and polishing parameters (the rotating speed of a main shaft is + / -500 RPM, the thrust of a bit force shaft is + / -10 N, and the adjustment amplitude is smaller than or equal to 20%) are dynamically adjusted in combination with a parameter optimization rule base and a PID algorithm, and the grinding and polishing precision is improved. A'detection-evaluation-regulation 'closed-loop control process is formed, the problem of material waste caused by the fact that the quality state cannot be fed back in real time in traditional offline detection is solved, self-adaptive response to surface topography changes is achieved by replacing experience preset parameters, and machining consistency is improved.

The invention provides a calibration-free visual guidance alignment method based on multiple ArUco codes, and belongs to the technical field of image processing, and the method comprises the steps: installing an ArUco code positioning module on an operation target; the positioning module deploys the number, position and angle of ArUco codes according to a positioning requirement; a mechanical arm is controlled to approach a target through a manual teaching operation method to complete alignment, mechanical arm alignment images are collected in real time through a camera at the tail end of the mechanical arm, and pose information of all joints of the mechanical arm when the mechanical arm alignment images are synchronized; a guiding track is generated through a mechanical arm alignment image collected by a camera and mechanical arm joint pose information; and when the robot performs automatic positioning, the mechanical arm is driven to complete alignment according to the guide track by using the image feature relationship. According to the method, the problem that a complicated calibration process and specific texture or feature identification are needed when workpieces with complicated surface characteristics such as transparent parts, reflective parts and black parts are processed by a traditional positioning method is solved.

The invention discloses a robot cooperative machining method under the constraint of form and position errors. The robot cooperative machining method comprises the steps that 1, a double-machine cooperative machining in-situ manufacturing system for machining through a symmetric ground rail and double mechanical arms is constructed; step 2, establishing a conversion relation of all coordinate systems of the dual-machine collaborative processing in-situ manufacturing system; step 3, establishing a machining process flow integrating scanning measurement, blank assembly, allowance machining and precision detection; and 4, the double-machine cooperative machining in-situ manufacturing system conducts visual guidance double-machine cooperative machining on the large-size component based on the conversion relation and the machining technological process, and double-mechanical-arm cooperative error calculation and cooperative error correction based on visual feedback are conducted according to the shape and position error constraint of the large-size component. According to the invention, repeated measurement, disassembly and machining of parts in large-size assembly parts and error superposition of single-machine precision in a double-machine operation process are avoided.

The invention relates to the technical field of visual navigation, and discloses a visual navigation AGV (Automatic Guided Vehicle) transportation system, which comprises a visual guidance module, a driving module, a path planning and control module and an obstacle avoidance and safety module, the visual guidance module is responsible for acquiring real-time image or video data of an environment through a camera and transmitting the real-time image or video data to the path planning and control module, and comprises a 3D deep high-definition camera, an image acquisition card, an image processor and a display screen; the driving module is responsible for driving the AGV to move according to the control instruction, so that the AGV moves along a predetermined path, receiving speed and direction instructions from the path planning and control module, and executing the instructions; the path planning and control module is responsible for receiving environment data from the visual guidance module and calculating an optimal driving path in combination with a built-in algorithm; the obstacle avoidance and safety module is used for detecting and preventing the AGV from colliding with an obstacle in the driving process. The material carrying device has the advantage of meeting the material carrying requirement under the complex and changeable environment.

Techniques for displaying an extended reality (XR) indication of a potential collision between a portion of an object and a portion of a computer-assisted system include the following. The computer-assisted system comprises a sensor system configured to capture sensor data of an environment, and a control system communicably coupled to the sensor system. The control system is configured to: determine a pose of a portion of an object in the environment based on the sensor data, determine a pose of a portion of the computer-assisted system, determine at least one characteristic associated with a potential collision between the portion of the object and the portion of the computer-assisted system, select the potential collision for display based on the at least one characteristic, and cause an XR indication of the potential collision to be displayed to an operator via a display system.

The invention provides an intelligent vision-guided mechanical arm precise grabbing system. The system comprises a bionic compound eye vision module for collecting multi-angle image information of a target object to generate a dynamic depth map, and a bionic touch sensing module for collecting touch information at the tail end of a mechanical arm to generate a pressure distribution heat map. And the dynamic self-optimization grabbing strategy module is used for generating a grabbing strategy by utilizing the two graphs. And the real-time trajectory planning and obstacle avoidance module optimizes a grabbing path according to the grabbing strategy, the environment obstacle information and the mechanical arm state data, and generates an optimal strategy. The robot can sense the movement of a target object and the change of environmental obstacles in real time, dynamically adjust the force and the posture in the grabbing process, and optimize the obstacle avoidance path in real time. Compared with the prior art, the system has the advantages that the aspects of dynamic obstacle avoidance, target object motion compensation, multi-modal information fusion robustness and the like are remarkably improved, the problem of insufficient adaptability in complex dynamic scenes in the prior art is solved, and high success rate and stability of a precise grabbing task under complex conditions are ensured.