363 results about "3d vision" patented technology

The invention discloses a full-automatic grinding method and system for steering knuckle flash based on 3D vision, and relates to the technical field of casting post-processing automation. A trimming-free direct polishing process is innovatively provided, a casting is scanned in real time through a high-precision 3D visual system, a flash space molded line is accurately captured, and one-key calibration of a camera, automatic generation and dynamic simulation of a flash path, multi-tool intelligent switching and self-adaptive adjustment of residual height are achieved in combination with self-developed off-line programming software. According to the system, a robot is adopted for grabbing workpieces to cooperate with a fixed type multi-cutter work station, a disordered feeding technology is integrated for 2D / 3D visual guiding grabbing, workpiece deformation and die difference are effectively dealt with, and flash residues are thoroughly eliminated. The method is characterized in that visual scanning, path planning and execution control are seamlessly fused, the dependence of traditional automatic grinding on the trimming process is broken through, the yield and efficiency are remarkably improved, and the industrial problem of full-process automation of casting flash is solved.

The invention discloses a robot unstacking and stacking support system based on a 3D visual system, and belongs to the technical field of 3D visual systems. Comprising a 3D visual data acquisition preprocessing module, a target positioning priority analysis module, a box type intelligent identification matching module, a dynamic path planning obstacle avoidance module, a self-adaptive stacking strategy generation module, an information binding tracing management module and a feedback optimization module. The problems of low efficiency and conflict caused by a single grabbing sequence in traditional stacking and unstacking operation are solved, the priority weight is dynamically adjusted by combining the inclination angle of the carton, weight distribution and reachability verification, and the optimality and safety of the grabbing action of the robot are ensured.

The invention relates to the technical field of 3D vision, and discloses a manipulator dispensing path optimization method and device based on 3D vision, and the method comprises the steps: carrying out the multi-angle three-dimensional vision scanning of a dispensing workpiece, and obtaining the point cloud data of the dispensing workpiece; performing semantic segmentation on the point cloud data, and decomposing the dispensing workpiece into a plurality of functional parts; according to the semantic identifier and the surface feature of the functional component, identifying a target area to be dispensed and a dispensing path constraint condition; generating an initial dispensing path based on the target area to be dispensed and the dispensing path constraint condition; and performing iterative linear secondary adjustment control on the initial dispensing path to generate a motion control instruction. According to the invention, the self-adaptive optimization adjustment of the dispensing path is realized, the problem that the traditional dispensing system cannot cope with the actual state change of the workpiece is solved, and the adaptability and stability of the dispensing system are greatly improved.

The invention belongs to the technical field of 3D visual reconstruction, and particularly provides a 3DGS segmentation method and system based on boundary adaptive Gaussian splitting, and aims to solve the problem of incomplete structure caused by directly deleting boundary Gaussian in the prior art by accurately positioning and segmenting the boundary Gaussian through a gradient consistency index through a boundary adaptive Gaussian segmentation technology. The boundary mBIoT is improved by 1.3%; meanwhile, a semantic-vision joint alternating optimization mechanism is innovatively proposed, continuous mask labels and alpha are utilized for synthetic rendering, and semantic loss and vision loss are combined for alternating optimization, so that compared with a traditional scheme singly depending on mask or feature optimization, the PSNR is improved by 0.29 dB, the texture definition is improved by 15%, and the segmentation precision and the vision quality are both considered; besides, through small-scale fuzzy Gaussian recognition and a targeted optimization strategy, mIoU is only reduced by 1.1% when the mask error rate reaches 20%, robustness to error masks is remarkably enhanced, and the bottleneck that an existing method is sensitive to pre-training mask errors is broken through.

The invention relates to the technical field of 3D vision measurement, and discloses a 3D binocular line laser weld contour data acquisition method and system. Two core steps of obtaining surface reflection characteristic information of a to-be-scanned area and adjusting output intensity of laser rays projected by a line laser device at different positions of the to-be-scanned area according to the surface reflection characteristic information are introduced, so that the problem that when a high-reflection welding seam is processed by a traditional method, the laser rays cannot be projected by the line laser device is effectively solved; the problem of image overexposure or underexposure caused by dramatic change of illumination characteristics is solved.

The invention relates to the field of three-dimensional vision measurement, in particular to an aluminum alloy CNC product defect detection method and system based on structured light scanning. The aluminum alloy CNC product defect detection system based on structured light scanning comprises an image acquisition and analysis module, a self-adaptive illumination control module, a point cloud reconstruction module, a defect detection module and a report generation module. According to the method, through texture direction perception and matrix type light source dynamic angle modulation, and in combination with a light intensity-wavelength self-adaptive scheme of saturated pixel grading, mirror surface glare of the aluminum alloy high-reflection surface is greatly reduced; in cooperation with multi-exposure gray scale weighted fusion, a stripe image which is uniform in contrast and free of overexposure can be obtained in single scanning; in the same scanning time, compared with a fixed illumination scheme, the point cloud integrity rate can be improved, and a more reliable three-dimensional data basis is provided for subsequent defect detection.

The invention provides a welding seam identification and trajectory optimization method and system based on three-dimensional vision, and the method specifically comprises the following steps: firstly, collecting the three-dimensional point cloud data of a welding seam, extracting the local geometric features of the welding seam, and carrying out the feature enhancement of the local geometric features, so as to screen out representative effective edge points; and then, collecting multi-source point cloud data of the welding seam area, constructing a multi-scale welding seam feature recognition network, inputting effective edge points obtained by screening into the network, realizing accurate recognition of the welding seam area, and further extracting a linear welding seam area. According to the recognition result, a fitting algorithm is adopted for conducting curve fitting on the weld joint area, and a preliminary welding track is generated; and for the condition that multiple sections of welding seams are intersected, an optimal fitting point searching mechanism is further introduced, the preliminary track is optimized, and the continuity and integrity of a welding path are ensured. And finally, based on the center line of the welding seam, the posture of the welding gun is precisely planned in combination with the dihedral structure model, and therefore high-precision track control and posture adjustment are achieved.

The invention discloses a numerical control machine tool autonomous tool setting method based on 3D visual guidance. The method comprises the following steps: firstly, acquiring three-dimensional point cloud data of a cutter and a workpiece by using a 3D visual sensor and preprocessing; then, segmenting the point clouds through a region growing algorithm, realizing the separation of the tool point clouds and the workpiece point clouds, and extracting the point clouds of each cylindrical section of the workpiece; thirdly, based on a cylinder fitting algorithm, cylinder model fitting is conducted on the segmented tool and workpiece point cloud, and geometric parameters such as the axis direction, the radius and the center coordinate are obtained; calculating a relative position relation between the tool and the workpiece according to the geometric parameters, generating a tool setting path and converting the tool setting path into an NC code; and finally, the NC code is transmitted to a grinding machine control system through a grinding machine communication module, and tool setting operation is completed. The automatic tool setting device has the advantages that the relative position relation between the workpiece and the tool can be automatically measured in the machining process, automatic and rapid tool setting is completed, and the machining quality and efficiency are guaranteed.

The invention discloses a badminton tracking method and system based on three-dimensional vision, and relates to computer vision. Badminton motion video data of a left view and a right view are synchronously collected through a binocular camera, a left view frame sequence and a right view frame sequence which are aligned in time are obtained, and the collected video data are preprocessed; obtaining standardized left and right view input frame data; according to input frame data, constructing a 2D key frame detection model based on an improved YOLOv8 network, and extracting the sum of 2D key frame coordinates of left and right views; according to the 2D key frame coordinate sum of the left view and the right view, 3D space positioning is carried out on the badminton through stereo matching and track prediction, 3D track data are obtained, according to the 3D track data, a broken track is repaired through a frame leakage compensation mechanism, and a final 3D track is obtained. In order to solve the problem that in the prior art, key frames are prone to being lost and track breakage is caused during high-speed movement such as smash, the breakage track of the shuttlecock is repaired.

The invention discloses an intelligent deviation correction control method and device based on 3D vision and a storage medium, and relates to the field of conveying belts. Carrying out edge searching calculation on the belt according to the no-load point cloud data of the belt, and determining a reference detection area and a point cloud reference curve of the fitting belt; screening belt point clouds and material point clouds based on the point cloud reference curve, performing offset detection on the belt point clouds based on the reference detection area, and performing gravity center deviation calculation on the materials based on the material point clouds; and according to the material gravity center deviation and the belt movement speed, the basic deviation correction angle and the flow compensation angle of a deviation correction motor are calculated, and belt deviation correction is conducted. According to the scheme, the reference detection area is established, point cloud screening, offset detection and gravity center deviation calculation in the load state are combined, the angle parameters of the deviation correction motor are dynamically adjusted, and the problems that 2D visual detection is poor in stability, 3D point cloud data processing is insufficient, and multi-motor cooperative control precision is low are solved; the method has the advantages of improving the detection stability, enhancing the data processing capability and optimizing the correction control precision.

The invention relates to the technical field of cigarette production, in particular to an automatic auxiliary material tray matching work station suitable for full-specification production. Comprising an industrial robot module, a 3D visual camera module, a code reader module, a tool clamp module, a clamp storage module, a cigarette auxiliary material and a tray. The tool clamp module and the clamp storage module are arranged on the two sides of the industrial robot module respectively, the 3D visual camera module is located above to-be-prepared auxiliary materials to be used for positioning and sending position information, the code reader module is arranged between an industrial robot and the prepared auxiliary materials to be used for scanning codes, and an automatic gun changing disc at the tail end of the industrial robot can switch corresponding clamps. And after the industrial robot receives the position information, the clamp is switched to grab the to-be-prepared auxiliary materials, and the auxiliary materials are placed on the prepared auxiliary material tray after being matched with the code reader to scan codes. Through multi-module cooperation, full-process automation of tray distribution is achieved, manual operation is reduced, production efficiency is improved, meanwhile, by means of accurate positioning and code scanning, the material recognition accuracy is guaranteed, and the workshop automation and intelligence level is remarkably improved.

The invention provides a precise workpiece positioning method and system based on three-dimensional vision, and the method comprises the steps: obtaining a scene three-dimensional vision data set containing a target workpiece, carrying out the multi-modal data fusion processing of the scene three-dimensional vision data set, generating a fusion data set with aligned space coordinates, extracting a workpiece feature identifier set from the fusion data set, and carrying out the recognition of the workpiece feature identifier set. And feature matching association processing is carried out on the workpiece feature identifier set and a pre-stored standard workpiece feature template, a feature matching corresponding relation set is generated, three-dimensional positioning coordinate information of the target workpiece in the scene is calculated based on the feature matching corresponding relation set, and the three-dimensional positioning coordinate information comprises workpiece centroid coordinates and space attitude angle parameters. The precision of precision workpiece machining is improved, and the stability of the positioning process is ensured.

The invention discloses a stereoscopic vision optimization method and system for a naked-eye 3D large screen, and particularly relates to the technical field of naked-eye 3D vision optimizing.The method comprises the steps that environment illumination and audience positions are sensed in real time through multi-sensor fusion, and an environment light field model is established; glare crosstalk noise is predicted based on physical simulation, and self-adaptive suppression and compensation are carried out in combination with human eye visual sensitivity and image content features; a virtual camera is dynamically generated according to the real-time positions of the eyes of the audience, and a lightweight neural radiation field renderer is used for real-time re-rendering, so that motion parallax is realized; and finally, intelligently fusing the glare compensation layer and the perspective correction layer, coding and outputting to a screen. The system correspondingly comprises an environment perception module, a glare compensation module, a perspective rendering module and a fusion coding module. The naked-eye 3D large-screen display method effectively inhibits ambient light interference, improves the quality and immersion of a stereoscopic picture under different visual angles, and is suitable for naked-eye 3D large-screen display under outdoor and complex illumination environments.

The invention belongs to the technical field of industrial automation, and particularly relates to a snake-shaped pipe grabbing point positioning method based on 2D vision and 3D vision fusion, and the method comprises the steps: collecting an original image set of a snake-shaped pipe, the original image set comprises original images of the snake-shaped pipe at a plurality of different visual angles, and the original images comprise depth images and texture images; based on each texture image in the original image set and the trained target detection model, determining a planar clip-free area; based on the original image set and the planar clamping piece-free area, determining a complete point cloud of the coiled pipe and a point cloud of the area to be positioned; based on the complete point cloud of the coiled pipe and the point cloud of the to-be-positioned area, determining a target positioning area; and determining a grabbing point of the coiled pipe based on the target positioning area. According to the technical scheme provided by the invention, the grabbing point is positioned in combination with the 2D image and the 3D image of the serpentine pipe, so that the accuracy and efficiency of positioning the grabbing point of the serpentine pipe are improved.

The invention discloses an intelligent grabbing system anomaly detection and processing method and device based on 3D vision, and belongs to the technical field of industrial robot 3D vision guide grabbing. The method comprises the steps of collecting 3D point cloud data of stacked flexible plates; carrying out point cloud preprocessing and template matching, and determining a plate pose and a grabbing point; generating a cushion block detection area and a deformation detection area based on the grabbed point pose; counting the number of point clouds in the detection area, comparing the number with a threshold value, and judging whether cushion block residues or plate excessive deformation exists or not; if yes, triggering processing, and if not, outputting a grabbing instruction. The device comprises a truss manipulator, an end effector, a 3D visual sensor, a visual processing unit and a robot control unit. According to the method, the detection area is dynamically generated, point cloud threshold judgment is fused, self-adaptive detection and processing of excessive deformation of the cushion block and the plate are achieved, the robustness, safety and automation level of a grabbing system are remarkably improved, and the method is suitable for automatic unstacking operation of stacked flexible plates.

The invention discloses a supply chain warehouse management system and method based on AI digital intelligence, and belongs to the technical field of warehouse management. A sensing unit comprises a 3D vision module, an environment sensing module and a cargo positioning module; the edge calculation unit comprises an Internet of Things gateway module, an edge database and an edge calculation module; the data twinning unit comprises a three-dimensional modeling engine module and a virtual deduction module; the intelligent decision-making unit comprises a path optimization module, a demand prediction module and an abnormality diagnosis module; the execution unit comprises an AGV conveying module and a stacking machine storage module. According to the method, closed-loop intelligent management of the whole warehousing process from cargo sensing, intelligent decision making to automatic execution is achieved, accurate prediction, scientific decision making and continuous optimization are driven based on mass multi-source data fusion and AI analysis, and the real-time performance of the whole warehousing process is improved through real-time sensing, edge calculation, digital twinborn deduction and intelligent decision making. The system has strong ability to deal with uncertainties such as order fluctuation, equipment fault and environment change, and improves operation toughness.

The invention discloses a robot disordered grabbing method and system based on 3D vision, and relates to the field of vision grabbing, and the method comprises the steps: configuring a 3D vision collection module to execute incoming material area continuous collection, executing definition of a robot base coordinate system, a camera coordinate system and an incoming material station coordinate system, and setting a tray in-place consistency constraint. Deterministic mapping of a clamp and a strategy is formed under multiple types of incoming materials, the grabbing success rate is improved, deformation and falling risks are reduced, collision safety is improved with minimum safety gap constraint and path cost selection in an unordered environment, and operation stability is maintained through action time accounting and beat conformity judgment on the beat side. And traceability is realized by a sorting accuracy target and a task log write-in mechanism, and finally, shutdown waiting and secondary faults are reduced through alarm and separate disposal rules in a static incoming material and structure separation risk scene.

The invention discloses a pig rib cutting path planning method based on 3D point cloud, and belongs to the technical field of intelligent cutting and robot automation. High-precision original point cloud of a rib area of a pig carcass is obtained in real time through a 3D visual sensor, and then a segmentation threshold value is dynamically adjusted by introducing a self-adaptive growth factor based on local point cloud density, so that the preprocessed point cloud is robustly and accurately segmented into rib and rib meat areas, and a centerline of a spine and a rib gap are synchronously positioned. And based on the segmentation result, a plurality of independent costal meat communication components are automatically identified and separated through voxelization clustering and graph theory connectivity analysis. For each component, the main extension direction of the component is determined by constructing an axis alignment bounding box, and a series of discrete path points closely attached to the actual surface of the point cloud are generated in the direction. Spatial fitting is carried out by utilizing a cubic polynomial curve to generate a smooth and continuous cutting path which is highly fit with an anatomical structure, and the cutting path is directly converted into an executable track of the robot.

The utility model relates to a double-channel PCB (Printed Circuit Board) bonding detection device, which comprises a machine base, a feeding and discharging mechanism is arranged on the machine base and used for completing feeding and discharging operation through two channels at the same time, and a transferring mechanism is used for grabbing PCBs on a feeding assembly and transferring the PCBs to a conveying mechanism at the same time and grabbing the PCBs detected on the conveying mechanism and transferring the PCBs to a discharging assembly at the same time. The detection mechanism is used for detecting the PCBs located on the conveying mechanism. According to the technical scheme, in the process of detecting the PCBs, the feeding and discharging processes of the PCBs can be completed at the same time, so that the feeding and discharging time of the PCBs can be shortened, the feeding and discharging time of the PCBs can be shortened, the PCBs located on the adsorption platform are shot and imaged through the 3D vision camera, and due to the fact that three-dimensional images can be formed, the detection accuracy of the PCBs is improved. Therefore, the PCB can be detected more accurately.

Improved methods and a system for feeding containers and caps into a filling line and a capping line are provided. A 3D vision inspection system identifies a container located proximal to a top of a heap within a containers bin. A first set of robotic arms picks the identified container, places it onto a conveyor input in an upright orientation with the open end of the container facing upwards. The container is then transferred into a conveyor or accumulation table for transport to a filling station. Similarly, a third 3D camera identifies caps within a caps bin. A second set of robotic arms picks the identified caps, places them onto an alignment station, and orients them for placement into the capping line. The system incorporates Artificial Intelligence (AI)-based computer vision models for object identification and orientation, enhancing the reliability and efficiency of automated bottling and capping operations.

The invention discloses an online monitoring method and device for an ore crushing and screening processing technology, and the method comprises the steps: deploying intelligent monitoring equipment at different technology nodes, and detecting the granularity data of ore in a manner of combining 3D vision with a deep learning model; analyzing by combining material quantity data to obtain a screen mesh damage condition and a screen mesh blockage condition of a screening machine used in the pre-screening process and the screening checking process; the material quantity data and the particle size data of the discharged materials of the medium crushing process and the fine crushing process are associated with the first association model and the second association model respectively, the real-time lining plate abrasion loss and the discharge port opening degree of the crusher of the medium crushing process and the fine crushing process are obtained, and the remaining service life is determined according to the real-time lining plate abrasion loss. And adjusting the discharge port of the crusher according to the real-time opening degree of the discharge port to enable the opening degree of the discharge port to reach the preset opening degree. According to the invention, full-process quality monitoring and equipment state feedback can be realized, so that the production is more intelligent.

The invention provides a stereoscopic warehouse positioning system and method based on 3D visual perception, and relates to the technical field of visual identification positioning. Determining a calibration element index of each storage location center in the storage location global image, and determining spatial pose descriptions in adjacent storage location bounding boxes on cargo storage locations of the stereoscopic warehouse according to the calibration element indexes of all the storage location centers; verifying and identifying the position identification image to obtain a visual primitive feature corresponding to each cargo in the cargo placement area, and further determining a positioning identification layer sequence of the cargo on a specified storage position in the stereoscopic warehouse according to the visual primitive features corresponding to all the cargoes; and carrying out fusion sensing positioning on the goods in the stereoscopic warehouse according to the space pose description and the positioning identification sequence. According to the invention, collaborative perception can be carried out on the storage location global structure and the cargo visual features in a multi-sensor fusion environment, so that the accuracy of cargo space positioning is improved.

The invention discloses an automatic loading system. The automatic loading system comprises the steps of initializing loading parameters and rules based on an order; the forklift conveys the material trays to an unstacking station; a laser radar scans a vehicle and generates point cloud data on site, and vehicle body and loading position coordinates are obtained through an AI model and an algorithm; the stand column type robot is guided to unstack through 3D vision; the telescopic belt line conveys the box body; scanning and removing abnormal box bodies during conveying; and the loading mechanism is guided by the 3D vision to complete loading according to rules. The system further comprises a task board, a step crossing module and other auxiliary modules, a cockpit and other auxiliary modules, loading automation and intelligence are achieved, and efficiency and quality are improved.

The invention relates to the technical field of 3D visual inspection, in particular to a dynamic shielding area multi-view time sequence point cloud completion method and system in 3D visual inspection. The method comprises the following steps: collecting a multi-view time sequence image of a dynamic detection object for providing a data basis for feature point tracking; and carrying out Lucas-Kanade optical flow method tracking on the feature points, associating the feature points of different visual angles, describing a motion track of the feature points through an optical flow vector, carrying out cross-visual-angle and cross-time feature point matching, and establishing time sequence feature association to obtain time sequence feature points. According to the dynamic occlusion area multi-view time sequence point cloud completion method in 3D visual detection, 360-degree time sequence data of a dynamic detection object is acquired through multi-view time sequence image acquisition, and comprehensive basic data support is provided for subsequent feature tracking.

The invention relates to the technical field of intelligent welding, in particular to a steel structure fillet weld intelligent identification method and system based on 3D vision, and the method comprises the steps: obtaining a point cloud data set of a workpiece, and extracting point cloud data of a weld region from the point cloud data set; dividing the welding seam area into a plurality of welding seam sub-areas, wherein one welding seam sub-area corresponds to one independent welding seam; carrying out welding seam coarse skeleton extraction on each welding seam sub-region, and carrying out merging processing on the welding seam coarse skeletons of which the distance between end points is smaller than a threshold value; and on the basis of the combined welding seam coarse skeletons, welding seam fine skeleton point set extraction is conducted, and a welding seam path expression is obtained based on welding seam fine skeleton point set fitting. The method solves the problem that an existing algorithm needs an operator to repeatedly adjust a threshold parameter according to experience, resulting in lack of adaptive ability, and solves the problems that the calculated amount is large, the speed is low and the real-time requirement of an industrial site cannot be met when the workpiece point cloud data is processed.

The utility model discloses a mine car large-torque bolt dismounting device which comprises a positioning ring (1), a first connecting plate (2), a tail end executing mechanism body (3), a 2D visual device (4), a 3D visual device (5), a clamping device (6), a hydraulic wrench (7), a floating device (8) and a quick screwing device (9). Precise positioning is realized; automatic dismounting and placing of mine car bolts are achieved through the floating device at the tail end of the bolt dismounting robot, manual operation is not needed in the whole process, and the operation efficiency is higher.

The invention discloses an engine and motor combined spline alignment system and method, and the system comprises a 3D camera detection module which is used for detecting the angle and position of an external spline of a motor, and the angle and position of an internal spline of an engine; the rotating assembly is in signal connection with the 3D camera detection module and is used for adjusting the angle of a motor spline according to a detection result; the robot is used for grabbing the motor and carrying out assembling operation, and the robot is provided with a force sensor used for monitoring the force in the assembling process; the bolt pre-screwing module is used for automatically pre-screwing screws after the assembly is completed; and the NG channel processing module is used for moving out and processing the motor through a specific channel when combination fails. According to the invention, real-time calibration is realized through cooperation of 3D vision and the rotating assembly, deviation is automatically corrected, and the alignment precision is improved; force sensor closed-loop control prevents overload damage; pre-twisting and NG channel shunting are integrated, an automatic closed loop is constructed, manpower dependence is reduced, and production line efficiency is improved.

The invention relates to the technical field of industrial robot control, in particular to a drilling robot automatic rod adding recognition system and control method based on 3D vision, and the system comprises a visual feature collection module, a drilling state analysis module, a fragmentation strategy decision module, an instruction dynamic generation module and a rod adding state verification module. According to the method, through axial sampling interval parameter setting of three-dimensional point cloud data and thread form depth gradient value extraction, in combination with guide groove space coordinate registration, precise quantification of thread lead angle density dispersion is realized, and spatial feature reconstruction precision is improved; dynamically coupling the density change rate of a thread meshing area with a hydraulic propulsion pressure value, and superposing a deflection angle detection value to construct a multi-dimensional offset model, so as to enhance the fault-tolerant capability of drill rod butt joint state analysis; and on the basis of real-time calculation of a density gradient parameter and a mechanical arm angular velocity ratio, three-dimensional fragmentation unit parameters are divided to form a self-adaptive threshold value, and the point cloud data processing efficiency is optimized.

The invention discloses a robot fine operation control system based on a diffusion model and 3D vision, particularly relates to the technical field of industrial robot fine operation control, and aims to solve the problem that when an existing industrial robot executes fine operations such as bottle cap screwing and connector plugging in and out in a complex three-dimensional operation scene, the operation efficiency is high. The problems that three-dimensional perception is unstable, the action resolution is fixed, space and contact constraints are difficult to meet, and the task success rate and safety are insufficient are solved. The method comprises the following steps: constructing a voxel feature field updated along with time under a self-body coordinate system, generating interaction complexity and a physical constraint field based on task coding, and linking adaptive resolution and action granularity scheduling, diffusion trajectory generation and closed-loop execution correction to uniformly map environmental geometry, contact conditions and task stages into tail end trajectory control, so as to realize real-time tracking control. Therefore, fine operations such as bottle cap screwing, connector inserting and pulling and flexible object folding can still be stably completed under complex shielding and contact constraint, and the effects of safety and task success rate are both considered.

The invention relates to the technical field of multi-sensor information fusion positioning, and discloses a multi-sensor fusion-based 3D visual guidance positioning method, which comprises the following steps of S1, acquiring data of an inertial measurement unit, a visual sensor and a laser radar, and performing time-space synchronization and preprocessing; s2, 3D / 2D feature points are extracted and matched, cross-modal feature matching is carried out, a local map is constructed, and the quality of the feature points and the motion state of a carrier are evaluated in real time. According to the method, the 3D / 2D feature points are extracted and matched, cross-modal feature matching is performed, the local map is constructed, and the local map is maintained by adopting a sliding window method, so that the map is updated in real time, and the calculation efficiency is kept. According to the efficient data fusion strategy, the real-time requirement can be met while the positioning precision is guaranteed, and the method is suitable for application scenes with high response speed requirements, such as real-time obstacle avoidance and path planning in automatic driving.