2931 results about "Vision sensor" patented technology

The invention provides a magnetic core intelligent cutting parameter self-adaptive optimization system based on multi-mode perception, and relates to the technical field of data processing.The method comprises the steps that a multi-mode sensor module is integrated on magnetic core cutting equipment, and the module comprises a force sensor, a visual sensor and a temperature sensor; the acquisition units are respectively used for acquiring cutting force dynamic signals, cutting track image sequences and cutter temperature time sequence data in real time; magnetic core surface texture features and three-dimensional contour data are captured through a visual sensor, and an initial cutting parameter set is generated in combination with a magnetic core material type recognition result, associated parameters in a historical process database and preset process constraint conditions; and first workpiece trial cutting is executed based on the initial cutting parameter set, multi-modal data fusion collection is synchronously started, cutting force frequency domain feature vectors, a tool temperature change rate curve and cutting surface defect image features are obtained, and multi-modal data are obtained. According to the invention, multi-objective collaborative optimization of processing efficiency and energy consumption is realized.

A multimodal intelligent agent system for dynamic environmental monitoring and user-centered support, consisting of: a multimodal sensor module configured to continuously acquire environmental and behavioral data from multiple input modalities, including at least one visual sensor, at least one acoustic sensor, at least one environmental conditions sensor, and at least one proximity or motion detection sensor, each generating modality-specific data streams representing visual images, audio waveforms, physical environmental parameters, and motion signatures within a monitored environment; a data preprocessing and fusion subsystem that is operationally coupled with the multimodal sensor module and configured to normalize, temporally align, and transform the modality-specific data streams into high-dimensional feature embeddings using a variety of encoders, wherein the visual encoder uses convolutional or vision transformer architectures, the audio encoder uses a spectral-temporal feature extractor, and the sensor encoder transforms raw analog data into context vectors suitable for multimodal alignment; a multimodal processing unit consisting of a transformer-based large language model (LLM) trained on paired multimodal datasets and configured to perform semantic fusion, context abstraction, and inference across the aforementioned aligned multimodal feature embeddings to generate a contextual understanding of environmental and behavioral states; an adaptive agent controller coupled to the multimodal inference processing unit and configured to instantiate, manage, and terminate a variety of task-specific intelligent agents, each agent being a software unit configured to perform a specialized function selected from meeting summarization, behavioral analysis, misplaced object detection, or environmental anomaly identification, with the agents dynamically interacting with the inference engine to retrieve contextually relevant multimodal embeddings for task execution; a personalization and adaptive learning subsystem consisting of a user preference database and a neural memory structure configured to update and refine model parameters based on user-specific interaction history, thereby enabling personalized output generation, prioritization of recommendations, and long-term behavioral adaptation; and An output generation interface is operationally connected to the adaptive agent controller and configured to produce multimodal output in textual, visual, and auditory form. The interface is capable of displaying human-readable summaries, notifications, and visual reconstructions of identified entities or environmental states.

The invention discloses a mechanical arm dynamic deviation correction method and system based on visual driving and a medium, and relates to the technical field of mechanical arm control. The method comprises the steps that when the tail end of a mechanical arm enters a preset machining space, an integrated 3D visual sensor is triggered to collect 3D point cloud of a workpiece to be machined; after pose recognition is carried out on the point cloud, the offset is recognized according to the teaching pose and the actual pose, and the initial offset is output; calling the multi-dimensional perception data, performing fusion correction, and outputting a correction offset; performing interference correction through an offset compensation model, and outputting a target offset; parameter adjustment and optimization are carried out according to the target offset, and a joint angle adjustment instruction is output; and performing correction closed-loop feedback according to the updated pose data. The technical problems of precision errors and low efficiency caused by deviation in the operation process of the mechanical arm are solved, and the technical effects that through dynamic deviation correction and multi-sensor data fusion, the operation precision and efficiency of the mechanical arm are improved, and stable operation in a complex environment is ensured are achieved.

The invention discloses an unmanned aerial vehicle flight control system and method with precise positioning and autonomous obstacle avoidance, and particularly relates to the field of control, and the system comprises a positioning module, an obstacle detection module, a data processing and analysis module, an autonomous obstacle avoidance algorithm module, and a flight control module. Static and dynamic obstacles are detected in real time by integrating the laser radar, the millimeter wave radar, the visual sensor and the ultrasonic sensor, and an obstacle information report is generated through a data fusion algorithm based on machine learning; based on the processed environment data, the system uses deep reinforcement learning and a model prediction control algorithm to generate a safe flight path in real time, and an obstacle avoidance decision is optimized; the flight control part adopts a hierarchical control framework, and precise flight of the unmanned aerial vehicle is realized through trajectory tracking, attitude control and actuator control.

The invention discloses an automobile steering controller fault detection method and system, and relates to the technical field of automobile electronic control. The surface image acquisition module captures an image through a visual sensor, the quality is improved through the image preprocessing module, the convolutional neural network accurately recognizes appearance defects, and the three-dimensional contour detection and thermal imaging module is triggered to be linked to re-check an abnormal area; the three-dimensional contour module confirms patch offset or tombstone standing abnormity by using a laser scanning technology; the thermal imaging analysis module is matched with a machine learning technology to analyze welding spot temperature abnormity; the predictive fault diagnosis module predicts a defect risk by using deep learning and performs early warning in advance; and the collaborative decision optimization module integrates a multi-module data dynamic optimization detection strategy. Through integration of defect detection, re-checking and prediction, the quality detection precision, coverage and efficiency of the automobile steering controller are greatly improved, the high-quality standard and production stability of products are ensured, and the requirement of the automobile industry for efficient detection is met.

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.

The invention discloses a visual language navigation method for cross-modal alignment in a dynamic shielding environment, and the method comprises the steps: collecting multi-modal data through a visual sensor, an inertial measurement unit, a laser radar and the like, and carrying out the preprocessing and time synchronization; sensing the dynamic shielding object through a model composed of a convolutional neural network and a long-short-term memory network, and estimating the future change of the dynamic shielding object in combination with a space-time sequence prediction algorithm; a double-branch convolutional neural network and a Transform based on a dynamic attention mechanism are adopted to respectively extract visual and semantic features and fuse the visual and semantic features; on the basis of occlusion prediction, potential occlusion region features are extracted in advance from a time dimension, an occluded image is repaired by using a generative adversarial network and geometric constraints in a space dimension, and cross-modal feature alignment is optimized through an attention mechanism; planning a path by using a hybrid reinforcement learning algorithm based on a deep Q network-space and a fast exploration random tree, and dynamically adjusting according to real-time shielding; according to the method, the accuracy, adaptability and reliability of visual language navigation in a dynamic shielding environment are improved.

The invention relates to the technical field of video monitoring, and discloses a camera cooperative monitoring method and system based on multi-modal perception, and the method comprises the steps: detecting an abnormal event signal in an environment through a non-visual sensor node, triggering the activation of a camera, and carrying out the visual capture of a target region; the camera performs relay tracking based on target feature matching to generate a continuous motion track; performing multi-view collaborative shielding on privacy sensitive information in the tracking target, generating desensitized monitoring data, and uploading the desensitized monitoring data; and optimizing the energy consumption of the camera in a non-event triggering period, and updating the behavior recognition model based on the desensitization data. According to the method, the contradiction between privacy protection and monitoring efficiency is solved, cross-regional model evolution and energy consumption reduction are realized, and the problems of response lag, data redundancy, privacy disclosure and high energy consumption of traditional monitoring are systematically avoided.

The invention relates to the field of excavation equipment obstacle avoidance based on image analysis, in particular to a coal mine excavation equipment obstacle avoidance method and system based on machine vision, and the method comprises the steps: obtaining the frequency domain characteristic parameters of the current electromagnetic noise during the operation of underground electromechanical equipment, and synchronously collecting the current original image frame output by a visual sensor; after a current de-noised image frame is obtained after dynamic de-noising processing, an obstacle contour feature point set is extracted through multi-scale edge detection, and an obstacle position confidence map is obtained according to space-time consistency analysis; obtaining point cloud coordinates of an image data loss area caused by electromagnetic interference, and mapping the point cloud coordinates to an image coordinate system to obtain a three-dimensional geometric complemented image; and constructing a dynamic occupation grid map of the underground environment according to the three-dimensional geometric complementation image and the obstacle position confidence map, and planning an obstacle avoidance path of the mining equipment according to the dynamic occupation grid map. And the obstacle avoidance capability of the mining equipment is improved, and safe and efficient coal mining operation is guaranteed.

An agricultural image analysis system comprising at least one vision sensor configured to view a seed trench; a storage module in communication with the at least one vision sensor; a processor in communication with the storage module, the processor executing at least one machine learning module for analysis of images from the at least one vision sensor. The system including at least one laser configured to emit a beam at an open seed trench and at least one vision sensor configured to view the open seed trench and the beam. The system including a thermal camera mounted to a row unit.

The invention belongs to the technical field of intelligent manufacturing and laser arc hybrid welding, and discloses a double-robot laser arc hybrid welding real-time regulation and control method based on digital twinning. According to the method, double robots are used, a laser-electric arc composite heat source digital twinborn model is constructed, a Gaussian distribution laser heat source and a double-ellipsoid electric arc heat source are integrated, and the dynamic behaviors (molten pool flowing and keyhole stability) of a molten pool and the generation probability of defects (air holes, cracks and the like) are predicted. And a visual sensor and an acoustic emission sensor on the two sides of the welding robot are used for capturing molten pool oscillation frequency and internal defect signals. And in combination with a reinforcement learning algorithm, welding path offset self-adaptive compensation and laser arc energy ratio adjustment are achieved, and closed-loop real-time regulation and control are formed by controlling the position of a welding gun of a welding robot and technological parameters. Based on a digital twinborn model, robot welding guns on the two sides can conduct synchronous welding on the two sides and can be independently adjusted while coupling is guaranteed according to the characteristics of specific materials, and high-quality welding is achieved.

The invention discloses a forestry intelligent spraying system and spraying method based on multi-source sensor space perception, and relates to the technical field of intelligent control. A target forest region is scanned through a laser radar and a visual sensor carried by an unmanned aerial vehicle, a three-dimensional forest region map is constructed, and a single tree is identified and positioned by using a tree body identification model; evaluating the leaf density of the canopy; identifying diseases and insect pests by utilizing multispectral imaging, and making a pesticide proportioning decision according to the disease and insect pest types and severity; planning and generating an optimal flight path for spraying of the unmanned aerial vehicle; adjusting nozzle parameters according to the leaf density of the canopy and the severity of diseases and pests, performing variable spraying, and storing the spraying operation parameters of the unmanned aerial vehicle. Through multi-source sensor fusion, an intelligent decision algorithm and precise spraying control, autonomous obstacle avoidance, high-precision map construction, tree body recognition and canopy analysis, pest and disease damage detection and dynamic pesticide dispensing and spraying in a forestry scene are realized, the forestry spraying efficiency is remarkably improved, and pesticide waste is reduced.

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.

The invention discloses an automatic dovetail welding seam grinding control method based on robot visual positioning, and relates to the technical field of visual positioning. The method comprises the steps that after welding is completed, a dovetail welding seam image is collected and converted into a three-dimensional coordinate through a vision algorithm, and a three-dimensional point cloud is generated; smooth interpolation is performed on the three-dimensional point cloud through a B spline curve method to obtain a parameterized curve, and a control point set is optimized through a genetic algorithm to generate a global optimal polishing path; the polishing robot executes a task according to a path, a tail end sensor collects a real-time path and calculates a deviation value with a global path, and when the deviation value exceeds a preset threshold value, inverse kinematics is triggered to solve and correct the path; the hardness of the dovetail weld is measured through laser-induced breakdown spectroscopy, a comprehensive hardness value is obtained in combination with a matrix hardness database, meanwhile, a visual sensor collects the surface state, and polishing process parameters are dynamically adjusted according to the surface state; after the task is completed, the welding seam angle and flatness are detected. According to the method, the optimal path is generated through visual positioning, and automatic grinding of the dovetail welding seam is achieved.

The invention provides a map construction method and system based on laser vision dynamic weighted fusion, and belongs to the technical field of data processing, and the method comprises the steps: obtaining an RGB image and a depth image through a vision sensor, and obtaining point cloud data through a laser radar; aligning the depth image with the RGB image, and removing invalid depth pixels; based on an ORB-SLAM2 algorithm, performing visual SLAM, extracting features of the depth image and the RGB image, and outputting sparse visual point cloud data; based on a Gmapping algorithm, performing laser radar SLAM, and outputting a 2D occupied grid map; projecting the sparse visual point cloud data into a 2D occupation grid map, and calculating the semantic occupation probability of each grid; calculating the geometric occupancy probability of each grid through an anti-sensor model; performing weighted fusion on the semantic occupancy probability and the geometric occupancy probability, and calculating a fusion occupancy probability; and generating a fusion map according to the fusion occupation probability.

The embodiment of the invention provides a mechanical arm insertion and extraction control method and device, test equipment and a storage medium, and the method comprises the steps that sensor data for sensing the working environment of a mechanical arm is received, and the working environment is the environment where the mechanical arm executes insertion and extraction operation on a target interface; according to the sensor data, the tail end acting force of a tail end executor of the mechanical arm and the pose of the target interface are determined; and under the condition that any one of the tail end acting force and the pose of the target interface is abnormal, control parameters of the mechanical arm are adjusted. According to the embodiment of the invention, the detection results of the visual sensor and the force / torque sensor are fused, the process of executing the plugging operation of the mechanical arm is monitored in real time, and once the target interface pose or the tail end acting force is abnormal, the motion track of the mechanical arm is immediately corrected and the insertion force is adjusted, so that self-adaptive adjustment is realized; and the success rate and the automation level of the plugging task are effectively improved.

The invention relates to the technical field of automatic manufacturing, and discloses a metal pipe part welding path intelligent planning and monitoring system which comprises a path planning module used for generating an optimal welding path according to the geometrical shape of a metal pipe part, welding process requirements and sensor feedback information; the sensor module monitors key parameters in the welding process in real time through a visual sensor, a thermal imaging sensor and a force sensor; the data processing and analyzing module is used for processing and analyzing the real-time data acquired by the sensor; the welding control module is used for controlling operation of welding equipment according to path planning and sensor feedback data; and the quality monitoring and feedback module is used for monitoring the quality in the welding process in real time. According to the system, the welding path is generated through the intelligent path planning module, and the path is dynamically optimized in combination with real-time sensor data, so that the accuracy of the welding process is improved, the system can adapt to changes in a complex welding environment, and the welding efficiency and quality are improved.

The invention discloses a radar and vision multi-mode fused three-dimensional multi-target tracking method and related equipment. The method comprises the following steps: establishing a three-dimensional constant turning rate and speed motion model for detecting a target vehicle; constructing a multi-modal measurement model, and obtaining a fusion target measurement result according to the point cloud data and the image data collected by the 4D millimeter wave radar; performing two-stage front and back frame matching and filtering according to a fusion target measurement result to obtain a matching target of the track; track life cycle management: newly building, confirming, keeping or deleting the track according to the matching result and the time step; according to the rule after track re-tracking, under the condition that the target is temporarily lost or shielded, a virtual track in the shielding period is constructed, state correction is conducted on the virtual track, the corrected state serves as the initial state of the current moment, and tracking continues. According to the invention, through fusion of complementary information of the 4D millimeter wave radar and the visual sensor, accurate sensing and tracking of three-dimensional multiple targets in a complex automatic driving environment are realized.

Disclosed are an apparatus, system, and method of a visual sensor of a tactical gear to use facial recognition technology to identify a person of interest and to cause a responsive device on the tactical gear to notify a wearer. In one embodiment, a personal protective equipment includes a responsive device integrated in a tactical gear and a visual sensor. The visual sensor of the tactical gear identifies a target person using an identity artificial intelligence model. The responsive device notifies a wearer of the tactical gear when the visual sensor identifies the target person. Further, the responsive device notifies an additional person when the visual sensor identifies a threat to a protectee of the wearer and the additional person. The responsive device may vibrate when the visual sensor of the tactical gear detects an ambient threat to the protectee of the wearer.

The invention relates to a display screen direction dynamic adjusting system and method based on user behaviors, and the system comprises a movable robot platform which is configured to achieve indoor autonomous movement through a bottom drive mechanism which comprises a high-precision motion assembly suitable for various ground materials; the six-degree-of-freedom mechanical arm module is vertically mounted at the top of the robot platform, and a display screen mounting interface is arranged at the tail end; the intelligent sensing module, the integrated infrared sensor array, the nine-axis inertial measurement unit and the depth vision sensor are configured to capture user biological characteristics and environment space data in real time, and through the movable robot platform and the six-degree-of-freedom mechanical arm module, autonomous movement and three-dimensional pose accurate adjustment of a display screen are achieved; and in combination with a machine learning algorithm and a sight tracking technology of the adaptive control module, highly autonomous and accurate user interaction experience is provided.

The invention relates to a shipbuilding safety monitoring system and a monitoring method thereof, and belongs to the technical field of shipbuilding monitoring, and the system comprises a multi-modal data collection module, an edge computing node, a digital twin platform, a digital twin platform, an analysis module, a block chain traceability module, and a feedback control module. The multi-modal data acquisition module comprises a visual sensor, an acoustic sensor, a thermal imager, a strain sensor, a laser displacement sensor and an environment sensor. The visual sensor, the acoustic sensor, the thermal imager, the strain sensor and the environment sensor are installed in a shipbuilding workshop. According to the invention, by fusing visual, acoustics, thermodynamics and other multi-sensor data and combining edge calculation real-time processing and digital twinning dynamic simulation, process quality prediction, defect detection and carbon emission tracking are realized, and the problems of insufficient real-time performance, difficulty in cross-system collaboration and lack of green manufacturing monitoring in the prior art are solved.

The invention discloses a flexible robot motion control system and method based on a visual language action model, and the system comprises visual sensors which are disposed at a plurality of joints of a flexible robot, and are used for observing the multi-joint sensing information of the flexible robot; the remote voice input unit is installed on the flexible robot body and used for inputting a voice instruction of remote operation to the flexible robot; the motion control module based on the VLA framework is mounted in the flexible robot and used for controlling the flexible robot to act based on a visual language action model according to the multi-joint sensing information and the voice instruction; through the design of the intelligent control system, the complexity of the system is remarkably reduced, the control precision and flexibility are optimized, the coordination control problem of complex body segment units is solved, high-performance autonomous motion control is achieved, the system cost is reduced, the adaptability of the robot in diversified environments is enhanced, and the robot can be widely deployed in complex application scenes.

The invention particularly relates to an intelligent security and protection dynamic risk assessment and prevention and control system based on multi-source data fusion, and the system comprises a data collection module which collects corresponding data and carries out the preprocessing of the data; the data processing module is used for analyzing the data to obtain a risk assessment coefficient; the risk assessment module is used for obtaining a risk level based on the risk assessment coefficient and making a corresponding prevention and control strategy; and a prevention and control decision optimization module. According to the invention, the data acquisition module integrates sensor data, service system data and external data, and realizes omnibearing data coverage of a security scene; real-time data collected by a visual sensor, an environment sensor and the like are combined with business system data of access control, vehicle management and the like, then external information of meteorology, public security, public opinions and the like is fused, feature extraction and weight distribution are carried out by using technologies of deep learning, an entropy weight method and the like through a data processing module, and finally an accurate risk assessment coefficient is obtained. More risk factors are comprehensively considered, and potential safety hazards are effectively identified.

The present disclosure provides a method for generating annotation data for robotic training using a hierarchical transformer-based model with multiple layers. The transformer-based model includes Alpha models generating low-level control outputs and Beta models generating high-level control outputs. The method receives multimodal input data comprising visual sensor data and natural language instructions, processes this data through the hierarchical transformer-based model to generate annotations at different abstraction levels, wherein Beta models create semantic annotations describing task objectives and Alpha models generate motor command annotations specifying robotic actions, and stores these annotations with the input data to create annotated training data for robotic control systems.

The invention discloses an intelligent welding control system and method based on multiple sensors and electronic equipment, and belongs to the technical field of automatic welding, the system comprises a sensing layer, a decision-making layer and an execution layer, through cooperative work of a front laser vision sensor, a rear laser vision sensor and a molten pool image sensor, a groove three-dimensional model can be accurately established before welding, and the welding precision is improved. And dynamic information of the molten pool is continuously collected in the welding process, a dual monitoring mechanism for the geometric characteristics of the welding line and the state of the molten pool is formed, and multi-source data collection and fusion in the whole welding process are achieved. The control method comprises the steps of pre-scanning modeling, feedforward parameter planning, real-time feedback adjustment of a molten pool, data fusion optimization and the like, and high-precision self-adaptive control over welding parameters is achieved by combining deep learning and the Kalman filtering technology. According to the method, the welding adaptability, the control precision and the intelligent level can be remarkably improved, and the method is suitable for high-precision welding scenes such as pipelines, pressure containers and steel structures and has remarkable engineering application value and popularization prospects.

The invention discloses a photovoltaic cleaning control method fusing RTK positioning and visual compensation, and belongs to the technical field of photovoltaic module cleaning. According to the invention, through the RTK track points acquired offline, the least square method is combined with the RANSAC algorithm to carry out segmented straight line fitting, and the original points are projected to the corresponding straight lines to realize collinear correction, so that the measurement error is eliminated, and a continuous and collimated navigation path can be generated; edge lines (photovoltaic module grid lines) with geometrical characteristics in the environment are detected in real time through a visual sensor, the line segment direction is extracted through Hough transform, the line segment direction is compared with the target course calculated through RTK, course deviation is estimated, dynamic correction is conducted through a control strategy, and it can be ensured that the photovoltaic cleaning robot operates along the expected path with high precision; and on the basis of a lightweight YOLOv7 model, the photovoltaic panel is subjected to pollution category division and mapped to optimal water pressure and brush speed parameters, cleaning parameters are dynamically modified, and then cleaning strategy control is completed.

A pixel array includes photodiodes and a color filter array. A first fraction of the photodiodes is included in CIS pixels and a second fraction of the photodiodes is included in hybrid CIS / EVS pixels. The photodiodes are arranged into groupings. The color filter array includes first, second, and third color filters arranged in a mosaic pattern over the photodiodes. Each grouping includes a plurality of subgroupings including a first subgrouping disposed under at least one of the first color filters, a second subgrouping disposed under at least one of the second color filters, and a third subgrouping disposed under at least one of the third color filters. Each of the first, second, and third subgroupings includes at least one CIS pixel, and at least one of the first subgrouping of photodiodes further includes at least one hybrid CIS / EVS pixel disposed under at least one of the first color filters.

The invention discloses an industrial robot intelligent obstacle avoidance control method and system based on visual identification, and relates to the technical field of robot obstacle avoidance control, and the method comprises the steps: carrying out the calibration of a multi-mode visual sensor, obtaining an initial depth map and point cloud information, and combining the visual identification and depth compensation technology; recognizing and positioning obstacles in the operation area, and constructing a dynamic environment map layer; and the current robot state is collected, kinematics calculation and collision distance analysis are carried out, whether obstacle avoidance operation needs to be executed or not is judged, if yes, an obstacle avoidance path is generated in combination with the dynamic environment map layer and the target point location, feasibility verification is carried out after the path is generated, and the path passing the verification serves as an execution track to be issued to the control module. According to the invention, the recognition precision and depth perception integrity of the industrial robot on obstacles in a complex environment are improved, high feasibility of path planning and high-reliability obstacle avoidance capability in a dynamic environment are realized, and the intelligent decision-making level and operation safety of the system are remarkably enhanced.

The invention belongs to the technical field of fusion welding, and particularly relates to a method for planning multi-layer and multi-welding-seam welding parameters according to welding seam feature information, which comprises the following steps: 1) laser emitted by a 3D visual sensor is perpendicular to a welding seam and scans along the direction of the welding seam, and the 3D visual sensor scans welding seam data and transmits the data to a robot controller in real time; (2) the robot controller processes the weld joint data, the whole weld joint is subjected to multi-layer and multi-pass planning calculation, and the number of welding layers and the number of welding passes of each layer are planned; (3) the robot controller calculates the layer number and the channel number of a welding channel needing to be welded currently, a welding gun track coordinate set on a welding path and welding parameters corresponding to welding path points; and (4) the robot controller adjusts the welding track and the welding parameters in real time according to the step (1) to the step (3) in the current welding bead welding process, so that the purpose of multi-layer and multi-pass planning self-adaptive welding of the medium-thickness plate is achieved.

The invention relates to the technical field of industrial robot control, and discloses a robot end trajectory optimization method and system based on industrial vision, and the method comprises the steps: obtaining a workpiece surface image in real time through a vision sensor, and obtaining deformation data through image enhancement and feature extraction; when the deformation quantity exceeds a threshold value, calculating a multi-axis coordination parameter by adopting an optimization algorithm to generate a trajectory correction instruction; integrating speed constraints by updating a control model, and determining a final trajectory by using Kalman filtering to fuse sensor feedback data; and the control parameters are iteratively adjusted in the deformation feedback loop, and a stable machining track is formed. The method can solve the problem that in the prior art, the robot tail end track precision is low.