2130 results about "Vision sensor" patented technology

A multimodal intelligent agent system for dynamic environmental monitoring and user-centered support, consisting of: a multimodal sensor module configured to continuously acquire environmental and behavioral data from multiple input modalities, including at least one visual sensor, at least one acoustic sensor, at least one environmental conditions sensor, and at least one proximity or motion detection sensor, each generating modality-specific data streams representing visual images, audio waveforms, physical environmental parameters, and motion signatures within a monitored environment; a data preprocessing and fusion subsystem that is operationally coupled with the multimodal sensor module and configured to normalize, temporally align, and transform the modality-specific data streams into high-dimensional feature embeddings using a variety of encoders, wherein the visual encoder uses convolutional or vision transformer architectures, the audio encoder uses a spectral-temporal feature extractor, and the sensor encoder transforms raw analog data into context vectors suitable for multimodal alignment; a multimodal processing unit consisting of a transformer-based large language model (LLM) trained on paired multimodal datasets and configured to perform semantic fusion, context abstraction, and inference across the aforementioned aligned multimodal feature embeddings to generate a contextual understanding of environmental and behavioral states; an adaptive agent controller coupled to the multimodal inference processing unit and configured to instantiate, manage, and terminate a variety of task-specific intelligent agents, each agent being a software unit configured to perform a specialized function selected from meeting summarization, behavioral analysis, misplaced object detection, or environmental anomaly identification, with the agents dynamically interacting with the inference engine to retrieve contextually relevant multimodal embeddings for task execution; a personalization and adaptive learning subsystem consisting of a user preference database and a neural memory structure configured to update and refine model parameters based on user-specific interaction history, thereby enabling personalized output generation, prioritization of recommendations, and long-term behavioral adaptation; and An output generation interface is operationally connected to the adaptive agent controller and configured to produce multimodal output in textual, visual, and auditory form. The interface is capable of displaying human-readable summaries, notifications, and visual reconstructions of identified entities or environmental states.

The invention discloses a mechanical arm dynamic deviation correction method and system based on visual driving and a medium, and relates to the technical field of mechanical arm control. The method comprises the steps that when the tail end of a mechanical arm enters a preset machining space, an integrated 3D visual sensor is triggered to collect 3D point cloud of a workpiece to be machined; after pose recognition is carried out on the point cloud, the offset is recognized according to the teaching pose and the actual pose, and the initial offset is output; calling the multi-dimensional perception data, performing fusion correction, and outputting a correction offset; performing interference correction through an offset compensation model, and outputting a target offset; parameter adjustment and optimization are carried out according to the target offset, and a joint angle adjustment instruction is output; and performing correction closed-loop feedback according to the updated pose data. The technical problems of precision errors and low efficiency caused by deviation in the operation process of the mechanical arm are solved, and the technical effects that through dynamic deviation correction and multi-sensor data fusion, the operation precision and efficiency of the mechanical arm are improved, and stable operation in a complex environment is ensured are achieved.

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.

The invention discloses a visual language navigation method for cross-modal alignment in a dynamic shielding environment, and the method comprises the steps: collecting multi-modal data through a visual sensor, an inertial measurement unit, a laser radar and the like, and carrying out the preprocessing and time synchronization; sensing the dynamic shielding object through a model composed of a convolutional neural network and a long-short-term memory network, and estimating the future change of the dynamic shielding object in combination with a space-time sequence prediction algorithm; a double-branch convolutional neural network and a Transform based on a dynamic attention mechanism are adopted to respectively extract visual and semantic features and fuse the visual and semantic features; on the basis of occlusion prediction, potential occlusion region features are extracted in advance from a time dimension, an occluded image is repaired by using a generative adversarial network and geometric constraints in a space dimension, and cross-modal feature alignment is optimized through an attention mechanism; planning a path by using a hybrid reinforcement learning algorithm based on a deep Q network-space and a fast exploration random tree, and dynamically adjusting according to real-time shielding; according to the method, the accuracy, adaptability and reliability of visual language navigation in a dynamic shielding environment are improved.

An agricultural image analysis system comprising at least one vision sensor configured to view a seed trench; a storage module in communication with the at least one vision sensor; a processor in communication with the storage module, the processor executing at least one machine learning module for analysis of images from the at least one vision sensor. The system including at least one laser configured to emit a beam at an open seed trench and at least one vision sensor configured to view the open seed trench and the beam. The system including a thermal camera mounted to a row unit.

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.

The invention discloses a radar and vision multi-mode fused three-dimensional multi-target tracking method and related equipment. The method comprises the following steps: establishing a three-dimensional constant turning rate and speed motion model for detecting a target vehicle; constructing a multi-modal measurement model, and obtaining a fusion target measurement result according to the point cloud data and the image data collected by the 4D millimeter wave radar; performing two-stage front and back frame matching and filtering according to a fusion target measurement result to obtain a matching target of the track; track life cycle management: newly building, confirming, keeping or deleting the track according to the matching result and the time step; according to the rule after track re-tracking, under the condition that the target is temporarily lost or shielded, a virtual track in the shielding period is constructed, state correction is conducted on the virtual track, the corrected state serves as the initial state of the current moment, and tracking continues. According to the invention, through fusion of complementary information of the 4D millimeter wave radar and the visual sensor, accurate sensing and tracking of three-dimensional multiple targets in a complex automatic driving environment are realized.

Disclosed are an apparatus, system, and method of a visual sensor of a tactical gear to use facial recognition technology to identify a person of interest and to cause a responsive device on the tactical gear to notify a wearer. In one embodiment, a personal protective equipment includes a responsive device integrated in a tactical gear and a visual sensor. The visual sensor of the tactical gear identifies a target person using an identity artificial intelligence model. The responsive device notifies a wearer of the tactical gear when the visual sensor identifies the target person. Further, the responsive device notifies an additional person when the visual sensor identifies a threat to a protectee of the wearer and the additional person. The responsive device may vibrate when the visual sensor of the tactical gear detects an ambient threat to the protectee of the wearer.

The invention relates to a shipbuilding safety monitoring system and a monitoring method thereof, and belongs to the technical field of shipbuilding monitoring, and the system comprises a multi-modal data collection module, an edge computing node, a digital twin platform, a digital twin platform, an analysis module, a block chain traceability module, and a feedback control module. The multi-modal data acquisition module comprises a visual sensor, an acoustic sensor, a thermal imager, a strain sensor, a laser displacement sensor and an environment sensor. The visual sensor, the acoustic sensor, the thermal imager, the strain sensor and the environment sensor are installed in a shipbuilding workshop. According to the invention, by fusing visual, acoustics, thermodynamics and other multi-sensor data and combining edge calculation real-time processing and digital twinning dynamic simulation, process quality prediction, defect detection and carbon emission tracking are realized, and the problems of insufficient real-time performance, difficulty in cross-system collaboration and lack of green manufacturing monitoring in the prior art are solved.

The present disclosure provides a method for generating annotation data for robotic training using a hierarchical transformer-based model with multiple layers. The transformer-based model includes Alpha models generating low-level control outputs and Beta models generating high-level control outputs. The method receives multimodal input data comprising visual sensor data and natural language instructions, processes this data through the hierarchical transformer-based model to generate annotations at different abstraction levels, wherein Beta models create semantic annotations describing task objectives and Alpha models generate motor command annotations specifying robotic actions, and stores these annotations with the input data to create annotated training data for robotic control systems.

The invention discloses an intelligent welding control system and method based on multiple sensors and electronic equipment, and belongs to the technical field of automatic welding, the system comprises a sensing layer, a decision-making layer and an execution layer, through cooperative work of a front laser vision sensor, a rear laser vision sensor and a molten pool image sensor, a groove three-dimensional model can be accurately established before welding, and the welding precision is improved. And dynamic information of the molten pool is continuously collected in the welding process, a dual monitoring mechanism for the geometric characteristics of the welding line and the state of the molten pool is formed, and multi-source data collection and fusion in the whole welding process are achieved. The control method comprises the steps of pre-scanning modeling, feedforward parameter planning, real-time feedback adjustment of a molten pool, data fusion optimization and the like, and high-precision self-adaptive control over welding parameters is achieved by combining deep learning and the Kalman filtering technology. According to the method, the welding adaptability, the control precision and the intelligent level can be remarkably improved, and the method is suitable for high-precision welding scenes such as pipelines, pressure containers and steel structures and has remarkable engineering application value and popularization prospects.

The invention discloses a photovoltaic cleaning control method fusing RTK positioning and visual compensation, and belongs to the technical field of photovoltaic module cleaning. According to the invention, through the RTK track points acquired offline, the least square method is combined with the RANSAC algorithm to carry out segmented straight line fitting, and the original points are projected to the corresponding straight lines to realize collinear correction, so that the measurement error is eliminated, and a continuous and collimated navigation path can be generated; edge lines (photovoltaic module grid lines) with geometrical characteristics in the environment are detected in real time through a visual sensor, the line segment direction is extracted through Hough transform, the line segment direction is compared with the target course calculated through RTK, course deviation is estimated, dynamic correction is conducted through a control strategy, and it can be ensured that the photovoltaic cleaning robot operates along the expected path with high precision; and on the basis of a lightweight YOLOv7 model, the photovoltaic panel is subjected to pollution category division and mapped to optimal water pressure and brush speed parameters, cleaning parameters are dynamically modified, and then cleaning strategy control is completed.

The invention discloses an industrial robot intelligent obstacle avoidance control method and system based on visual identification, and relates to the technical field of robot obstacle avoidance control, and the method comprises the steps: carrying out the calibration of a multi-mode visual sensor, obtaining an initial depth map and point cloud information, and combining the visual identification and depth compensation technology; recognizing and positioning obstacles in the operation area, and constructing a dynamic environment map layer; and the current robot state is collected, kinematics calculation and collision distance analysis are carried out, whether obstacle avoidance operation needs to be executed or not is judged, if yes, an obstacle avoidance path is generated in combination with the dynamic environment map layer and the target point location, feasibility verification is carried out after the path is generated, and the path passing the verification serves as an execution track to be issued to the control module. According to the invention, the recognition precision and depth perception integrity of the industrial robot on obstacles in a complex environment are improved, high feasibility of path planning and high-reliability obstacle avoidance capability in a dynamic environment are realized, and the intelligent decision-making level and operation safety of the system are remarkably enhanced.

The invention relates to the technical field of industrial robot control, and discloses a robot end trajectory optimization method and system based on industrial vision, and the method comprises the steps: obtaining a workpiece surface image in real time through a vision sensor, and obtaining deformation data through image enhancement and feature extraction; when the deformation quantity exceeds a threshold value, calculating a multi-axis coordination parameter by adopting an optimization algorithm to generate a trajectory correction instruction; integrating speed constraints by updating a control model, and determining a final trajectory by using Kalman filtering to fuse sensor feedback data; and the control parameters are iteratively adjusted in the deformation feedback loop, and a stable machining track is formed. The method can solve the problem that in the prior art, the robot tail end track precision is low.

A visual tactile sensor, a manipulator and a control method thereof, the visual tactile sensor comprising: a contact layer, at least a portion of which is transparent; when the contact layer is subjected to external force, at least one part of a plurality of markers in the marker array generates displacement and / or deformation; and the image acquisition unit is used for performing image acquisition on the marker array and performing image acquisition on the outside through the transparent part of the contact layer. The visual tactile sensor provided by the invention not only can capture the external visual information before operation to identify the operation object, but also can provide the tactile information in the operation process, so that the perception capability of an operation system is enhanced, and the operation success rate is improved.

The invention discloses a metal processing detection system and method based on computer vision, and relates to the technical field of defect detection.Rotatable polarization filters are installed in front of a lens of a vision sensor, the polarization filters are arranged at four angles respectively, and four sets of polarization images are collected; calculating the polarization degree of each pixel point on the surface during metal processing by using the gray values of the four groups of polarization images, setting a polarization region interval, and dividing a metal surface reflection region into a specular reflection region and a diffuse reflection region; acquiring images of different wavebands by using a multispectral vision sensor, and constructing a multispectral image set; for the same wave band, the images of different exposure times are synthesized to obtain an HDR high dynamic range image; setting a weight for the HDR image of each wave band, and combining the weighting of the HDR image and the suppression of the polarization degree to carry out multispectral image fusion so as to obtain a final image; and a gray threshold value and an area threshold value are respectively calculated through the average value and the standard deviation, and the metal processing defects are judged by using the two threshold values.

The invention belongs to the technical field of inspection robot systems, and particularly relates to an inspection robot path planning method and system.The visual semantic perception module collects an industrial environment image through a top industrial camera, after graying and Gaussian filtering preprocessing, feature points are detected and matched through an ORB algorithm, and a path planning result is obtained; in combination with an illumination self-adaptive threshold screening mechanism, mismatching points are eliminated, semantics are marked, and a semantic feature map is constructed; the initial path planning module generates an initial path through a semantic cost-containing A * algorithm based on the map; the dynamic obstacle avoidance module captures a moving obstacle by using a visual sensor, and predicts a trajectory through Kalman filtering; the path optimization module combines an initial path and an obstacle track, optimizes the path by using quadratic programming of a fusion curvature constraint and an energy consumption model, and corrects positioning by fusing vision and IMU data through a dynamic weight fusion algorithm; and the execution feedback module generates an instruction according to the optimized path, re-triggers path optimization, forms a closed loop, and ensures the inspection stability.

The present disclosure provides a method for calibrating a humanoid robot, comprising obtaining a humanoid robot with original kinematic biasing values, controlling the humanoid robot through predetermined poses, capturing image data of body parts using vision sensors mounted on the humanoid robot while moving through the poses, determining revised kinematic biasing values by processing the image data using a bipedal spatial perception model trained using synthetic image data containing keypoints, and replacing the original kinematic biasing values with the revised kinematic biasing values. The bipedal spatial perception model processes captured image data to generate observed keypoint locations on robot components, which are compared with kinematic-based locations from joint encoder measurements to minimize discrepancies through optimization algorithms.

The invention discloses a deep reinforcement learning robot-assisted ultrasonic scanning system and method based on security constraints. The invention discloses a deep reinforcement learning mechanical arm ultrasonic scanning method and system based on security constraints. A scanning system comprising a mechanical arm, an ultrasonic probe, a joint torque and visual sensor and a control unit is constructed. The environment state space covers the joint angle, the tail end pose, the contact force, the scanning path and the safety state variable of the mechanical arm, and the action space is a tail end linear speed instruction of the mechanical arm. A deep reinforcement learning model is built based on a near-end strategy optimization (PPO) algorithm, and a reward function including image quality, task completion reward and multiple penalty terms is designed. A control barrier function (CBFs) is used for implementing safety constraint on motion of the mechanical arm, and the safety constraint is fused into deep reinforcement learning training through a penalty function method. A model is trained in a simulation environment simulating an actual scanning scene and then deployed to an actual system, and a mechanical arm is controlled to scan and monitor safety constraints according to a real-time environment state. The mechanical arm can efficiently execute the ultrasonic scanning task under the condition that the safety constraint is met, and a reliable scheme is provided for ultrasonic scanning automation. Meanwhile, the system is provided with a data acquisition module, a state construction module, a reward calculation module, a security constraint module, a control module and the like, and the model performance can be improved through simulation training.

The invention belongs to the technical field of robot navigation, and particularly relates to a humanoid robot navigation method based on visual semantic segmentation and radar obstacle detection, which comprises the following steps: synchronously acquiring an RGB image and a depth image of a current environment of a humanoid robot by using a visual sensor, and acquiring point cloud data by using a laser radar; performing semantic segmentation on the preprocessed RGB image; radar point cloud obstacle detection; fusing the semantic segmentation map and the laser point cloud map, introducing a Bayesian decision to judge whether the laser point cloud map is passable or not, and then calculating a fusion cost value to obtain a fusion cost map; adopting an RRT * / TEB algorithm to output an optimal path; using a nonlinear optimization solver to generate a foothold sequence accurate to each step; according to the method, a semantic-geometric two-dimensional navigation cost model is constructed; bayesian reasoning is deeply bound with a navigation scene, so that the navigation adaptability of an unstructured environment is improved; the navigation method is suitable for the humanoid robot, and is low in energy consumption, low in navigation deviation and high in safety.

The invention provides a multi-agent collaborative visual completion method and system based on semantic communication, and the method comprises the steps: a first agent collects an occluded overall original image through a visual sensor, generates an occluded region mask through a lightweight semantic segmentation network, extracts mask semantic information, and transmits the mask semantic information to a second agent; the second intelligent agent matches the global semantic feature of the second intelligent agent with the received mask semantic information, locates a sheltered area, then cuts the sheltered area to obtain a semantic feature vector of the area, and returns the semantic feature vector to the first intelligent agent; and after the first intelligent agent receives the image, performing high-fidelity reconstruction on the occlusion area by using a conditional diffusion model to obtain a complemented image with consistent semantics, and performing weighted fusion and local color matching and splicing on the complemented image and the original image to obtain a global perception result. According to the scheme, perceptual information sharing and occlusion area reconstruction with low bit rate, high robustness and high real-time performance can be realized under limited communication resources.

The invention discloses an unmanned aerial vehicle autonomous address selection method based on terrain evaluation and approach cost analysis, and relates to the technical field of unmanned aerial vehicle autonomous address selection. After the unmanned aerial vehicle receives a landing instruction in an unknown environment, the geometric safety of the terrain and the cost problem of the approach path can be comprehensively considered, and the landing point selection strategy of the unmanned aerial vehicle is optimized from the global perspective. Firstly, semantic segmentation is performed on an RGB image acquired by a visual sensor through deep learning to acquire a semantic image, then point cloud data acquired by a laser radar and the semantic image are fused to acquire a three-dimensional semantic point cloud, and a terrain safety area conforming to terrain geometric feature constraints is screened out on the basis of considering semantic tags. And finally, calculating the cost of the linear approach path of each candidate landing point in the terrain safety area through traversal, and screening out the landing point with the optimal comprehensive cost, thereby improving the accuracy and practicability of terrain safety area identification. According to the method, an unreachable or high-risk area is avoided, and the global point selection optimization effect is improved.

The invention discloses a fan blade clearance intelligent monitoring method based on multi-modal fusion. A laser radar and a visual sensor are adopted to collect point cloud data and image data of the fan blade; environment parameters are collected, validity judgment is conducted on the point cloud data and the image data of the fan blade according to the environment parameters, and radar monitoring data and visual monitoring data are obtained; modeling the fan blade according to the radar monitoring data to obtain a three-dimensional model of the fan blade, and further obtaining a radar measurement value of the blade clearance according to the three-dimensional model of the fan blade; performing image segmentation, edge extraction and depth estimation on the visual monitoring data in sequence to obtain a visual measurement value of the blade clearance; and the radar measurement value and the vision measurement value of the blade clearance are combined, and a multi-modal fusion algorithm is adopted for processing to obtain the fusion clearance distance of the fan blade. The accuracy and real-time performance of fan blade clearance monitoring are remarkably improved, blade collision accidents can be effectively prevented, and reliable guarantee is provided for safe operation of a wind turbine generator.

The invention provides an unmanned aerial vehicle intelligent decision-making method and system based on deep reinforcement learning, and relates to the technical field of intelligent decision-making, and the method comprises the steps: obtaining a landing region image and depth information through a visual sensor, processing feature representation and three-dimensional environment description through a risk perception type strategy gradient algorithm, and adaptively constructing an information association relationship. According to the method, feature fusion, optimal landing position determination, dynamic obstacle trajectory prediction and track point sequence generation are realized, a control strategy is optimized when the environment is suddenly changed, safe and efficient landing of the unmanned aerial vehicle is realized, and the safety and adaptability of landing of the unmanned aerial vehicle are improved.

The invention discloses a numerical control machine tool machining track real-time correction method based on multi-source visual perception, and the method comprises the following steps: S1, synchronously collecting a multi-source real-time image sequence of a machined workpiece in a machining process through a plurality of visual sensors disposed at different visual angles of a machining region of a numerical control machine tool; s2, performing preprocessing and feature extraction on the multi-source real-time image sequence, and fusing to obtain real-time three-dimensional coordinate information of key feature points, corresponding to the theoretical processing track model, on a processing workpiece; s3, the real-time three-dimensional coordinates of the key feature points are compared with theoretical coordinates of corresponding points in a pre-stored numerical control machine tool theoretical machining track model, and the track deviation between the actual machining track and the theoretical machining track is obtained through calculation; and S4, based on the trajectory deviation, generating a trajectory correction value through a preset correction algorithm. The method can better correct the machining track of the numerical control machine tool in real time.

The invention provides an unmanned aerial vehicle autonomous obstacle avoidance and dynamic path planning method and system based on multi-modal perception and hybrid intelligent decision. Environmental parameters such as wind speed, illuminance, temperature and humidity and image quality indexes are collected in real time through an airborne weather station, an IMU and a visual sensor, a flight parameter-image quality coupling model is established, and multi-target optimization is achieved through support vector regression (SVR) and particle swarm optimization (PSO). And designing a dynamic strategy optimization module based on Q-learning, and designing a reward function in combination with image quality, obstacle avoidance safety and energy consumption. An obstacle three-dimensional model is constructed in real time through ORB-SLAM3, a dynamic danger coefficient is calculated, and an obstacle avoidance track is generated by adopting an improved APF-RRT * algorithm. And the online cooperative control module optimizes the control quantity by using a BFGS algorithm so as to ensure the flight stability and the task efficiency. The method realizes high-precision obstacle avoidance and path planning of the unmanned aerial vehicle in a complex environment, has the characteristics of high robustness and wide adaptability, and is suitable for practical application scenes such as routing inspection, surveying and mapping and the like.

The invention discloses a high-precision ink path bubble detection and rapid circulation compensation method and a related device. The method comprises the following steps: inputting visual sensor data, ultrasonic data and pressure fluctuation data into a trained ink path bubble semantic segmentation network model for segmentation; calculating the equivalent diameter, the bubble volume, the bubble density, the bubble flow velocity and the category of the bubbles according to the binary mask sequence; inputting the equivalent diameter, the bubble volume, the bubble density, the bubble flow velocity and the category of the bubbles into the trained multi-agent collaborative decision network model, and outputting dynamic cyclic compensation and prevention actions; and according to the dynamic circulation compensation and prevention action, the ink path pumping pressure, the valve opening degree and the ink temperature are adjusted in real time, and a high-frequency micro-pulse exhausting, reverse suction or vibration bubble removing strategy is triggered. Through high-precision detection and intelligent compensation, bubbles are effectively removed, nozzle blockage and cleaning frequency are reduced, and therefore the downtime of equipment is shortened, and waste of ink and consumables is reduced.

The invention discloses a pipeline flange butt joint method based on multi-sensor cooperation and a storage medium, and the method comprises the following steps: when a mechanical arm carries a movable pipeline to approach a fixed pipeline, a magnetic encoder is utilized to detect the axial relative deflection angle of the pipeline, and the mechanical arm is controlled to adjust the axial posture of the movable pipeline to realize axial alignment. Under the axial alignment condition, the mechanical arm moves horizontally, the laser distance measuring sensor measures the linear distance, and meanwhile the axial deviation is continuously corrected by combining data of the magnetic encoder. And after the linear distance meets a preset threshold value, the industrial camera collects images of the movable pipeline flange and the fixed pipeline flange, a bolt hole coordinate set is recognized through an image processing algorithm, the rotating angle of the mechanical arm is calculated, the mechanical arm is controlled to rotate, bolt holes of the two flanges are accurately aligned, and butt joint is completed. According to the method, precise positioning of the pipeline flange in the axial direction, the plane direction and the rotating direction is achieved through closed-loop cooperation of the magnetic encoder, the laser ranging device and the visual sensor, and the butt joint precision and reliability are improved.

The invention provides an intelligent infusion monitoring method and system based on multi-sensor fusion, and relates to the technical field of intelligent infusion monitoring. The method comprises the following steps: acquiring infusion process data through a visual sensor, a weight sensor and an infrared sensor; aligning the time sequence labels and synchronously generating a fusion data stream; extracting a liquid level height characteristic, a weight change rate and a real-time dripping speed value, and carrying out dynamic weighting calculation; acquiring infusion doctor's advice data of a hospital information system, and extracting a preset dripping speed range and a total liquid amount; calculating the residual infusion time based on the weighted data and the total liquid amount correlation; when the weighted real-time dripping speed value deviates from a preset range, the weighted weight change rate is not matched with the liquid level height trend or the dripping speed pulse signal is lost, an early warning instruction is generated; and the remaining time and the early warning state are visually displayed at the nurse station terminal. Through multi-source data fusion and cross validation, the problems of incomplete monitoring, time sequence dislocation and early warning lag are solved, and infusion safety and management efficiency are improved.