42 results about "Robot perception" patented technology

The application discloses a robot complex scene perception system and method based on environment adaptive multi-modal fusion, relates to the technical field of robot perception, and comprises four modules of a flexible sensing matrix, environment feature self-calibration, cross-modal causal reasoning and scene prediction.The flexible sensing matrix collects environment parameters and multi-modal data of vision, inertia and touch.The self-calibration module corrects data errors through environment-error mapping and gradient descent algorithm.The causal reasoning module realizes data fusion and failure completion based on a Bayesian network.The scene prediction module outputs a prediction result through a pre-trained LSTM model and feeds back an optimized collection strategy.The application breaks through the bottleneck of traditional perception technology, improves the environment adaptability, system fault tolerance and decision initiative of complex unstructured scenes, is suitable for scenes such as disaster rescue and outdoor inspection, and helps robots to work autonomously and safely.

The present application relates to the field of ocean engineering and intelligent detection technology, specifically to a kind of multi-robot collaborative underwater oil and gas pipeline weld intelligent detection method and system, it includes obtaining and fusing each robot perception and task state data, assesses communication and perception ability, predicts task load trend and optimizes scheduling, adjusts multimodal fusion priority, re-plans task path and optimizes execution strategy, generates collaborative detection feedback data.The present application improves the collaborative ability of multi-robot in communication limited and ocean current disturbance environment through the dynamic task scheduling mechanism of global collaborative state perception;Combined with the time sequence alignment of multimodal sensing data and the dynamic adjustment of fusion priority, the stable recognition ability in complex underwater environment is enhanced;Through the joint optimization of task path planning and data fusion strategy, the detection coverage and defect recognition accuracy are improved, and the adaptability and reliability of the system in deepwater long-distance pipeline weld detection are enhanced.

The application discloses a kind of collaborative robot system and service process for active power grid marketing, belong to artificial intelligence and electric power marketing service technical field.System includes: robot perception interaction end, cloud intelligent analysis hub and dynamic customer memory bank, robot perception interaction end deploys lightweight vision model, real-time identification customer emotion, body state and explicit feature, and initiatively trigger service;Cloud intelligent analysis hub uses power grid knowledge enhanced large language model, carries out depth demand analysis and individualized strategy generation to multi-modal data;Dynamic customer memory bank stores customer historical interaction record and portrait information in the form of knowledge graph, realizes the continuity and memory of service.The application is through edge small model+cloud big model collaborative architecture, general AI capability and power grid business knowledge are deeply fused, so that robot can actively identify customer demand, provide marketing service, improve customer experience and marketing conversion rate of electric power business hall.

This invention discloses a robot sensor calibration method, apparatus, robot, and storage medium. The method includes: determining the transformation relationship between the calibration board coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the robot's radar sensor on the calibration board and the attribute information of the calibration board; determining the second position of the target label in the radar coordinate system based on the first position of the target label in the calibration board coordinate system and the transformation relationship; determining the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor on the calibration board; and calibrating the image sensor and the radar sensor based on the second and third positions. The technical solution of this invention can determine a more accurate transformation relationship between the calibration board coordinate system and the radar coordinate system, improving the performance of the robot's perception system and providing a new scheme for robot sensor calibration.

The application discloses a kind of fusion ToF and monocular vision's robot joint layer perception system, belong to robot perception field, system uses Eye-in-Hand architecture, integrates ToF camera and RGB camera;Through joint bilateral filtering and Kalman filtering, improve depth map quality;Based on SURF feature matching and the ICP registration of KD-Tree acceleration is realized target six degree of freedom pose estimation;Combined with improved RRT algorithm and hierarchical collision detection planning capture path.The robot joint layer perception system of fusion ToF and monocular vision provided in the application, through the space alignment and fusion of two kinds of visual information, make up for respective limitations, improve the robustness and precision of overall perception system;In single-arm and double-arm capture experiment, system positioning accuracy reaches ±3mm, dynamic scene capture success rate is high.It is suitable for industrial automation, logistics sorting and the like.

The application relates to the field of intelligent agricultural machinery and automatic driving technology and discloses a livestock industry feed pushing robot feed pushing control system based on deep learning. The system comprises a perception layer, a decision layer and an execution layer. The perception layer fuses a laser radar, ultrasonic waves and a camera; an edge computing unit runs an instance segmentation model to fit a feed edge; the decision layer utilizes an extended Kalman filtering algorithm containing a dynamic adjustment mechanism of an observation noise covariance matrix, adjusts a weight in real time according to a satellite signal quality, realizes seamless navigation indoors and outdoors, and executes a double closed loop PID strategy of a visual position loop and a torque limiting loop, and adaptively adjusts a pushing plate and a vehicle speed according to visual feedback and motor current. The application solves the problems of weak perception ability of the feed pushing robot, unstable navigation switching and poor operation flexibility, and realizes precise and efficient unmanned feed pushing.

The application discloses an indoor-oriented three-dimensional Gaussian diffusion model point cloud repairing method, expresses a missing point cloud as a three-dimensional point set, introduces a point cloud centroid as a spatial reference center, constructs a progressive-axis distance component of each point relative to the centroid, and calculates a second-order statistic in each axis direction as a directional scale signal; according to the deviation of the directional scale signal from a reference scale, the noise intensity is re-calibrated in each axis direction to form a diagonal covariance form of a directional adaptive three-dimensional Gaussian forward diffusion model; a denoising network is used to predict noise components in each diffusion step and update according to the progressive-axis diffusion intensity, while an observation consistency constraint is introduced; finally, a diffusion standard loss function is used to train the error between the real noise injected in the forward diffusion model and the predicted noise, and an optimized diffusion model is obtained. The method can be applied to indoor three-dimensional reconstruction, digital twinning, robot perception and indoor point cloud dataset enhancement scenes.

The application discloses a kind of based on double robot perception positioning's square tube fence variable parameter welding system and method, belong to welding automation technical field, it includes center control system, conveying and variable position mechanism, double machine welding execution mechanism, three-dimensional perception unit and welding power supply.This application identifies weld gap in real time by perception positioning unit, and according to gap size automatically matches expert database to adjust welding current, speed and swing amplitude;With single vertical square tube as basic welding unit, control double robot opposite synchronous welding, build symmetric heat field.This application can effectively guide galvanized steam directional escape, significantly reduce porosity defects, automatically compensate the gap fluctuation caused by blanking error, while ensuring weld forming consistency, inhibit structural deformation, greatly improve the production efficiency and intelligent level of square tube fence.

This invention relates to the field of intelligent service robot perception technology, and discloses an automatic generation system for tourist interpersonal interaction body language based on deep learning. The system acquires scene images through an image acquisition and skeleton extraction module and extracts the coordinate sequences of all two-dimensional skeletons of people, storing them in a database. A joint motion feature extraction module calculates the changes in joint coordinates frame by frame and inputs them into a trained deep learning model. An action category differentiation module classifies continuous actions into discrete actions such as waving, approaching, standing, leaving, and passing by. A depth information acquisition module simultaneously acquires distance information of people. A target service priority calculation module integrates action category, depth information, and action sequence to calculate and sort the service priority score for each person, generating a target service priority queue. This invention can automatically parse tourist interaction intentions and intelligently prioritize services.

This invention provides a radar inversion method and apparatus for robot perception. The method involves acquiring radar observation data, constructing a corresponding observation model, and calculating a data consistency metric. A local geometric consistency metric is calculated based on the observation data and the constructed weight matrix. A global geometric consistency metric is calculated based on the observation data and the constructed transmission matrix. The transmission matrix is ​​then optimized to obtain an optimal transmission matrix. Finally, a unified optimization model is constructed based on the data consistency metric, local geometric consistency metric, global geometric consistency metric, and optimal transmission matrix, and the inversion result is calculated. Within this unified optimization model framework, the signal observation model, the local geometric consistency in the radar observation data, and the global geometric consistency are jointly modeled to achieve stable estimation of target parameters, thereby improving the robot's perception stability and structural representation capability in complex environments.

This application relates to the field of robot perception and intelligent operation technology, and discloses an object recognition method based on visual-tactile-text multimodal fusion. The method includes acquiring multimodal input data of the object to be recognized, including visual, tactile, and text modal data; extracting visual, tactile, and text modal feature sequences based on the visual, tactile, and text modal data; employing a parameter-sharing multi-head cross-attention mechanism to obtain six sets of cross-modal fusion features; concatenating two sets of cross-modal fusion features corresponding to each modality in the visual, tactile, and text modal data to obtain enhanced visual, tactile, and text modal feature representations; cascading the enhanced visual, tactile, and text modal feature representations and inputting them into a self-attention module to output multimodal fusion features; and inputting the multimodal fusion features into a classification head network for recognition, outputting object category prediction results. This method can improve the accuracy and robustness of object recognition.

The application discloses a self-supervised monocular depth estimation method based on an enhanced multi-scale pose network, and relates to the field of computer vision.The method solves the problems of simple pose network structure, insufficient time sequence modeling capability and missing geometric constraints in existing self-supervised monocular depth estimation methods.The method comprises the following steps: constructing a self-supervised joint training framework composed of a depth estimation subnetwork and an enhanced pose estimation subnetwork; the pose estimation subnetwork extracts multi-scale spatial structure features through a layer-by-layer feature fusion encoder, and adopts a context fusion decoder based on time sequence attention to model the motion dependence relationship between continuous frames; meanwhile, a self-supervised pose consistency loss function is introduced, the geometric continuity of the camera trajectory is enhanced through positive and negative transformation consistency constraints and closed loop geometric constraints, and the collaborative optimization of depth prediction and pose estimation is realized.The application is also applicable to the fields of automatic driving, robot perception and augmented reality.

This invention discloses an online flow field estimation path planning method based on Bayesian inference, aiming to solve the problem of efficient detection and estimation of unknown flow fields by mobile robots. The method first discretizes the two-dimensional flow field region into a grid and constructs a probabilistic roadmap. In each planning round, a weighted maximum a posteriori (MAP) estimation problem is constructed based on the residuals of velocity measurements and fluid equations to solve for the flow field state and calculate the posterior accuracy matrix. Candidate paths are generated through bundle search and then prospectively expanded using subsequent information gain. After mapping path nodes to a set of proposed measurements, a joint prediction covariance matrix is ​​calculated. Based on a task weight matrix dynamically weighted according to velocity magnitude, a task-weighted posterior uncertainty reduction score is calculated for the candidate paths. The path with the highest score is selected for execution, and new measurements are collected. This process is repeated until a preset termination condition is met. This method integrates physical constraints and prospective search, guiding priority exploration of high-velocity regions and significantly improving the efficiency of online robot perception.

This invention provides a method and system for perception and scene understanding of a flying robot in a microgravity environment, belonging to the field of robot intelligent perception and scene understanding. It addresses the problems of crowded facility layouts, complex spatial lighting, and floating object obstruction in microgravity indoor work scenarios, which lead to difficulties in image feature extraction and obstructed sensor lines of sight, affecting the accuracy of 3D environment modeling and target recognition. A lightweight convolutional neural network structure is adopted to reduce computational burden; in terms of multimodal data fusion, data from RGB images and laser rangefinders are combined, and a multimodal information fusion module is added to target detection and pose estimation, suitable for the identification and localization of target objects required for high-precision operations; in terms of task-adaptive semantic segmentation, the network is trained on a specific dataset for specific task scenarios in a microgravity environment; and transfer learning is used to enhance the model's adaptability.

This invention relates to the field of robot perception technology, and provides a sensor and its manufacturing and perception methods. The sensor's perception method includes the following steps: S1: Pre-calibrating the parameter curve between the tentacle and the substrate to construct a calibration model; S2: A camera unit acquires images of the root of each tentacle on the substrate surface to form an image sequence; S3: A processing unit receives the image sequence and combines it with the calibration model and a built-in image processing algorithm to calculate and form multimodal perception information. This invention employs an optical imaging structure with the tentacle embedded in a silicone layer and a camera mounted on its back. It can calculate multimodal information such as three-dimensional contact force, fluid velocity and direction, and object surface texture / contour, achieving integrated detection of contact force perception and non-contact flow field perception, and has the advantages of being lightweight, compact, and highly robust.

This invention discloses a multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method and related devices, relating to the fields of robot perception and multi-agent collaboration. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method of this invention combines collaborative perception with path planning by maintaining obstacle lists and global abnormal path lists for the agents, enabling multi-agent collaborative perception and multi-agent collaborative planning and scheduling tasks. A replanning mechanism is introduced, allowing the agent system to make dynamic decisions based on perception information, improving its adaptability to dynamic and complex environments. The path following algorithm, which incorporates time-window path planning, can dynamically adjust its running speed according to time conditions, providing control support for the collaborative path planning algorithm. Furthermore, by adding an improved obstacle avoidance algorithm, when encountering an obstacle, it first determines whether the agent can autonomously avoid it; if it cannot, replanning is performed, reducing the frequency of replanning and improving the agent's operating efficiency.

The disclosure provides a multi-task cooperative calibration method and device applied to a robot and related equipment, and relates to the technical field of robots, in particular to the technical field of robot perception and control. The specific implementation scheme is: generating a control instruction of a mechanical arm of a robot based on motion constraint information; the motion constraint information is used to control the mechanical arm to cover multiple postures in the motion process; in the process that the robot controls the motion of the mechanical arm in response to the control instruction, collecting respective preset data of at least two calibration tasks; and performing corresponding calibration operations based on the preset data of each calibration task.

The application provides a biped robot foot bottom contact state sensing method and system, relates to the field of robot sensing technology, and comprises the following steps: acquiring multi-modal foot bottom sensing data and performing pretreatment; constructing a slip-sensitive tactile feature based on the pretreated multi-modal foot bottom sensing data; performing time sequence serialization processing on the slip-sensitive tactile feature based on a current time window, arranging feature vectors at continuous time points in time sequence, and forming a time sequence feature sequence; inputting the time sequence feature sequence into a pre-trained time sequence coding model to obtain high-level time sequence features for contact state recognition and uncertainty estimation; inputting the high-level time sequence features into a classification branch network to obtain probability distributions of various foot bottom contact state categories, and determining a predicted foot bottom contact state corresponding to the current time window based on the probability distributions of the various foot bottom contact states. The application can realize accurate judgment of the foot bottom contact state through the above method.

The application relates to the technical field of robot perception, and provides a visual-tactile sensor and a tactile reconstruction method.The visual-tactile sensor comprises a partitioned elastic contact layer, an illumination control module and an image acquisition module; the partitioned elastic contact layer comprises a high-sensitivity zone, a transition zone and a high-load-bearing zone; the elastic modulus of the high-sensitivity zone, the transition zone and the high-load-bearing zone increases in turn; the illumination control module is used for providing illumination light, and the image acquisition module is used for acquiring a deformation image of the partitioned elastic contact layer under the illumination light.The application divides the elastic contact layer into the high-sensitivity zone, the transition zone and the high-load-bearing zone, and adopts differentiated design of gradient elastic modulus for the zones, so that different zones can accurately perceive and reflect different degrees of contact force through deformation, thereby giving consideration to sensitivity and range, showing high sensitivity in different stress zones, significantly reducing the probability of deformation saturation, and ensuring the accuracy of the deformation image.

This invention relates to the field of robot perception technology, and particularly to a robot perception method and system based on human-machine collaboration. The method includes: scanning a cargo using a MEMS galvanometer-driven laser beam reflected by a ring-shaped curved mirror to acquire a three-dimensional point cloud and establish a standard outer edge contour; determining the corresponding illumination path based on the standard outer edge contour to form a cargo edge trajectory; generating a virtual protective line based on the standard outer edge contour, and determining the corresponding laser emission angle sequence and the curved mirror illumination path accordingly to form an obstacle protection trajectory; during AGV operation, alternately scanning the edge trajectory and the protection trajectory, and simultaneously performing data analysis to determine if any movement anomalies exist. This invention, through a cross-scanning mechanism, simultaneously monitors cargo displacement and external obstacles, solving the blind spot problem of traditional AGVs being unable to perceive the risks outside oversized cargo, and providing real-time warnings of cargo loosening or collision hazards, significantly improving transportation safety.

The invention relates to a multi-modal large language model-based three-dimensional affordability positioning method and system for a body robot, and belongs to the technical field of robot perception and body intelligence. According to the scheme, a multi-modal large model availability positioning framework is provided, the framework effectively combines semantic understanding, multi-modal fusion and 3D space positioning through an end-to-end learning mode, and the features of 3D scene data and natural language instructions are deeply mined and fully utilized. A novel scene-level affordability data representation is constructed, so that robot interaction tasks, such as door opening, drawer pulling and object grabbing, in various complex unstructured environments become more accurate and efficient; according to the method, adaptive sampling and a coarse-to-fine reasoning thought are fused, the calculation efficiency and robustness of understanding of a robot on a large-scale three-dimensional scene are improved by optimizing a data retention strategy and enhancing space consistency, and the limitation of a traditional method under the complex background and the ambiguous instruction in the real world is effectively solved.

This invention relates to the field of robot perception and control technology, specifically to a multimodal perception fusion system and method for a linkage-type dexterous hand. The multimodal perception fusion system of this linkage-type dexterous hand includes an array of tactile sensors distributed on the finger contact surface. In this invention, by setting up a spatially coordinated layout of multimodal sensors and a noise suppression preprocessing module, the accuracy of environmental perception is comprehensively improved. A dense array of tactile sensors is arranged on the finger contact surface to accurately capture the microscopic texture features of the object surface. High-sensitivity pressure sensors are embedded in the joint load-bearing nodes to provide real-time feedback on changes in grasping force. A proximity sensor is integrated into the fingertip to provide millimeter-level distance warning before contact with an object. A three-dimensional spatial positioning system is constructed in conjunction with a global vision sensor. To address sensor signal interference issues, a sliding window mean filtering technique is used to effectively suppress abnormal fluctuations caused by electromagnetic noise.

1. Name of the product in this design: Robot Perception Module. 2. Purpose of this design: For robots to perceive the external environment. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: a 3D model.

Disclosed is a robot perception based terrain recognition and gait control method, system, medium, and device. The method includes: acquiring pitch angle data and foot force data of a bipedal robot based on optimal foot forces; inputting the pitch angle data and the foot force data into a trained k-nearest neighbor (KNN) model to recognize a terrain on which the bipedal robot is currently walking, and obtaining a terrain recognition result; a training process of the KNN model including: driving the bipedal robot to pass through a rough terrain and a flat terrain with a fixed step frequency gait, inputting collected pitch angle sample data and foot force sample data from different terrains into the KNN model, and using the KNN model to perform terrain classification; receiving the terrain recognition result, and adjusting step frequency and gait based on the terrain recognition result according to a preset gait control strategy.

This invention discloses a remote pose estimation system and method for a neuromorphic sensor robot based on the Zinotopic set, belonging to the field of robot perception and state estimation technology. It includes a neuromorphic event sensor data acquisition module, a Zinotopic observability analysis module, a prior set construction module, a measurement information set construction module, an iterative pose estimation module, and a communication and transmission module. A delta triggering mechanism is used to filter asynchronous event data, perform Zinotopic observability verification, construct the prior set and measurement information set, and achieve asymptotically bounded estimation of the pose state through iterative fusion of the optimal intersection parameter matrix. The remote pose estimation system and method for a neuromorphic sensor robot based on the Zinotopic set provided by this invention are suitable for remote pose perception of wheeled robots in complex environments.

This invention belongs to the field of robot perception technology, specifically relating to a perception method for a magnetic quadruped robot based on binocular vision and elevation map fusion. The method includes the following steps: synchronously acquiring image data and angular velocity and linear acceleration data, and preprocessing them; extracting and tracking features from the images, and pre-integrating IMU data between consecutive keyframes to obtain relative motion increments; constructing a joint optimization objective function within a sliding window to solve for the robot's six-degree-of-freedom pose; using the pose to transform the current depth point cloud to the world coordinate system, and using probabilistic fusion to update the two-dimensional grid elevation map to obtain a local terrain map containing the elevation mean and uncertainty. This invention deeply integrates high-precision state estimation and terrain mapping, effectively solving the problems of positioning drift and inaccurate perception of small terrains in magnetic climbing robots on vertical curved surfaces and in environments with weak texture, significantly improving the robot's autonomous operation capability and safety.

The present application belongs to the field of computer vision and robot perception technology, and is especially a kind of image geometric estimation method based on Delaunay triangulation guidance.The present application includes the following steps: S1: Delaunay triangulation;S2: multi-dimensional loss function design;S3: joint weight optimization;S4: parameter fine-tuning and output.The present application initiatively encodes the neighborhood topological relationship of inliers by Delaunay triangulation, combines edge-level geometric consistency constraint and spatial distribution regularization, and fuses multi-dimensional loss to construct a weighted optimization model.The present application not only inherits the core advantages of traditional estimation algorithms, such as anti-out-point interference and basic geometric stability, but also effectively solves the problems of insufficient cooperation between geometry and space optimization, poor robustness in complex scenes and difficult guarantee of accuracy, realizes the current optimal inlier screening and model refinement, and completes the high-precision and high-robust image geometric parameter estimation in a sparse inlier and strong noise environment.

This invention discloses a transparent high-precision touch-sensitive skin for robots and its fabrication method, belonging to the field of robot perception and human-computer interaction technology. The skin is designed with two architectures: one is a single-functional layer structure integrating a transparent mesh touch layer to achieve high-precision touch positioning; the other is a dual-functional layer structure with a superimposed transparent pressure-sensing touch layer, simultaneously achieving touch positioning and precise pressure quantization. This invention uses a fine metal mesh with a line width of 2-20 μm and a spacing of 300-600 μm as the sensing unit, achieving a positioning accuracy of ≤0.1 mm while ensuring ≥84% light transmittance. Reliable interlayer bonding, pressure conduction, and electrical isolation are achieved through a transparent optical adhesive layer of a specific thickness. This skin combines high transparency, high sensing accuracy, excellent flexibility, and environmental adaptability, and can be widely used in service robots, medical robots, and industrial robots on complex curved surfaces requiring visual tactile interaction.

The application relates to the technical field of robot perception and autonomous navigation. The application discloses a target searching method and system based on robot central scene memory, a robot and a storage medium, which can improve the searching efficiency and positioning reliability of a target object and improve user experience. The method comprises the following steps: acquiring a robot central scene memory map and query features of a target object; the robot central scene memory map is updated in real time with the movement of the robot; similarity calculation and processing are performed on each filled grid unit in the robot central scene memory map and the query features, so that a first similarity score corresponding to each filled grid unit is obtained; all the first similarity scores are filtered according to a preset rule, a plurality of target scores are obtained, and each filled grid unit corresponding to each target score is used as a candidate unit; and the positioning information of the target object is searched based on all the candidate units.