145 results about "Robot environment" patented technology

The invention relates to the technical field of autonomous navigation and robot environment perception, and discloses a multi-source sensor fused adaptive navigation system, which comprises a multi-source sensor space-time synchronization module, a quantum particle filtering positioning estimation module, a space-time element learning controller module, a cross-modal quantum fusion module and an adaptive navigation control module. Multi-source data space-time alignment is realized through Lie group SE (3) calibration and dynamic time warping; the positioning robustness of particle filtering is improved based on quantum state coding and annealing optimization; dynamically distributing a fusion weight and injecting a physical constraint by utilizing a meta-learning network; feature level fusion of laser radar, vision and inertial data is realized by means of a quantum entanglement mechanism; and constructing closed-loop adaptive navigation by combining model predictive control and quantum purity trigger feedback. According to the method, the navigation reliability problem caused by misalignment of multi-modal sensor data fusion, divergence of state estimation and insufficient cross-modal relevance in a dynamic environment is solved.

The invention provides an industrial multi-robot intelligent collaborative planning method based on deep learning. The industrial multi-robot intelligent collaborative planning method comprises six parts including environment modeling, feature extraction, task allocation, trajectory planning, control instruction generation and rule distillation. The method comprises the following steps: acquiring multi-robot environment information by constructing a probability grid map and a topological structure, and extracting state and task features to form a comprehensive feature matrix; training an optimal task allocation strategy by adopting deep reinforcement learning, and combining CVAE and CEM joint modeling to optimize trajectory generation; an adaptive impedance controller based on MADDPG is further designed, and dynamic adjustment of interaction parameters is achieved; and finally, the control strategy is converted into a decision rule set through knowledge distillation, the control interpretability is improved, and the deployment complexity is reduced. According to the invention, the task cooperation efficiency and the control stability of the multi-robot system in a complex industrial environment can be effectively improved.

The invention relates to a robot environment perception method based on single-view three-dimensional scene generation, and the method comprises the steps: collecting a two-dimensional image containing target environment information through employing a monocular camera, generating multi-view information through combining depth estimation, normal prediction and a two-stage semantic guidance diffusion model, and constructing a high-quality three-dimensional scene. And reconstructing three-dimensional point cloud data through the neural radiation field, and performing texture rendering optimization on the point cloud data. A Point Net + + network is used for carrying out semantic analysis on point clouds, a multi-frame time sequence point cloud registration and Kalman filtering tracking method is introduced, modeling is carried out on a dynamic target, and a dynamic semantic map with a motion state is constructed. And finally, structured output environment information is used for robot navigation, path planning and task execution. The problems of high cost, high complexity and insufficient real-time performance and robustness of a three-dimensional scene generation technology in the field of robot environment perception are solved, and the method is high in structuring degree, standard and unified in output format and high in universality and engineering adaptation capacity.

The invention discloses an environment modeling method, system and equipment based on laser radar and vision fusion, and belongs to the technical field of mobile robot environment modeling. Through synchronous acquisition of three-dimensional point cloud data and image data, feature extraction and distortion correction are completed on the data acquired through the two different ways; accurate modeling features can be obtained, a geometric pose state quantity and a luminosity pose state quantity are calculated through an implicit moving least square iterative nearest point algorithm, the calculation precision of a curved surface in an environment is improved, and through point cloud splicing, a two-dimensional grid map and a three-dimensional point cloud map are generated. The two-dimensional grid map and the three-dimensional point cloud map are fused to complete environment modeling, the modeling speed is increased, the environment object information is aligned, information dislocation is avoided, and the environment modeling efficiency and accuracy are improved.

The invention provides a robot environment sensing method, device and system based on multi-sensor fusion. The method disclosed by the embodiment of the invention can comprise the following steps: receiving original observation data of each sensor configured on a robot body, and obtaining equivalent observation data of each sensor under system global time based on a time drift model of each sensor and the original observation data of each sensor, health degree indexes of all the sensors are calculated, time-varying credibility weights of all the sensors are generated according to the health degree indexes, activated sensor subsets are determined according to the health degree indexes of all the sensors, and state vectors of the robot system are determined according to the time-varying credibility weights of all the sensors in the activated sensor subsets and the equivalent observation data; and generating environment representation data based on the state vector of the robot system, the time-varying credibility weight of each sensor and the equivalent observation data. According to the invention, the influence caused by the time synchronization error and the space registration deviation between the sensors can be effectively avoided, and the sensing precision is improved.

The invention relates to the technical field of embankment inspection, and provides an embankment inspection robot environment detection method and system, which can determine a motion blur direction, extract linear features of an image and determine a linear feature direction by acquiring an embankment image and performing motion blur judgment when the image has motion blur. And on the basis, according to the geometrical relationship between the motion blur direction and the linear feature direction, whether the linear feature is the embankment crack or not is judged. By using the motion blur direction as a judgment basis, a pseudo crack and a real crack caused by motion blur can be effectively distinguished, and early-stage potential safety hazard detection omission caused by image quality reduction is avoided, so that the accuracy and the reliability of embankment crack detection are remarkably improved, and the continuity and the effectiveness of an embankment inspection task are guaranteed.

The invention discloses a search and rescue robot environment perception method based on a multi-task semantic recognition network, and relates to the field of search and rescue robots, and the method comprises the following steps: obtaining an RGB image and a depth image of a search and rescue environment, and constructing a multi-modal search and rescue data set; multi-task semantic recognition network training is carried out, and transfer learning and model lightweight strategy optimization are combined; the multi-task semantic recognition network comprises an encoder and a decoder; wherein the encoder comprises a double-flow parallel backbone network and a multi-stage feature fusion module, and the neck network decoder comprises a target detection branch, an obstacle semantic segmentation branch and a drivable channel segmentation branch; and deploying the optimized multi-task semantic recognition network to a search and rescue robot, and outputting an environment perception result in real time. According to the method, the precision of environment perception of the search and rescue robot is improved, the real-time performance and the computing resource requirement are effectively balanced, and the adaptability and robustness of the search and rescue robot in a complex and changeable post-disaster scene with scarce samples are enhanced.

The invention discloses a robot environment identification method based on multi-modal fusion, and the method comprises the following steps: collecting image data, point cloud data and acoustic data in a robot operation environment, and carrying out the preprocessing; inputting the structured multi-modal input sample into a modal perception type quantum encoder of a multi-modal quantum graph neural network; constructing a modal coupling quantum diagram based on the modal quantum state representation set; inputting the modal coupling quantum diagram into an entangled modal propagation unit, and performing cross-modal information propagation of nodes of the modal coupling quantum diagram through a parameterized quantum circuit; quantum measurement operation is executed on the fusion quantum graph embedded representation, and an environment recognition result is output; and inputting an environment identification result into a robot control module, and driving the robot to execute path adjustment, obstacle avoidance or action response. According to the method, the multi-mode quantum graph neural network is adopted, and autonomous recognition of the robot in a complex environment is achieved.

The invention provides a robot environment sensing method and device based on semantic completion, which are used for improving the accurate sensing ability of a robot with a body to obstacles in a complex environment. The method comprises the following steps: S1, preprocessing collected original three-dimensional environment data to obtain standardized input data; s2, inputting the standardized input data into a semantic completion model for reasoning, and generating a three-dimensional semantic voxel graph; s3, extracting a voxel region representing a barrier category according to the semantic tag, and fusing the voxel region with historical perception information to generate barrier region representation; and S4, mapping the obstacle region representation to a robot sensing structure, and constructing an occupation graph or an obstacle code for supporting path planning and obstacle avoidance control. According to the method, multi-scale sparse voxel representation and a semantic completion strategy are combined, the continuity, robustness and scene adaptability of obstacle recognition are remarkably improved, and the method is suitable for various application scenes such as navigation of a robot with a body, obstacle avoidance of a service robot and the like.

The invention discloses a robot environment sensing and positioning system based on an ROS. Effective data of multiple sensors can be efficiently processed, and the timestamps are unified for data fusion. According to the invention, an NUC11 industrial personal computer receives sensor data of an inertial sensor, a laser radar and an industrial camera and transmits the sensor data to a corresponding algorithm module for processing; a grid map is established after the surrounding environment is sensed; the speed and the direction are sent to the STM32 single-chip microcomputer through serial port communication, the STM32 single-chip microcomputer calculates the speed and the direction and then converts the speed and the direction into current values, current value data are sent to the motor through CAN communication, the Mecanum wheels are driven to move in all directions, and the navigation robot capable of achieving self-positioning and omni-directional movement is achieved. According to the omni-directional moving efficient wheeled robot, Mecanum wheels are innovatively combined to achieve omni-directional moving, a universal positioning and navigation robot chassis frame system is constructed, and the omni-directional moving efficient wheeled robot can become a universal solution for a cleaning robot, a meal delivery robot, an express delivery robot and the like in combination with different application scenes.

The invention provides a design method of a two-wheeled wheel-legged robot, which relates to the field of wheeled robots, and comprises the following steps: determining a machine body of the two-wheeled wheel-legged robot, adopting a leg four-bar mechanism as a leg configuration, and taking the leg four-bar mechanism and the machine body as a five-bar mechanism together, a universal leg rod length parameter calculation method is deduced based on a planar five-connecting-rod model, key geometric parameters are determined, and robot structure design is completed; establishing a wheel type motion dynamical model and a leg type jump phase motion dynamical model, deducing a universal motor model selection method, calculating driving wheel and joint motor parameters, and completing wheel train design and whole machine motion simulation; carrying out finite element analysis and lightweight design on the key parts; and finally, an LQR control strategy is adopted, and cooperative control verification is completed through analysis and simulation software. Wheeled efficient movement and leg type terrain adaptive capacity are fused, limitation of a single movement mode is broken through, and the environment adaptive boundary of the robot is expanded.

The invention discloses a robot variable stiffness vision-impedance control method and system based on optimization, and the method comprises the steps: building a coordinate transformation relation between a camera and a six-dimensional force sensor and a coordinate transformation relation between the camera and the tail end of a robot, calibrating the six-dimensional force sensor, and analyzing the interaction dynamics relation between the robot and the environment. A robot-environment interaction mathematical model is constructed, and a feature space human-robot-environment interaction dynamical model is established by solving a visual servo acceleration model and combining a dynamical equation of the robot in the assembly process; establishing a feature space human-guided dynamics model as a planner to obtain a reference feature trajectory implying human recognition contact dynamics; a characteristic space variable stiffness impedance controller based on QP online planning is designed, impedance parameters can be dynamically adjusted in the assembly process, interference force caused by environment uncertainty is compensated in real time, constant force is kept in a rich contact task, and it is ensured that the robot can stably complete the assembly task with high precision.

The invention relates to the field of mowers, and discloses a mowing robot full-coverage operation system and method based on multi-source information fusion, and the method comprises the steps: fusing the environment perception information of a mowing robot, multi-robot cooperation global sharing map information and historical operation data; through core steps of operation environment modeling, dynamic operation partitioning, multi-machine task allocation, collaborative path planning, operation execution monitoring, dynamic adjustment and the like, whole-process intelligent management and control of mowing operation is realized. Meanwhile, operation tasks are matched in combination with the operation state and capability parameters of the robot, partitions, tasks and paths are dynamically optimized according to the operation progress, the environment and the state change of the robot, and the matched system achieves automatic execution of all the processes through modular design. The adaptability and efficiency of multi-robot collaborative operation are improved, full coverage and low conflict of mowing operation are guaranteed, and the method is suitable for collaborative operation of mowing robots in various large-area and multi-obstacle scenes.

The invention provides a semantic scene reconstruction and interaction method and system based on a visual language model, and relates to the technical field of artificial intelligence and robots. The method comprises the following steps: acquiring image data shot by a camera; estimating a world pose of the camera; inputting the image data to a Visual Language Model (VLM) to generate a structured description containing an object semantic tag and quantitative spatial information thereof relative to the camera; calculating the world pose of the object according to the camera pose and the quantitative space information; a node representative of the object is created or updated in a hierarchical scene graph. The invention further provides a corresponding interaction system. According to the method, a rich and visual three-dimensional semantic map facing natural language interaction is constructed by deeply fusing the semantic understanding ability of the VLM and the geometric mapping ability of the SLAM, and the intelligent level of robot environment perception and man-machine interaction is greatly improved.

The invention relates to the field of robot environment perception, is used for solving the problem that an existing robot environment perception system lacks a unified and efficient hierarchical fusion architecture to systematically process multi-source heterogeneous data, and particularly relates to a robot real environment perception early warning system based on multi-sensor data fusion. Through cooperative work of the laser radar, the visual camera, the millimeter wave radar and the ultrasonic sensor, comprehensiveness and accuracy of environment perception are remarkably improved, confidence ranking and feature extraction are carried out on multi-source information through a progressive processing flow from space-time registration, cross verification and feature definition to deep fusion, and the accuracy of the environment perception is improved. And the heterogeneous feature fusion module generates an environment dynamic semantic map rich in texture, structure, motion and semantic information by means of a convolutional neural network, so that an accurate and comprehensive information basis is provided for early warning decision, and meanwhile, a dynamic hierarchical early warning mechanism provides reliable guarantee for safe and autonomous operation of the robot.

The invention belongs to the technical field of robot control, and aims to provide a robot environment sensing method and system, electronic equipment and a program product. The method comprises the following steps: acquiring initial environment detection data acquired by various sensors in the robot, and performing space-time alignment processing on the initial environment detection data to obtain aligned environment detection data of each sensor; environment grid occupancy probability estimation models are constructed for the multiple sensors respectively, the aligned environment detection data of the sensors are input into the corresponding environment grid occupancy probability estimation models respectively, initial grid occupancy probabilities of the sensors are obtained, fusion processing is carried out on the initial grid occupancy probabilities, and fused grid occupancy probabilities are obtained; constructing a dynamic occupation grid map of the robot based on the fusion grid occupation probability; and performing path planning processing on the robot based on the state prediction result and the dynamic occupation grid map. According to the method, the sensing precision and the path planning response capability of the robot in a complex dynamic environment can be effectively improved.

Provided is a robot, including: a chassis; a set of wheels coupled to the chassis; a plurality of sensors; a processor; and a non-transitory, machine-readable media storing instructions that when executed by the processor effectuates operations including: establishing, with the processor, a wireless connection with at least one of a computing device, a charging station, and a second robot; capturing, with at least one sensor, spatial data of an environment of the robot; generating or updating, with the processor, a map of the environment based on at least a portion of the spatial data; dividing, with the processor, the map into two or more rooms; storing, with the processor, the map in a memory accessible to the processor during a subsequent work session; and transmitting, with the processor, the map to an application of the computing device, wherein the application is configured to display the map.

The application provides a robot environment perception method based on multi-sensor fusion, comprising the following steps: performing space-time synchronization processing on original multi-sensor data to generate synchronized perception data; judging whether there is a semantic landmark matching the original or synchronized data in a topological map, wherein the matching condition is that the geometric primitive of the semantic landmark in the topological map and the same geometric primitive existing in the data are matched; when it is confirmed that there is a matching landmark, selecting the matching landmarks to form a set; based on the set, performing pose correction on the original multi-sensor data, fusing different pose constraints in the matching landmarks into the original data, and obtaining corrected data for robot path planning. The application can improve the consistency of multi-sensor data and the accuracy of robot environment perception, and provides reliable correction data for path planning.

The application relates to a STEAM teaching robot environment adaptation method based on deep reinforcement learning, which comprises the following steps: S1: collecting classroom multi-modal environment data and preprocessing, and obtaining unified multi-modal data after preprocessing; S2: constructing a classroom semantic map and an environment change factor; S3: multi-dimensionally modeling a student state, and obtaining a student state matrix; S4: constructing a unified state representation processable by a reinforcement learning model; S5: according to the unified state representation processable by the reinforcement learning model, generating a robot teaching adaptation strategy through a hierarchical deep reinforcement learning model, and obtaining a candidate action; S6: performing safety and teaching suitability checking on the robot action, obtaining a robot finally allowed action, and sending the robot finally allowed action to a robot motion control module, a voice interaction module and a teaching content display module. The application effectively improves the classroom perception ability and autonomous adaptation ability of the STEAM teaching robot.

The invention belongs to the field of multi-sensor fusion, and provides a lawn trimming robot environment sensing method and device based on vision and millimeter wave radar fusion. The method comprises the following steps: constructing a lawn data set, optimizing an activation function of a YOLOv11 instance segmentation model in a training stage, simultaneously adjusting and optimizing a model hyper-parameter and an optimizer to inhibit overfitting, and deploying the model to an AI chip end after conversion and quantification, so as to realize real-time visual reasoning of camera data; a DBSCAN algorithm is introduced at a millimeter wave radar end to cluster original point clouds so as to extract stable clustering center points, and the influence of stray points on the detection precision is significantly reduced; the system completes time synchronization of cross-sensor data by using a software timestamp in combination with linear interpolation, realizes spatial alignment through coordinate transformation, then projects a radar clustering center point to a visual detection plane, and performs position matching and decision-level information fusion through a geometrical relationship between the center point and a detection frame, so as to obtain a detection result. And the operation overhead is remarkably reduced while high-precision target positioning is ensured.

This invention discloses a robot environment perception method and system, belonging to the field of intelligent robot technology. The method and system collect environmental perception data through multiple perception links; establish a dynamic confidence model for each perception link and calculate the dynamic confidence level in real time; construct an environmental complexity assessment module to calculate the environmental complexity index in real time; and dynamically adjust the perception mode and fusion strategy within an adaptive fusion framework based on the dynamic confidence level and environmental complexity index. The adaptive fusion framework includes a predictive mode switching layer, a confidence-weighted fusion layer, and an adaptive computational resource scheduling layer. A navigation module receives the fused data and performs path planning and motion control. This achieves predictive switching of perception modes and confidence-weighted fusion, enhances system robustness, optimizes computational resource utilization, and improves the robot's autonomous navigation capability in complex dynamic environments.

The invention discloses an environment sensing method for a photovoltaic module paving robot. The environment sensing method comprises the step of realizing full-direction scanning sensing of a construction environment ground in a peripheral area of the middle lower part of the photovoltaic module paving robot. When the photovoltaic module paving robot walks on the ground of a construction environment site, scanning sensing is carried out through front and rear side sensing areas and corner sensing areas of the left and right four corners of the photovoltaic module paving robot, and all-direction scanning sensing of the peripheral area of the lower middle portion of the photovoltaic module paving robot is achieved. The problem that the photovoltaic module paving robot collides with the photovoltaic panel module in the process of walking and paving the photovoltaic panel is solved.

The invention relates to the technical field of moving track deduction, and provides a method comprising the following steps: step 1, a target robot generates a running route according to a three-dimensional topology network of a path in a park and a transportation task, and executes the running route; step 2, the target robot reads a current wireless position node, and generates a running route according to the wireless position node and the running route; for the high-risk robot, the target robot monitors the movement direction and the movement speed of the high-risk robot in real time, the movement habits of the high-risk robot are generated according to the movement direction and the movement speed of the high-risk robot, and the movement direction deduction of the high-risk robot is generated based on the movement habits of the high-risk robot and the movement track of the target robot. According to the method, through fusion of the motion model and the event-driven mechanism, the cooperative safety and the operation efficiency of the AGV in a complex multi-robot environment are remarkably improved. The problem of collision risk caused by information isolation between systems and sudden steering behaviors in the prior art is effectively solved.

The application discloses a robot environment perception method and system, a robot, a medium and a product. The method comprises the following steps: acquiring key frame pose data containing image data, scene structure data representing the association between an image frame and a target environment space, and semantic processing frames determined from the image frame; performing semantic perception processing on the semantic processing frames to obtain first semantic instance information, and / or matching and obtaining second semantic instance information from a semantic model; and performing fusion processing based on the obtained semantic instance information to generate or update semantic map data containing semantic road signs. The scheme decouples the geometric data acquisition and the semantic reasoning task in time sequence, so that the real-time positioning and tracking are not blocked by the semantic reasoning delay, the semantic reasoning can be triggered as needed, and the real-time positioning performance and the semantic map construction quality are considered under limited computing resources.

This invention provides a method and system for constructing a negative obstacle risk map, relating to the field of robot environmental perception technology. The method includes: acquiring multimodal sensing data of a legged robot in the current operating environment to obtain the types of negative obstacles, multiple candidate regions, and the corresponding confidence and uncertainty of each candidate region; using a risk diffusion algorithm, performing ink blurring processing based on the type, confidence, and uncertainty to generate a continuous risk field corresponding to each candidate region; dividing the continuous risk field to generate a continuous risk gradient distribution corresponding to each candidate region; performing terrain modeling based on laser point clouds, updating the continuous risk gradient distribution using Bayesian fusion to obtain a multi-level grid map, and then performing tactile closed-loop correction on the multi-level grid map to generate a risk cost map. This invention improves the accuracy of constructing a negative obstacle risk map.

The invention belongs to the technical field of intelligent cleaning robots, and relates to a fusion perception mapping method and device, a robot and a storage medium. Multi-modal fusion sensing data are obtained through the fusion sensing camera, information loss caused by shielding, illumination and material difference of a single sensor is made up, and comprehensive and stable sensing of the environment is achieved. A dynamic target is identified and processed through a preset auxiliary learning strategy, interference of the dynamic target on mapping is reduced, and map data stability and positioning precision are improved; an overlook semantic feature map is obtained based on an overlook semantic auxiliary mapping strategy, so that environment modeling has semantic understanding ability, and different regions or object categories can be distinguished; finally, the auxiliary mapping information and the overlook semantic feature map are fused to construct a high-precision overlook semantic map, and the robot environment understanding ability and task execution reliability are improved.

The application belongs to the technical field of agricultural robots, and discloses a path and height coordination planning system of a height-variable wheel-foot robot in weeding, which comprises a height-variable wheel-foot robot, an environment perception and weed identification module, a weed target tracking module, a navigation and path planning module, a height-path coordination planning module, an execution control module and a spraying execution module. On the basis of generating a basic navigation path, the height-path coordination planning module divides the navigation path into multiple sub-path segments by using weed spatial distribution, shielding indexes, spraying accessibility and stability constraints, and calculates a body running height for each sub-path segment, thereby forming a path-height coordination plan. In the execution stage, the height sequence is adaptively corrected according to online perception, and is coordinated with a low-level wheel-foot height control closed loop, so that the stability and accuracy of weeding operation in crop height variation and unstructured terrain are improved.

The invention provides a target detection method and device for a robot. The method comprises the following steps: acquiring environment point cloud data and scene description data acquired by robot environment acquisition equipment; performing data enhancement on the environment point cloud data and the scene description data to obtain enhanced data; performing feature extraction on the enhanced data to obtain feature data; performing alignment processing on the feature data to obtain aligned data; and performing target detection according to the alignment data to obtain a target detection result. Environment point cloud data and scene description data acquired by robot environment acquisition equipment are acquired, then data enhancement is performed on the environment point cloud data and the scene description data to obtain enhanced data, feature extraction is performed on the enhanced data to obtain feature data, and alignment processing is performed on the feature data to obtain alignment data. And finally, target detection is performed according to the alignment data to obtain a target detection result, so that the precision of the target detection result is improved.

The present application belongs to the technical field of cleaning robots, and particularly relates to a cleaning robot environment recognition method and system based on visual images. The method acquires an image of a recognition area and recognizes a plurality of target objects to extract parameter information, outputs obstacle attributes for the plurality of target objects, outputs a final evaluation line of sight between the image acquisition device for obstacle target objects, judges whether the image acquisition device is in a surrounding state based on whether there are two or more final evaluation lines of sight that cross each other, and finally determines the interaction correlation state of the cleaning robot according to the judgment result and outputs a recognition result. The present application can accurately recognize complex enclosed spaces, accurately distinguish obstacle target objects, and flexibly decide cleaning tasks, thereby improving the reliability of environment perception and the accuracy of the environment model, and further improving the autonomous operation efficiency of the cleaning robot.

The invention relates to the technical field of robot environment perception, in particular to a real-time ground segmentation method and system. The method comprises the following steps: receiving and preprocessing laser radar point cloud data, and dividing the data into a plurality of concentric annular areas; carrying out adaptive ground fitting on each area and processing the multilayer structure through vertical plane fitting to obtain an initial ground point set; correcting an under-segmentation region of the current frame by using a time sequence backspacing mechanism and updating parameters; constructing an elevation map based on the initial point set, and obtaining a fine ground point set through adaptive resolution CSF simulation; and carrying out confidence evaluation on the two types of point sets, and generating a final result through multi-criterion fusion. Accurate segmentation of the complex terrain is realized, and robustness and real-time performance of a dynamic environment are improved.