183 results about "Robot environment" patented technology

The invention discloses a pig breeding intelligent management and control system based on multi-modal data monitoring, and belongs to the technical field of pig breeding management, and the system comprises a data collection and individual recognition module which comprises an RFID ear tag, an inspection robot, an environment sensor and an intelligent feeder, and is used for determining the identity information of a pig individual based on the data collected by the RFID ear tag and the inspection robot, associating the collected multi-modal data of the live pigs in the pig house, and uploading the multi-modal data to edge computing equipment through an MQTT protocol for preprocessing; the feature extraction module extracts physiological and behavior features of live pigs by using a multi-modal deep learning fusion model; the key breeding index analysis module constructs a live pig health index and an environment quality index; the abnormity early warning module adopts a CUSUM control chart to accumulate index deviation and respond to abnormity; the management regulation and control model constructs a Markov decision process by minimizing energy consumption and maximizing health indexes, intelligently adjusts the rotating speed of a fan and a feeding strategy, and realizes precise environmental control and feeding optimization.

The invention relates to the technical field of autonomous navigation and robot environment perception, and discloses a multi-source sensor fused adaptive navigation system, which comprises a multi-source sensor space-time synchronization module, a quantum particle filtering positioning estimation module, a space-time element learning controller module, a cross-modal quantum fusion module and an adaptive navigation control module. Multi-source data space-time alignment is realized through Lie group SE (3) calibration and dynamic time warping; the positioning robustness of particle filtering is improved based on quantum state coding and annealing optimization; dynamically distributing a fusion weight and injecting a physical constraint by utilizing a meta-learning network; feature level fusion of laser radar, vision and inertial data is realized by means of a quantum entanglement mechanism; and constructing closed-loop adaptive navigation by combining model predictive control and quantum purity trigger feedback. According to the method, the navigation reliability problem caused by misalignment of multi-modal sensor data fusion, divergence of state estimation and insufficient cross-modal relevance in a dynamic environment is solved.

Some aspects include a method for operating an autonomous robot, including: capturing, with a first sensor disposed on the robot, data of an environment of the robot; generating, with the processor, a map of the environment based on at least the data of the environment; localizing, with the processor, the robot within the environment; capturing, with a second sensor disposed on the robot, data of a floor surface; determining, with the processor, a floor type of areas of the environment based on the data of the floor surface; and determining, with the processor, settings of the robot based on at least the floor type of the floor surface, wherein the settings comprise at least an elevation of each of at least one component of the robot from the floor surface.

The invention provides an industrial multi-robot intelligent collaborative planning method based on deep learning. The industrial multi-robot intelligent collaborative planning method comprises six parts including environment modeling, feature extraction, task allocation, trajectory planning, control instruction generation and rule distillation. The method comprises the following steps: acquiring multi-robot environment information by constructing a probability grid map and a topological structure, and extracting state and task features to form a comprehensive feature matrix; training an optimal task allocation strategy by adopting deep reinforcement learning, and combining CVAE and CEM joint modeling to optimize trajectory generation; an adaptive impedance controller based on MADDPG is further designed, and dynamic adjustment of interaction parameters is achieved; and finally, the control strategy is converted into a decision rule set through knowledge distillation, the control interpretability is improved, and the deployment complexity is reduced. According to the invention, the task cooperation efficiency and the control stability of the multi-robot system in a complex industrial environment can be effectively improved.

The invention discloses a collision identification method and system for a window cleaning robot, and relates to the technical field of robot environment perception, and the method comprises the steps: carrying out the time difference compensation and space synchronization based on a glass material type and collision detection parameters through a distributed perception collaborative analysis method, and generating a three-dimensional space vibration information matrix; enhancing the three-dimensional space vibration information matrix through a dynamic wavelet packet decomposition algorithm and a sliding window analysis method according to the type of a glass material; according to the enhanced three-dimensional space vibration information matrix, collision coordinates are calculated through a time difference positioning algorithm, the characteristic frequency of the vibration waveform is analyzed, and collision characteristic information is obtained; based on the collision feature information, combining safety parameters of glass material types, adopting a hybrid decision algorithm to analyze the collision feature information, and generating and executing a robot obstacle avoidance instruction; according to the method, the high-resolution three-dimensional vibration field is constructed through space-time collaborative modeling, and the collision coordinate calculation precision and the anti-interference capability are improved.

The invention relates to a robot environment perception method based on single-view three-dimensional scene generation, and the method comprises the steps: collecting a two-dimensional image containing target environment information through employing a monocular camera, generating multi-view information through combining depth estimation, normal prediction and a two-stage semantic guidance diffusion model, and constructing a high-quality three-dimensional scene. And reconstructing three-dimensional point cloud data through the neural radiation field, and performing texture rendering optimization on the point cloud data. A Point Net + + network is used for carrying out semantic analysis on point clouds, a multi-frame time sequence point cloud registration and Kalman filtering tracking method is introduced, modeling is carried out on a dynamic target, and a dynamic semantic map with a motion state is constructed. And finally, structured output environment information is used for robot navigation, path planning and task execution. The problems of high cost, high complexity and insufficient real-time performance and robustness of a three-dimensional scene generation technology in the field of robot environment perception are solved, and the method is high in structuring degree, standard and unified in output format and high in universality and engineering adaptation capacity.

A computer-implemented method for controlling a robot collaborating with a human in an environment of the robot comprises: obtaining, by at least one sensor, multimodal information on the environment of the robot including information on a human acting in the environment; converting, by a first converter, the obtained multimodal information into text information; estimating, by an intent estimator, an intent of the human based on the text information; determining, by a state estimator, a current state of the environment including the human based on the text information; planning, by a behavior planner, based on the current state of the environment and the estimated intent of the human, a behavior of the robot including at least one multimodal interaction output for execution by the robot, and generating control information including text information on the at least one multimodal interaction output; converting, by a second translator, the generated text information into multimodal actuator control information; and controlling at least one actuator of the robot based on the multimodal actuator control information.

The invention belongs to a robot control method, and provides a human-guided robot-environment interaction self-adaptive control method and a related device for solving the technical problems that an existing human-guided robot-environment interaction control method is insufficient in motion stability, low in human operator safety and low in control precision. And estimating an optimal impedance parameter through a state space model, and learning a human-guided reference trajectory through a neural network in combination with the optimal impedance parameter. Then, in combination with a robot Jacobian matrix and a human-guided reference trajectory, a reference joint speed and an estimated acceleration of a joint space are obtained through closed-loop inverse kinematics, then network parameters of a robot dynamic model are obtained through approximation of an uncertain dynamic model, and finally, the robot dynamic model is obtained through combination with the reference joint speed and the network parameters of the joint space. And the mechanical arm control torque is obtained through the offset width fuzzy neural network. Accurate and compliant control under uncertain motion and dynamics conditions is realized.

The invention discloses an environment modeling method, system and equipment based on laser radar and vision fusion, and belongs to the technical field of mobile robot environment modeling. Through synchronous acquisition of three-dimensional point cloud data and image data, feature extraction and distortion correction are completed on the data acquired through the two different ways; accurate modeling features can be obtained, a geometric pose state quantity and a luminosity pose state quantity are calculated through an implicit moving least square iterative nearest point algorithm, the calculation precision of a curved surface in an environment is improved, and through point cloud splicing, a two-dimensional grid map and a three-dimensional point cloud map are generated. The two-dimensional grid map and the three-dimensional point cloud map are fused to complete environment modeling, the modeling speed is increased, the environment object information is aligned, information dislocation is avoided, and the environment modeling efficiency and accuracy are improved.

The invention provides a robot environment sensing method, device and system based on multi-sensor fusion. The method disclosed by the embodiment of the invention can comprise the following steps: receiving original observation data of each sensor configured on a robot body, and obtaining equivalent observation data of each sensor under system global time based on a time drift model of each sensor and the original observation data of each sensor, health degree indexes of all the sensors are calculated, time-varying credibility weights of all the sensors are generated according to the health degree indexes, activated sensor subsets are determined according to the health degree indexes of all the sensors, and state vectors of the robot system are determined according to the time-varying credibility weights of all the sensors in the activated sensor subsets and the equivalent observation data; and generating environment representation data based on the state vector of the robot system, the time-varying credibility weight of each sensor and the equivalent observation data. According to the invention, the influence caused by the time synchronization error and the space registration deviation between the sensors can be effectively avoided, and the sensing precision is improved.

The invention relates to the technical field of embankment inspection, and provides an embankment inspection robot environment detection method and system, which can determine a motion blur direction, extract linear features of an image and determine a linear feature direction by acquiring an embankment image and performing motion blur judgment when the image has motion blur. And on the basis, according to the geometrical relationship between the motion blur direction and the linear feature direction, whether the linear feature is the embankment crack or not is judged. By using the motion blur direction as a judgment basis, a pseudo crack and a real crack caused by motion blur can be effectively distinguished, and early-stage potential safety hazard detection omission caused by image quality reduction is avoided, so that the accuracy and the reliability of embankment crack detection are remarkably improved, and the continuity and the effectiveness of an embankment inspection task are guaranteed.

The invention discloses a search and rescue robot environment perception method based on a multi-task semantic recognition network, and relates to the field of search and rescue robots, and the method comprises the following steps: obtaining an RGB image and a depth image of a search and rescue environment, and constructing a multi-modal search and rescue data set; multi-task semantic recognition network training is carried out, and transfer learning and model lightweight strategy optimization are combined; the multi-task semantic recognition network comprises an encoder and a decoder; wherein the encoder comprises a double-flow parallel backbone network and a multi-stage feature fusion module, and the neck network decoder comprises a target detection branch, an obstacle semantic segmentation branch and a drivable channel segmentation branch; and deploying the optimized multi-task semantic recognition network to a search and rescue robot, and outputting an environment perception result in real time. According to the method, the precision of environment perception of the search and rescue robot is improved, the real-time performance and the computing resource requirement are effectively balanced, and the adaptability and robustness of the search and rescue robot in a complex and changeable post-disaster scene with scarce samples are enhanced.

The invention discloses a forest fruit pose estimation method and system suitable for a depth information loss scene, and belongs to the field of picking robot environment awareness, and the method comprises the steps: carrying out the forest fruit pose estimation of a forest fruit image of an orchard through employing a trained model, and enabling the model to comprise a target detection network, a feature enhancement module and a pose prediction head, using the forest fruit image sample training model and the target detection network to extract a multi-scale feature map from the forest fruit image sample, and generating a target detection frame; a feature enhancement module extracts global semantic Token from the multi-scale feature map, constructs a feature sequence in combination with target features in a target detection frame, and extracts global features from the feature sequence; the pose prediction head predicts the target pose by using the global features, and trains the model to convergence to obtain a trained model. According to the method, the pose estimation precision is improved in the absence of depth information, and the generalization ability is high, so that the operation robustness and reliability of the picking robot in a complex outdoor orchard environment are enhanced.

The invention discloses a self-adaptive control method and system for an unmanned forcible entry robot based on environmental perception, and relates to the related field of robot control, and the method comprises the steps: dynamically obtaining the target information of a target region through a perception device group; performing clustering analysis on the target environment information by taking the target IMU information as a constraint to obtain a target clustering result, and extracting a first clustering cluster; introducing a step-by-step fusion mechanism to perform fusion analysis on the environment information in the first cluster to obtain first sensing information; based on a first mapping relation between the first IMU information and the first perception information, constructing a target environment visual graph of the target area; and performing adaptive control on the target unmanned forcible entry robot according to the target environment visual graph after the rear-end closed-loop optimization. The technical problem that due to the fact that an existing unmanned forcible entry robot is poor in environmental adaptability during control, the working capacity and precision in the complex environment are insufficient is solved, and the technical effects that the environmental adaptability of the robot is enhanced, and the working capacity and precision in the complex environment are improved are achieved.

The invention discloses a robot environment identification method based on multi-modal fusion, and the method comprises the following steps: collecting image data, point cloud data and acoustic data in a robot operation environment, and carrying out the preprocessing; inputting the structured multi-modal input sample into a modal perception type quantum encoder of a multi-modal quantum graph neural network; constructing a modal coupling quantum diagram based on the modal quantum state representation set; inputting the modal coupling quantum diagram into an entangled modal propagation unit, and performing cross-modal information propagation of nodes of the modal coupling quantum diagram through a parameterized quantum circuit; quantum measurement operation is executed on the fusion quantum graph embedded representation, and an environment recognition result is output; and inputting an environment identification result into a robot control module, and driving the robot to execute path adjustment, obstacle avoidance or action response. According to the method, the multi-mode quantum graph neural network is adopted, and autonomous recognition of the robot in a complex environment is achieved.

The invention provides a robot environment sensing method and device based on semantic completion, which are used for improving the accurate sensing ability of a robot with a body to obstacles in a complex environment. The method comprises the following steps: S1, preprocessing collected original three-dimensional environment data to obtain standardized input data; s2, inputting the standardized input data into a semantic completion model for reasoning, and generating a three-dimensional semantic voxel graph; s3, extracting a voxel region representing a barrier category according to the semantic tag, and fusing the voxel region with historical perception information to generate barrier region representation; and S4, mapping the obstacle region representation to a robot sensing structure, and constructing an occupation graph or an obstacle code for supporting path planning and obstacle avoidance control. According to the method, multi-scale sparse voxel representation and a semantic completion strategy are combined, the continuity, robustness and scene adaptability of obstacle recognition are remarkably improved, and the method is suitable for various application scenes such as navigation of a robot with a body, obstacle avoidance of a service robot and the like.

A method and system for controlling an artwork-generating robot using a robot interface system. The method includes generating a digital prototype of an artwork using a rendering algorithm of the robot interface system; setting a digital canvas using the robot interface system; displaying the digital prototype of the artwork on a display of the robot interface system; calibrating the robot and a robotic environment to enable the robot to generate a robot-generated artwork corresponding to the digital prototype; converting the digital prototype into a physical robot-generated artwork using the artwork-generating robot; and using the robot interface system to adjust settings and to interact with the robot while the digital prototype is being converted into the robot-generated artwork. The step of calibrating the robot is performed by using a calibrating tool of the robot interface system.

The invention discloses a robot environment sensing and positioning system based on an ROS. Effective data of multiple sensors can be efficiently processed, and the timestamps are unified for data fusion. According to the invention, an NUC11 industrial personal computer receives sensor data of an inertial sensor, a laser radar and an industrial camera and transmits the sensor data to a corresponding algorithm module for processing; a grid map is established after the surrounding environment is sensed; the speed and the direction are sent to the STM32 single-chip microcomputer through serial port communication, the STM32 single-chip microcomputer calculates the speed and the direction and then converts the speed and the direction into current values, current value data are sent to the motor through CAN communication, the Mecanum wheels are driven to move in all directions, and the navigation robot capable of achieving self-positioning and omni-directional movement is achieved. According to the omni-directional moving efficient wheeled robot, Mecanum wheels are innovatively combined to achieve omni-directional moving, a universal positioning and navigation robot chassis frame system is constructed, and the omni-directional moving efficient wheeled robot can become a universal solution for a cleaning robot, a meal delivery robot, an express delivery robot and the like in combination with different application scenes.

A computer-implemented method for controlling a robot collaborating with a human in an environment of the robot comprises: obtaining, by at least one sensor, multimodal information on the environment of the robot including information on a human acting in the environment; converting, by a first converter, the obtained multimodal information into text information; estimating, by an intent estimator, an intent of the human based on the text information; determining, by a state estimator, a current state of the environment including the human based on the text information; planning, by a behavior planner, based on the current state of the environment and the estimated intent of the human, a behavior of the robot including at least one multimodal interaction output for execution by the robot, and generating control information including text information on the at least one multimodal interaction output; converting, by a second translator, the generated text information into multimodal actuator control information; and controlling at least one actuator of the robot based on the multimodal actuator control information.

A method for training a data processing system for determining robot poses comprises: providing at least one piece of output training information comprising a training robot pose and training input data comprising image data of an output image of a robot environment associated with this training robot pose, wherein a locked or unlocked image sub-region is identified in at least one output image;Generating at least one piece of additional training information for at least one piece of initial training information, which comprises the training robot pose of the initial training information and training input data, the image data of an additional image of a robot environment assigned to this training robot pose, wherein this additional image is generated by augmenting at least a part of the released or unlocked image sub-area in the initial image of the initial training information for which this additional training information is generated, or in the additional image of another piece of additional training information generated for this initial training information; Training a data processing system based at least partially on machine learning for determining robot poses on the basis of image data of a robot environment, wherein the data processing system is trained on the basis of one or more of the additional training information. The invention also relates to a method for operating a robot and a system or computer program (product).

The invention provides a design method of a two-wheeled wheel-legged robot, which relates to the field of wheeled robots, and comprises the following steps: determining a machine body of the two-wheeled wheel-legged robot, adopting a leg four-bar mechanism as a leg configuration, and taking the leg four-bar mechanism and the machine body as a five-bar mechanism together, a universal leg rod length parameter calculation method is deduced based on a planar five-connecting-rod model, key geometric parameters are determined, and robot structure design is completed; establishing a wheel type motion dynamical model and a leg type jump phase motion dynamical model, deducing a universal motor model selection method, calculating driving wheel and joint motor parameters, and completing wheel train design and whole machine motion simulation; carrying out finite element analysis and lightweight design on the key parts; and finally, an LQR control strategy is adopted, and cooperative control verification is completed through analysis and simulation software. Wheeled efficient movement and leg type terrain adaptive capacity are fused, limitation of a single movement mode is broken through, and the environment adaptive boundary of the robot is expanded.

The invention discloses a robot variable stiffness vision-impedance control method and system based on optimization, and the method comprises the steps: building a coordinate transformation relation between a camera and a six-dimensional force sensor and a coordinate transformation relation between the camera and the tail end of a robot, calibrating the six-dimensional force sensor, and analyzing the interaction dynamics relation between the robot and the environment. A robot-environment interaction mathematical model is constructed, and a feature space human-robot-environment interaction dynamical model is established by solving a visual servo acceleration model and combining a dynamical equation of the robot in the assembly process; establishing a feature space human-guided dynamics model as a planner to obtain a reference feature trajectory implying human recognition contact dynamics; a characteristic space variable stiffness impedance controller based on QP online planning is designed, impedance parameters can be dynamically adjusted in the assembly process, interference force caused by environment uncertainty is compensated in real time, constant force is kept in a rich contact task, and it is ensured that the robot can stably complete the assembly task with high precision.

This disclosure provides systems, methods, and apparatuses, including computer programs encoded on computer storage media, that provide for training, implementing, or updated machine learning logic, such as an artificial neural network, to model a manufacturing process performed in a manufacturing robot environment. For example, the machine learning logic may be trained and implemented to learn from or make adjustments based on one or more operational characteristics associated with the manufacturing robot environment. As another example, the machine learning logic, such as a trained neural network, may be implemented in a semi-autonomous or autonomous manufacturing robot environment to model a manufacturing process and to generate a manufacturing result. As another example, the machine learning logic, such as the trained neural network, may be updated based on data that is captured and associated with a manufacturing result. Other aspects and features are also claimed and described.

A method for the automatic production of a digital twin of at least one section of a building interior using at least one robot which is located in the interior of the building, includes the at least one robot having at least one sensor for the capture of a robot environment in the building interior. In the robot environment, there is at least one object that is classified in at least one object type from a number of predetermined object types. The method has the following features: capture of the robot environment by the at least one sensor of the robot to obtain captured environment data; and association of at least one object type with the object in the robot environment by means of a neural network on the basis of the environment data. The neural network is configured to recognize object types, and is executed by a processor.

In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.

The invention relates to the field of mowers, and discloses a mowing robot full-coverage operation system and method based on multi-source information fusion, and the method comprises the steps: fusing the environment perception information of a mowing robot, multi-robot cooperation global sharing map information and historical operation data; through core steps of operation environment modeling, dynamic operation partitioning, multi-machine task allocation, collaborative path planning, operation execution monitoring, dynamic adjustment and the like, whole-process intelligent management and control of mowing operation is realized. Meanwhile, operation tasks are matched in combination with the operation state and capability parameters of the robot, partitions, tasks and paths are dynamically optimized according to the operation progress, the environment and the state change of the robot, and the matched system achieves automatic execution of all the processes through modular design. The adaptability and efficiency of multi-robot collaborative operation are improved, full coverage and low conflict of mowing operation are guaranteed, and the method is suitable for collaborative operation of mowing robots in various large-area and multi-obstacle scenes.

The invention provides a semantic scene reconstruction and interaction method and system based on a visual language model, and relates to the technical field of artificial intelligence and robots. The method comprises the following steps: acquiring image data shot by a camera; estimating a world pose of the camera; inputting the image data to a Visual Language Model (VLM) to generate a structured description containing an object semantic tag and quantitative spatial information thereof relative to the camera; calculating the world pose of the object according to the camera pose and the quantitative space information; a node representative of the object is created or updated in a hierarchical scene graph. The invention further provides a corresponding interaction system. According to the method, a rich and visual three-dimensional semantic map facing natural language interaction is constructed by deeply fusing the semantic understanding ability of the VLM and the geometric mapping ability of the SLAM, and the intelligent level of robot environment perception and man-machine interaction is greatly improved.

The invention relates to the field of robot environment perception, is used for solving the problem that an existing robot environment perception system lacks a unified and efficient hierarchical fusion architecture to systematically process multi-source heterogeneous data, and particularly relates to a robot real environment perception early warning system based on multi-sensor data fusion. Through cooperative work of the laser radar, the visual camera, the millimeter wave radar and the ultrasonic sensor, comprehensiveness and accuracy of environment perception are remarkably improved, confidence ranking and feature extraction are carried out on multi-source information through a progressive processing flow from space-time registration, cross verification and feature definition to deep fusion, and the accuracy of the environment perception is improved. And the heterogeneous feature fusion module generates an environment dynamic semantic map rich in texture, structure, motion and semantic information by means of a convolutional neural network, so that an accurate and comprehensive information basis is provided for early warning decision, and meanwhile, a dynamic hierarchical early warning mechanism provides reliable guarantee for safe and autonomous operation of the robot.

The invention belongs to the technical field of robot control, and aims to provide a robot environment sensing method and system, electronic equipment and a program product. The method comprises the following steps: acquiring initial environment detection data acquired by various sensors in the robot, and performing space-time alignment processing on the initial environment detection data to obtain aligned environment detection data of each sensor; environment grid occupancy probability estimation models are constructed for the multiple sensors respectively, the aligned environment detection data of the sensors are input into the corresponding environment grid occupancy probability estimation models respectively, initial grid occupancy probabilities of the sensors are obtained, fusion processing is carried out on the initial grid occupancy probabilities, and fused grid occupancy probabilities are obtained; constructing a dynamic occupation grid map of the robot based on the fusion grid occupation probability; and performing path planning processing on the robot based on the state prediction result and the dynamic occupation grid map. According to the method, the sensing precision and the path planning response capability of the robot in a complex dynamic environment can be effectively improved.

The present invention discloses a robot environment detection method based on Bayesian kernel reasoning, comprising the following steps: initializing a set of historical waypoints of the robot, an information gain threshold, and inference model training parameters; randomly sampling a set of kinematically reachable poses within the robot's current perception range as candidate control actions; explicitly calculating the mutual information of each candidate action and establishing a training sample set; continuing to sample a certain number of candidate actions as query samples, and predicting the information gain and uncertainty of the query samples using a Bayesian kernel reasoning method; deciding the optimal candidate action based on the predicted results of the query samples, and finally executing the optimal candidate action and updating the environment map until the detection is completed. The Bayesian kernel reasoning detection method of the present invention can predict the information gain and uncertainty of candidate points in the environmental space, has low computational complexity, and is suitable for robots to achieve online and safe detection in large-scale, cluttered, unknown environments.