6960 results about "Robot control" patented technology

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving a user text sequence that characterizes a robotic task to be performed by the agent in the environment; converting the user text sequence into an input text sequence in a standardized format for a visual language understanding task; obtaining an observation image characterizing a state of the environment; processing (i) the input text sequence in the standardized format and (ii) the observation image using a policy neural network to generate a policy output that defines an action to be performed by the agent in response to the observation image; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

The invention relates to the technical field of robot control, in particular to a multi-degree-of-freedom mechanical arm obstacle avoidance control method based on machine vision, which comprises the following steps: acquiring 360-degree panoramic image data around a mechanical arm through a multi-camera system, and combining two-dimensional and three-dimensional information acquired by an RGB camera and a depth camera to construct an environment model. A pre-trained convolutional neural network model is used for identifying fixed and moving obstacles in an image, a Kalman filter is combined to predict the motion trail of the moving obstacle, an improved fast expansion random tree algorithm is used for path planning, it is ensured that the efficient and accurate obstacle avoidance capacity of the mechanical arm is kept in a dynamic environment, and the obstacle avoidance efficiency of the mechanical arm is improved. The motion state of the mechanical arm is controlled and monitored through real-time feedback, and deviation caused by inertia or errors is avoided. The dynamic adaptive capacity and the safe operation performance of the mechanical arm in the complex environment can be improved, and the mechanical arm is widely applied to robots in the fields of industry, medical treatment and the like.

The invention discloses a robot control method and device based on multi-modal data fusion, and the method comprises the steps: responding to a control instruction inputted by a user, receiving and preprocessing sensing information fed back by a plurality of types of sensors preset on a robot, and obtaining multi-modal data; dynamically adjusting the weight of each modal data, and extracting the feature vector of each modal data; according to the dynamically adjusted weight of each modal data, fusing the extracted feature vectors of each modal data to generate a comprehensive feature vector for representing the comprehensive state of the current environment; according to the comprehensive feature vector, the control instruction is analyzed through a preset large language model, and a final decision result of the robot is obtained; and according to the final decision result, calling and executing an instruction set required to be executed by the robot so as to drive the robot to complete related actions. Therefore, by adopting the embodiment of the invention, the robot can effectively process and utilize information from different modes to the maximum extent, and better decision and performance of the robot are realized.

The invention relates to an industrial robot cooperative control method, system and device and a storage medium. The method comprises the following steps: acquiring machine system information, working environment conditions, task demand contents and cooperative mode information of a plurality of industrial robots; performing spatial analysis on the working environment condition to obtain environment structure characteristics; performing task decomposition on the task demand content according to the cooperation mode information to obtain a sub-task list; mapping the machine system information into the environment structure features to obtain a corresponding mapping relationship, and performing association analysis on the mapping relationship to obtain a task allocation scheme; performing individual optimization on the comprehensive control parameters based on a task allocation scheme to obtain robot control parameters; and performing cooperative strategy analysis according to the sub-task list and the robot control parameters to obtain robot cooperative strategies, and performing comprehensive analysis on all the robot cooperative strategies to obtain an overall cooperative control strategy. According to the invention, the whole industrial robot system can be more suitable for complex and changeable actual production scenes.

A system may include a robotic control circuit configured to control one or more robotic functions of a robot. A system may include a plurality of sensors configured to collect data. A system may include a governance analysis circuit configured to analyze the data and select one or more governance frameworks based on the analyzed data. A system may include a governance model circuit configured to generate a model that applies the one or more governance frameworks to determine one or more governance actions, wherein the robotic control circuit is configured to control the one or more robotic functions in accordance with the one or more governance actions, wherein the robotic control circuit, the governance analysis circuit, and the governance model circuit are integrated on a single substrate.

The invention discloses an unhooking robot dynamic unhooking control method and system based on a feedback mechanism, and relates to the technical field of robot control, and the method comprises the steps: obtaining an unhooking robot, moving the unhooking robot to a preset unhooking position, and starting a visual sensor to collect unhooking image information and environment perception image information; edge detection and feature extraction are carried out on the unhooking image information, and hook space coordinate information is determined; performing feature abstraction on the environment perception image information to obtain a target unhooking path; and dynamic characteristic parameters and motion constraint information of the unhooking robot are obtained, unhooking analysis is carried out, unhooking control parameters are obtained, and dynamic unhooking control is carried out. The technical problems that in the prior art, an unhooking robot is low in unhooking efficiency and insufficient in control precision, and certain safety risks exist are solved, and the technical effects that the unhooking efficiency and the control precision of the unhooking robot are improved, and operation safety is guaranteed are achieved.

The invention discloses a mobile storage and charging robot remote scheduling and path planning system based on the Internet of Things, and relates to the technical field of robot control. Comprising a path dependency modeling module, a path conflict analysis module, a resource dependency graph construction module, a decoupling rearrangement scheduling module, a time sequence offset evaluation module and a path weight regulation and control module, and obtaining a path node sequence and an access time period of a current to-be-executed task of each mobile storage and charging robot, and generating a task path pre-occupation graph. By constructing the path dependence model, the conflict prediction mechanism and the task decoupling rearrangement strategy, accurate identification and effective intervention of path conflicts and resource deadlocks in the multi-robot scheduling process are realized, the stability of the scheduling system and the task execution continuity are improved, efficient completion of energy supply is ensured, and the scheduling efficiency is improved. And the operation efficiency and safety of the system are obviously optimized.

The embodiment of the invention provides a humanoid robot self-adaptive control method and system, electronic equipment and a storage medium, and belongs to the technical field of robot intelligent control. The method comprises the steps that electroencephalogram signals and electromyographic signals of a target object are collected; determining a first time-frequency domain feature of the electroencephalogram signal according to the electroencephalogram signal, and determining a second time-frequency domain feature of a muscle activation state according to the electromyographic signal; fusing the first time-frequency domain feature and the second time-frequency domain feature by adopting a dynamic weighting algorithm to obtain a multi-modal feature; inputting the multi-modal features into an action intention recognition model to obtain a predicted action intention; and determining a robot control instruction according to the predicted action intention and the current state of the robot so as to control the humanoid robot. The precision of self-adaptive control of the humanoid robot can be improved.

The invention discloses an unhooking robot control method and system based on depth visual recognition, and relates to the field of visual recognition, and the method comprises the steps: obtaining an original optical signal and ambient light intensity, carrying out the dynamic focusing of the original optical signal, generating a target pose sequence according to a frequency domain fusion algorithm and a gradient maximization positioning algorithm, and carrying out the dynamic focusing of the original optical signal; meanwhile, the environment light intensity is substituted into adjacent frame pose difference, and a real-time speed vector is obtained; performing modeling on the target pose sequence and the real-time velocity vector by adopting neuromorphic space-time motion to obtain a predicted trajectory of a dynamic coupling target body and a space-time constraint domain of an end effector of the robot; a bionics principle is adopted to simulate a biological motion coordination mechanism, a target motion track and mechanical arm dynamic constraint are jointly described, and the limitation that track prediction and mechanical arm motion ability are disjointed in traditional isolated modeling is broken through.

The present application relates to the field of robot control. Disclosed are a control method and apparatus for a legged robot, and a legged robot and a medium. The method comprises: acquiring body sensing information and external sensing information, wherein the body sensing information is used for representing the motion state of the body of a legged robot, and the external sensing information is used for representing environmental information around the legged robot; inputting the body sensing information and the external sensing information into a deep neural network, so as to obtain a first prediction residual that is output by the deep neural network; adjusting a trajectory generation parameter of a foot trajectory generator on the basis of the first prediction residual; and on the basis of a first joint motion parameter that is output by the parameter-adjusted foot trajectory generator, controlling the motion state of the legged robot. By using the solution provided in the embodiments of the present application, the flexibility and stability of a legged robot when moving in a complex environment are improved.

The invention belongs to the technical field of robot control, particularly provides a force feedback teleoperation control system and method for the tail end of a mechanical arm of a quadruped robot, and aims at the key challenges that control errors are caused by communication time delay and soft obstacles are difficult to recognize in a dynamic environment. Modeling and judgment are conducted on the contact state of the tail end of the mechanical arm in advance, and feedforward control and buffer adjustment oriented to communication time delay are achieved. And meanwhile, a dynamic semantic map is constructed in combination with multi-source sensing information, and soft obstacle reasoning and path optimization are performed by fusing a tail end force sense change trend, so that the recognition and avoidance capabilities of the system in a complex and invisible obstacle environment are remarkably improved. The system has good perspectiveness, self-adaptability and high redundancy safety characteristics, is suitable for multi-task inspection operation of industrial sites such as a thermal power plant, and is especially suitable for a remote man-machine cooperative operation scene in a narrow space.

The invention provides a multi-modal image analysis-based robot-assisted wound treatment method and system, and relates to the technical field of robots, and the method comprises the steps: collecting a multi-modal image of a to-be-treated wound, determining an image registration result, obtaining a wound region image through a multi-scale fusion algorithm and wound region segmentation, and carrying out the type classification of wound tissues. A wound tissue type classification result is obtained; performing surrounding shooting on a wound area to obtain a wound multi-view image, registering subsequent depth maps frame by frame by taking a first frame of depth map as a reference, accumulating point cloud data, determining a wound three-dimensional point cloud, generating a wound three-dimensional reconstruction result by combining a curved surface evolution algorithm, and determining a wound parameter measurement result; based on the wound tissue type classification result and the wound parameter measurement result, a robot path planning algorithm is adopted, the optimal movement track of the robot is generated, the movement parameters of the robot are adjusted in real time through a robot control algorithm of vision and force feedback, and wound adjuvant therapy is completed.

The invention relates to a multi-modal large model robot control method based on meta-learning fine tuning, and the method comprises the steps: carrying out the meta-learning pre-training of an OpenVLA model, and obtaining a pre-trained VLA model; performing task specific fine tuning to obtain a fine-tuned VLA model; and inputting the current visual data, the language cue word, the robot sensor data and the task target data graph into the fine-tuned VLA model, and outputting to obtain a decision control instruction for correspondingly controlling the working state of the robot. Compared with the prior art, the method has the advantages that meta-learning pre-training is carried out on diversified tasks, and universal features and strategies are extracted, so that the generalization ability of the model is improved; in addition, in a specific task, rapid fine tuning is performed through a small number of new samples, so that the model can adapt to new task requirements in a short time; and multi-modal information of vision, character cues and robot control is combined, so that the perception and decision-making capabilities of the model are improved, and the adaptability and operation efficiency of the robot in a complex environment are effectively improved.

The invention discloses a sparse reward environment-oriented collaborative knowledge fusion reinforcement learning method, and relates to the field of collaborative knowledge fusion reinforcement learning methods. By constructing a lightweight collaborative knowledge fusion model and a dynamic reward remodeling mechanism, the problems of low intelligent agent exploration efficiency and difficulty in strategy convergence in a sparse reward environment are solved. The method comprises the following steps: constructing a reinforcement learning framework comprising a policy network and a value network; designing an action space mutation supervision mechanism and a lightweight collaborative knowledge fusion model, and generating a smooth substitution action when a strategy is detected to be unstable; and a reward function is designed in combination with the task target and the dynamic constraint, and reward remodeling is realized by activating rewards through sub-target potential energy difference and knowledge fusion. According to the method, effective intermediate feedback can be provided for the agents in the sparse reward environment, the exploration efficiency is improved, the convergence time is shortened, the stability and cross-scene migration ability of the strategy are enhanced, and an effective solution is provided for sparse reward scenes such as robot control and multi-agent game.

The invention discloses a multi-modal collaborative decision-making method and device for a humanoid robot in an industrial scene, and relates to the technical field of robot control, and the method comprises the steps: obtaining a multi-frame monitoring image, force touch sensor data and equipment operation sensor data of a visual camera of the humanoid robot in the industrial scene; extracting feature vectors of the multi-frame monitoring image, the force touch sensor data and the equipment operation sensor data as multi-modal feature vectors; taking the multi-modal feature vector as a training sample to train a cross-modal model, so that the cross-modal model learns an association relationship among vision, haptic and equipment data; obtaining dynamic industrial scene data, extracting feature vectors, and inputting the feature vectors into the trained cross-modal model to generate scene description information; and outputting an operation decision instruction according to the scene description information so as to control the humanoid robot. Through cross-modal feature fusion and dynamic knowledge reasoning, the task execution precision and environmental adaptability of the humanoid robot in a complex industrial environment can be improved.

The invention relates to the technical field of robot control, and discloses a robot control method, system, equipment and medium based on a multi-modal large model, and the method comprises the steps: collecting the multi-source modal data of a scene where an operation task is located, and carrying out the processing through a machine learning model, obtaining a multi-modal feature, and carrying out the position coding and Transform fusion processing, multi-modal fusion features are obtained, the multi-modal fusion features and the constructed job task knowledge base are input into a large language model to decompose a target job task, a human-in-the-loop mechanism is introduced to optimize a decomposition result, and a sub-task sequence is obtained; according to a subtask type in the subtask sequence, processing the subtask sequence through a visual language action model or a reinforcement learning model, and generating a motion instruction to enable the robot to start an execution process of the target operation task; live-line work tasks are processed through the multi-modal large models LLM, VLA and the like, and the work efficiency of the autonomous distribution network live-line work robot is improved.

The invention discloses a multi-welding-robot collaborative operation system based on digital twinning, and belongs to the technical field of intelligent manufacturing and robot control, the system comprises a physical space module, a virtual space module, a twinning data layer and a control module, and communication connection is established among the physical space module, the virtual space module, the twinning data layer and the control module; the physical space module comprises a plurality of welding robots, a sensor group and a data transmission network; the virtual space module comprises a multi-welding robot twinning body and a workpiece twinning model; the twin data layer is used for connecting the physical space module and the virtual space module; and the control module generates a welding path optimization scheme, a multi-welding-robot collaborative collision avoidance strategy and a welding quality feedback control instruction based on the twin data layer. According to the method, a closed-loop control framework of'physical-virtual 'bidirectional mapping is constructed, so that dynamic collaborative optimization of multi-robot motion trails, welding seam forming and thermal deformation is realized.

The invention belongs to the technical field of robot control, and discloses a high-gear-ratio humanoid robot robust walking control method and system based on potential dynamics self-adaption, and the method comprises the steps: designing a control strategy training frame based on deep reinforcement learning, and carrying out the optimization through an Actor-Critic structure and a PPO algorithm; constructing a potential dynamic adaptive network LDAN, and extracting environment and ontology dynamic parameters through a variational auto-encoder; designing a multi-dimensional reward function; constructing a periodic gait library by using von Mises distribution and motion capture data; gradually introducing terrain disturbance and dynamic change through curriculum type simulation training; the trained strategy and the LDAN module are deployed on the high-gear-ratio driven humanoid robot, the problems that the high-gear-ratio humanoid robot is poor in motion stability, poor in adaptability and insufficient in action expression in a variable environment are solved, and the high-gear-ratio driven humanoid robot has the advantages of being high in robustness, high in natural expressivity and high in migration ability.

The invention relates to the technical field of robot control, and discloses a heavy-load robot motion trail method and system based on machine learning. The method comprises the steps that historical movement track data of the heavy-load robot in a working scene are collected, and the data comprise a joint position sequence, an end effector pose sequence and environment obstacle distribution information; the data is preprocessed, track features are extracted, a space-time correlation matrix is constructed, and the matrix is used for representing the dynamic coupling relation between joint movement and the tail end pose; training a trajectory prediction model containing a long and short-term memory network and an attention mechanism based on the matrix, and generating a collaborative mapping relation between a joint position and a tail end pose; obtaining a current task target pose sequence and an environment constraint condition in real time, and outputting a candidate track set meeting dynamic constraint through a model; and adopting a multi-objective optimization algorithm to screen candidate tracks, generating an optimal track instruction and issuing the optimal track instruction to an execution mechanism. The method adapts to the complex characteristics and variable working conditions of the heavy-load robot, and the track adaptability is improved.

Disclosed is technology for controlling deep reinforcement learning-based legged robot locomotion by inferring implicit terrain information. A legged robot control method may include inferring an action of a quadrupedal robot from proprioception through a deep reinforcement learning-legged robot model, and a locomotion policy that implicitly infers properties of terrains through which the quadrupedal robot moves may be learned in the legged robot model.

The invention provides a robot control method, device and equipment, a robot, a medium and a product, and belongs to the technical field of robots, and the method comprises the steps: generating a behavior tree based on task intention information corresponding to a user instruction; the task intention information represents the intention of the user for indicating the robot to execute the operation task, tree nodes of the behavior tree represent subtasks of the operation task executed by the robot, and node information of the tree nodes represents execution conditions of the subtasks corresponding to the tree nodes; controlling the robot to execute the operation task based on the behavior tree; task execution state information fed back by the robot in the operation task execution process is received; and when the task execution state information represents that the execution of the operation task is abnormal, dynamically adjusting the behavior tree to obtain an updated behavior tree, and controlling the robot to continuously execute the operation task based on the updated behavior tree. According to the method, the robot can efficiently work in a dynamically changing complex scene.

The invention provides a logistics robot control method and system based on artificial intelligence. After the surface material type, the three-dimensional geometric contour and the surface stress distribution data of a to-be-sorted package are obtained through the logistics robot, the hardness grade and the friction coefficient interval of the package are determined, the geometric model of the package is reconstructed, the minimum external cube size is calculated, and the fragile area is recognized. The adsorption or clamping mode and the unfolding angle are adjusted according to the information, the contact point and track of the grabbing action are optimized, and meanwhile a sorting strategy is generated by considering dynamic factors such as the flow speed of a sorting line. In the execution process, action parameters and decision weights are updated through an artificial intelligence algorithm according to the actual effect, and the sorting strategy is continuously optimized. According to the technical scheme provided by the invention, the control efficiency and precision of the logistics robot can be improved.

The invention discloses a robot control method and system for an industrial pipeline, and relates to the technical field of robot control. The method comprises the steps that a flange plate is installed on an end effector of the robot, a binocular vision camera is installed in the center of the front end of the flange plate, and four point laser displacement sensors are symmetrically arranged on the flange plate at different heights; the binocular vision camera collects a pipeline image; the upper computer determines an initial pose control result of the robot according to the pipeline image so as to control the robot to move preliminarily; the four point laser displacement sensors are used for collecting spatial displacement data of the pipeline at different height points after the robot moves preliminarily; and the upper computer determines a target pose control result of the end effector of the robot through the spatial displacement data so as to control the end effector of the robot to carry out pose adjustment. According to the method, accurate control over the axial parallelism of the robot end effector and the pipeline can be achieved, and therefore the pipeline assembling accuracy is improved.

The invention relates to a biped robot complex terrain self-adaptive gait planning method and a biped robot, and belongs to the technical field of robotics.The biped robot complex terrain self-adaptive gait planning method comprises the steps that a fractional order PID controller is constructed, and the fractional order PID controller comprises a multi-sensor network module, a self-adaptive module and a fractional order calculus control module; constructing a reinforcement learning model for biped robot gait planning, setting a state, an action, a decision and a reinforcement learning reward function of reinforcement learning, performing continuous iteration and optimization adjustment on the reinforcement learning model by using a strategy gradient method, updating strategy parameters, and outputting control parameters of the biped robot; and performing decision fusion on control parameters output by the fractional order PID controller and the reinforcement learning model through a comprehensive loss function to obtain globally optimal biped robot control parameters. According to the method, more accurate gait planning can be provided for the biped robot in complex terrains, and better walking flexibility and adaptability are achieved.

A robot comprises a memory, a processor, a body and a drive system which are coupled. The drive system comprises one of auger-based surface interface portions and continuous tread surface interface portions. The auger-based surface interface portions and the continuous tread surface interface portions are interchangeable to adapt the robot to one of different operating conditions and different uses. The processor is configured to: control movement of the robot, via the drive system, to traverse across a first surface, wherein the first surface comprises piled granular material, in response to the drive system being configured with the auger-based surface interface portions; and control movement of the robot via the drive system to traverse across a second surface, which is a solid or semi-solid surface other than the piled granular material, in response to the drive system being configured with the continuous tread surface interface portions.

The invention discloses a mechanical arm control method, device and equipment and a storage medium, and relates to the technical field of intelligent control, and the mechanical arm control method comprises the steps that voice information and image information are obtained; the voice information and the image information are analyzed based on a large language model, and grabbing task target information of the mechanical arm is obtained; the grabbing pose of the mechanical arm is obtained according to the grabbing task target information; a mechanical arm is controlled to complete a grabbing task based on a mechanical arm grabbing pose controller, according to the method, external voice information and environment images are obtained through analysis of a large language model, matched targets in the voice information and the environment images are understood, and the grabbing task is automatically generated according to the positions of the matched targets in the images; the large language model is fully utilized to process the task input information, the task target is identified to generate the capture task information, the success rate of the capture task is improved, and the task scene is widened.

Embodiments of the disclosure provide a solution for action planning. A method includes: generating a sequence of images for an action execution plan based on description information and a reference image related to an environment with an action executor located, the description information describing the action execution plan to be executed by the action executor; extracting a sequence of visual feature representations from the sequence of images, respectively; and for a respective visual feature representation of the sequence of visual feature representations, determining control information for controlling an action to be executed by the action executor in the environment to complete the action execution plan at least based on the respective visual feature representation, a reference visual feature representation prior to the respective visual feature representation in the sequence and observed information of the action executor in the environment during execution of a reference action prior to the action.

The invention provides a double-arm picking robot system and a control method, a robot body is mounted on a moving module through a rotary lifting waist and hip joint, and a storage module is arranged on the moving module; the moving module can drive the robot body and the storage module to move; a right multi-degree-of-freedom mechanical arm and a left multi-degree-of-freedom mechanical arm are arranged on the two sides of the robot body, a right arm tail end picking gripper is arranged at the right end of the right multi-degree-of-freedom mechanical arm, and a left arm tail end picking gripper is arranged at the tail end of the left multi-degree-of-freedom mechanical arm; an image acquisition module is arranged at the top of the robot body, a laser radar module is arranged on the moving module, and the image acquisition module and the laser radar module perform data transmission with the robot control module. Two arms are adopted for cooperative work, the structure of the humanoid mechanical arm is more suitable for the picking scene, and cooperation can be more effectively carried out.

The invention discloses a control method and device for a mounting and dismounting integrated robot of a sound barrier, and relates to the field of robot control. According to the method, a digital twinborn model and a dynamic environment model are constructed according to original sensor data, and a current environment risk value is determined; a mounting or dismounting sequence is generated based on a sound barrier topology rule, and a motion path and a mechanical arm motion track are generated according to the digital twin model, the dynamic environment model and the mounting or dismounting sequence; rehearsing the motion path and the motion track of the mechanical arm in the digital twin model, and when a rehearsing result meets a preset requirement, generating an initial instruction based on the motion path and the motion track of the mechanical arm; the current clamping force is determined through an impedance control algorithm, the current hoisting parameter is determined according to the load weight, the initial instruction is adjusted according to the current clamping force, the current hoisting parameter and the current environment risk value to obtain a control instruction, and the control instruction is executed. By implementing the technical scheme provided by the invention, the mounting and dismounting efficiency is improved, and the operation safety is enhanced.

The invention relates to the technical field of artificial intelligence welding, and discloses an automatic welding robot based on artificial intelligence and a welding system thereof. The system comprises a welding feature acquisition module, a process parameter generation module, a real-time regulation and control module and a motion planning module. The welding characteristic acquisition module is used for acquiring three-dimensional contour data, material component information and welding seam geometric parameters of a workpiece to be welded, and generating a welding characteristic spectrum through characteristic fusion; a process parameter generation module retrieves a matching template from a welding knowledge base according to the parameters, and outputs a reference welding process parameter combination in combination with an environment temperature and humidity compensation coefficient; the real-time regulation and control module dynamically corrects the reference parameters to generate an optimization instruction according to the molten pool form, thermal radiation distribution and electric arc voiceprint characteristics in the welding process; and the motion planning module calculates a motion track, attitude parameters and a speed curve of the welding execution mechanism according to the optimization instruction to form a robot control instruction set. The system can improve the intelligent level of welding, guarantees stable welding quality, and is suitable for welding scenes of multiple manufacturing industries.