943 results about "Motion planning" patented technology

The invention discloses a manipulator grabbing planning system and method based on visual identification, and relates to the technical field of robots, the manipulator grabbing planning system comprises a visual perception module, a coordinate conversion module, a motion planning module, a control execution module, a tail end perception module and a system integration and communication module; the visual perception module realizes multi-view target detection, semantic segmentation and three-dimensional pose estimation through a binocular depth camera; the coordinate conversion module is used for converting a target pose under a camera coordinate system into a world coordinate under a mechanical arm base coordinate system; the motion planning module is responsible for generating a mechanical arm grabbing path, optimizing a strategy and supporting generalization migration of a multi-form mechanical arm; the control execution module drives the six-axis cooperative mechanical arm to complete the grabbing action, and vision-force mixed feedback closed-loop control is achieved. The tail end sensing module monitors the grabbing state in real time through a touch sensor and dynamically adjusts a grabbing strategy; and the system integration and communication module is used for realizing real-time data interaction and cooperative work among the modules of the system.

The invention provides a motion control method and system for an intelligent robot, and the method comprises the steps: collecting environment and state data through an intelligent sensor group of a humanoid robot, inputting the environment and state data into a pre-training first neural network model, and obtaining motion prediction data and an environment analysis result; constructing a motion planning model, and performing energy consumption-stability multi-objective optimization on the joint motion track by adopting a preset first algorithm; the central controller generates a joint position, speed and torque reference trajectory based on an optimization result; and the local second controller of each joint locally adjusts the reference trajectory within the prediction time domain according to the real-time feedback. According to the invention, multiple sensors are combined with the mixed attention neural network to realize environment and self state intelligent perception, and the problem of multi-sensor data fusion time sequence dependence is solved; through energy consumption-stability multi-objective optimization, the complex environment movement efficiency is remarkably improved; the central controller and the local controller work cooperatively, and in combination with an edge computing architecture, the communication delay is reduced, and the system response speed is improved.

The invention relates to the technical field of body-equipped intelligent robots, in particular to a multi-mode body-equipped intelligent robot control method and device, and the method comprises the steps: synchronously collecting visual, auditory, tactile, force sense and body perception information, and unifying the information to the same time-space reference through a cross-mode time-space stamp alignment mechanism; hierarchical feature extraction and fusion are carried out on the multi-modal information, and unified multi-modal scene state representation is generated; reasoning a decision based on the representation by using a body agent framework, and outputting a control instruction; motion planning and control, visual servo tracking in a non-contact stage and dynamic parameter correction in a contact stage are executed according to instructions; optimizing the multi-modal strategy network through an incremental strategy distillation mechanism based on the interactive data flow; the problem of space-time asynchronization of multi-modal sensing information is solved through a cross-modal space-time stamp alignment mechanism.

An introduced autonomous aerial vehicle can include multiple cameras for capturing images of a surrounding physical environment that are utilized for motion planning by an autonomous navigation system. In some embodiments, the cameras can be integrated into one or more rotor assemblies that house powered rotors to free up space within the body of the aerial vehicle. In an example embodiment, an aerial vehicle includes multiple upward-facing cameras and multiple downward-facing cameras with overlapping fields of view to enable stereoscopic computer vision in a plurality of directions around the aerial vehicle. Similar camera arrangements can also be implemented in fixed-wing aerial vehicles.

The invention provides a robot adaptive motion planning method and system based on compliance control, the method adopts admittance control as a compliance algorithm, the compliance is stronger compared with the traditional rigid control, and different compliance degrees can be selected according to the environment; combining inverse kinematics, trajectory interpolation and environmental parameters to construct an optimal reset trajectory under the constraint of a joint space safety region; by sensing external acting force and dynamically adjusting control parameters, the flexibility of the robot in a complex working environment is improved; a chassis-mechanical arm joint kinematics model is provided, and the problem that the working space of a traditional mechanical arm is limited is solved.

A system includes a camera, a robot, and a management system. The camera is configured to capture one or more images of a material handling environment. The material handling environment normally includes one or more items. The robot is at least in part configured to move along a travel path relative to the items, and the robot is configured to pick the items. The management system is configured to monitor the material handling environment through the images from the camera. The management system is configured to determine dynamically multiple picking solutions for the items in real-time while the robot moves along the travel path.

The invention discloses a mechanical arm natural language instruction control system and method based on a large language model, and belongs to the field of intelligent manufacturing. Aiming at the limitation that traditional mechanical arm control depends on pre-programming and a static rule library, a dynamic mapping mode from a natural language instruction to an atomic action sequence is designed, an atomic skill library including detection, grabbing, moving, placement and other operations is constructed, and semantic analysis and a multi-mode cooperation technology are combined, so that the atomic action sequence is obtained. And support is provided for man-machine cooperation of a flexible assembly task. The method specifically comprises the steps that a DeepSeek-Distil-Llam-8B large model and a LoRA fine tuning technology are adopted, and a natural language instruction is converted into an executable atomic action sequence; based on a transfer learning optimized YOLOv8 target detection technology and a binocular vision positioning technology, a sensing module adaptive to an assembly scene is constructed and is fused with a mechanical arm motion planning module, and positioning grabbing of parts and tools is achieved. And an interactive interface is built by combining a voice-to-text large model and a Gradio front-end framework, so that the convenience of man-machine interaction is improved. By optimizing large model reasoning and motion planning cooperation efficiency, response delay from instructions to execution is reduced, and an efficient and extensible solution is provided for man-machine cooperation in intelligent manufacturing.

An introduced autonomous aerial vehicle can include multiple cameras for capturing images of a surrounding physical environment that are utilized for motion planning by an autonomous navigation system. In some embodiments, the cameras can be integrated into one or more rotor assemblies that house powered rotors to free up space within the body of the aerial vehicle. In an example embodiment, an aerial vehicle includes multiple upward-facing cameras and multiple downward-facing cameras with overlapping fields of view to enable stereoscopic computer vision in a plurality of directions around the aerial vehicle. Similar camera arrangements can also be implemented in fixed-wing aerial vehicles.

The invention relates to the technical field of surgical robot motion control, and discloses a surgical robot motion planning method based on a diffusion model, and the method comprises the steps: obtaining obstacle point cloud data, an initial posture and a target posture in a surgical environment; encoding the obstacle point cloud data into a potential space, and combining the encoded information with the initial attitude and the target attitude into a condition code; a Transform structure of an encoder is adopted to replace a U-Net structure in a traditional diffusion model, and the diffusion model is trained through forward diffusion and reverse denoising processes; in training, using a comprehensive loss function to optimize model performance, combining configuration space loss, geometric task space loss and collision loss, and introducing physical constraints to ensure that the trajectory conforms to kinematic characteristics and avoid collision; and in the operation task, generating a motion track based on the trained diffusion model. According to the invention, the motion planning efficiency and safety of the surgical robot in a complex environment can be improved.

In various examples, a gradient-based motion planner evaluates a cost function corresponding to routes for a machine and an obstacle to jointly update the routes. The cost function may include terms to penalize deviation from an initial route predicted for the obstacle and acceleration or jerk for the obstacle. The routes for the machine and the obstacle that are updated may be selected using motion classes that characterize relative motion between a route for the machine and a route for the obstacle. A motion class may be based at least on an angular distance between the machine and the agent and free-end homotopy, where members of the class execute the same relative motion with respect to other agents while being continuously transformable to any other member of the class. The members of the class may have the same start point and different end points.

The invention relates to the technical field of artificial intelligence welding, and discloses an automatic welding robot based on artificial intelligence and a welding system thereof. The system comprises a welding feature acquisition module, a process parameter generation module, a real-time regulation and control module and a motion planning module. The welding characteristic acquisition module is used for acquiring three-dimensional contour data, material component information and welding seam geometric parameters of a workpiece to be welded, and generating a welding characteristic spectrum through characteristic fusion; a process parameter generation module retrieves a matching template from a welding knowledge base according to the parameters, and outputs a reference welding process parameter combination in combination with an environment temperature and humidity compensation coefficient; the real-time regulation and control module dynamically corrects the reference parameters to generate an optimization instruction according to the molten pool form, thermal radiation distribution and electric arc voiceprint characteristics in the welding process; and the motion planning module calculates a motion track, attitude parameters and a speed curve of the welding execution mechanism according to the optimization instruction to form a robot control instruction set. The system can improve the intelligent level of welding, guarantees stable welding quality, and is suitable for welding scenes of multiple manufacturing industries.

The invention discloses an operation and maintenance manipulator intelligent control method and system based on visual identification, and relates to the technical field of intelligent manipulator control, and the method comprises the steps: collecting RGB image data and depth image data of an operation and maintenance operation area, and obtaining a standardized image matrix and a mapping relation matrix; inputting the standardized image matrix into an improved ResNet residual network model, generating a comprehensive feature descriptor, and calculating a spatial position coordinate and an attitude angle of the target equipment based on the mapping relation matrix; based on the current joint angle state of the manipulator, an improved Jacobian matrix inverse kinematics algorithm is used for solving a target angle sequence of each joint, a preset operation mode library is matched based on the comprehensive feature descriptor, and a grabbing force parameter and a motion speed parameter are determined; and converting the target angle sequence into a control instruction, and sending the control instruction to each joint driver of the manipulator to drive the manipulator to complete action planning. According to the invention, full-process automation from environment perception to task execution is realized.

The invention provides an optimization method for whole-body motion planning of a mobile manipulator, and relates to the technical field of motion planning of the manipulator, and the method comprises the steps: determining environment features and a position gradient when a joint of the mobile manipulator changes based on a depth image shot by the mobile manipulator; performing strategy planning on the motion state and the motion action of the mobile mechanical arm through an actor-commentator network subjected to reinforcement learning training to obtain an expected speed of an end effector in the mobile mechanical arm; position constraint conditions used for obstacle avoidance of the movable mechanical arm are constructed based on the position gradient, and joint constraint parameters used for constraining the decision variables are constructed; and optimizing the decision variable based on a quadratic programming solver determined by the position constraint condition and the joint constraint parameter to obtain a planning result of the joint speed. According to the method and the device, the problem of limitation in the aspects of planning task completion, environment perception, constraint expression, obstacle avoidance and motion coordination during motion planning of the mobile mechanical arm is solved.

The present disclosure provides an example method that includes: (a) obtaining context data descriptive of an environment surrounding an autonomous vehicle, the context data based on map data and perception data; (b) generating, by a proposer and based on the context data: (i) a plurality of candidate trajectories, and (ii) a plurality of actor forecasts for a plurality of actors in the environment; (c) generating, by a ranker and based on the context data, the plurality of candidate trajectories, and the plurality of actor forecasts, a ranking of the plurality of candidate trajectories; and (d) controlling a motion of the autonomous vehicle based on a candidate trajectory selected based on the ranking of the plurality of candidate trajectories, wherein the proposer comprises a first machine-learned model and the ranker comprises a second machine-learned model, and wherein the first machine-learned model and the second machine-learned model use a common backbone architecture.

The present disclosure provides an example method for obtaining labeled trajectories. The example method can include obtaining log data describing a trajectory of a vehicle traveling through an environment. The example method can include determining a suboptimal condition associated with the trajectory. The example method can include generating label data that characterizes the suboptimal condition along one or more constraint dimensions of a motion planner of the autonomous vehicle control system. The example method can include generating a training example for training the one or more machine-learned models of the autonomous vehicle control system to decrease a probability of the autonomous vehicle control system inducing the suboptimal condition.

The invention discloses a quadruped robot autonomous decision control system based on multi-modal perception, and belongs to the technical field of robot motion, and the system specifically comprises the steps: collecting visual, force and sound information through multiple sensors of a camera, a force sensor and a microphone array, and carrying out the preprocessing; according to the processed information, a diagonal gait, leg track and motion path planning scheme is generated by adopting a motion planning strategy; generating a specific control instruction to drive the robot to move, and monitoring and adjusting in real time; after the task is completed, the motion performance of the robot is evaluated, and a motion planning and control method is optimized according to an evaluation result; the autonomous motion ability and adaptability of the quadruped robot in a complex environment are improved.

In various examples, systems and methods are disclosed relating to training machine learning models using human demonstration of segments of a task, where other segments of the task are performed by a planning method, such as a Task and Motion Planning (TAMP) system. A method may include segmenting a task to be performed by a robot into segments, determining a first set of instructions of a plurality of sets of instructions for operating the robot to perform a first objective of a first segment, determining that the plurality of sets of instructions is inadequate to perform a second objective of a second segment, receiving from a user device a second set of instructions for operating the robot for the second segment following an end of the first segment, and updating a machine learning model for controlling the robot using the second set of instructions for the second segment.

A method is for training a base model for object detection, trajectory prediction, and / or motion planning of a vehicle. The method includes providing a training data set of image data, with each piece of image data having information about at least one driving scene from a point of view of the vehicle, and providing a knowledge graph including domain-specific knowledge of the at least one driving scene. The method further includes optionally partitioning the image data into a plurality of image sections, and generating information matrices corresponding to the image sections by assigning domain-specific knowledge about the at least one driving scene extracted from the knowledge graph and / or directly from the image data to the plurality of image sections of the image data. The method also includes training the base model based on the information matrices.

The structures and algorithms described herein employ look ahead motion planning, in which motion planning for a least two goals is performed before a robot executes the resulting motion plans, and the ability to transition between the preceding one of the motion plans to a subsequent (e.g. following) one of the motions plans is assessed. Thus, the system can determine whether a robot will get trapped (e.g., blocked by another robot) at the end of a first motion plan, preventing, limiting or delaying execution of a second motion plan. Detection of such a condition can cause one or more remedial actions can be taken, for example generating a new, revised or replacement first motion plan. Other remedial action can moving another robot, performing motion planning for the other robot, to alleviate a blocking condition and / or determining a new order for the goals.

The invention belongs to the field of intelligent equipment, and relates to a mechanical arm control method and system based on image point cloud cross-modal fusion and action block Transformer, and the method comprises the steps: constructing a scene point cloud according to a multi-view RGB image and a corresponding depth map; a virtual view of multi-view fusion image-point cloud information is generated around the working space of the mechanical arm through the scene point cloud; inputting the virtual view of the multi-view fusion image-point cloud information into a feature extraction network to obtain a feature map; obtaining each joint state and a historical action sequence in the mechanical arm, and projecting the joint states; inputting the feature map, the projected joint state and the historical action sequence into an encoder to obtain encoding features; the coding features are input into a decoder, and a mechanical arm action sequence is predicted based on an action partitioning strategy; the mechanical arm is controlled according to the mechanical arm action sequence; according to the method, the multi-modal sensing data of the point cloud image are integrated into uniform feature representation, and intelligent action planning of the mechanical arm is realized in combination with the action block Transform, so that the global understanding and decision-making capability of the robot on complex tasks is improved.

The invention relates to the field of mechanical arm motion planning, relates to a multi-mechanical arm motion planning method based on an improved artificial potential field method and deep reinforcement learning, and aims to solve the problems that in a multi-mechanical arm system, early-stage rewards are sparse, the training speed is low, and effective strategies are difficult to learn in deep reinforcement learning. The invention provides an improved dynamic artificial potential field expert demonstration scheme, and in the training process, an artificial potential field reward function and a dynamic artificial potential field expert demonstration mechanism which dynamically change along with the training process are added, so that the mechanical arm obtains higher learning efficiency and more positive samples in the early-stage training process; in later training, the mechanical arm is assisted to reach a high-precision target, and then the learning speed of the mechanical arm is increased. The training process mainly comprises the following steps of constructing a simulation environment, describing and modeling a multi-mechanical-arm path planning task, generating a training sample, executing the training task and performing simulation, dynamically adjusting expert data according to a simulation result, collecting experience and updating a strategy. Compared with a traditional method, the method has the advantages that the convergence speed is higher, the training time is shorter, the precision is higher, and particularly, the early-stage convergence speed is higher, and the fitting precision is higher.

The invention belongs to the related technical field of mobile robots, and discloses a rolling-walking composite motion control method and device for a double-wheel-foot robot, and the method comprises the steps: (1) simplifying the rolling motion of wheels of the double-wheel-foot robot into the sliding motion of a sliding block; (2) constructing a system kinetic equation of the double-wheel-foot robot based on the simplified kinetic model; constructing a constraint equation based on the contact state of the two legs and the ground; discretizing the system kinetic equation, the cost function and the constraint equation, and converting model prediction control problems corresponding to the system kinetic equation, the cost function and the constraint equation into nonlinear problems for numerical solution so as to complete leg motion planning; (3) calculating the rotation speed of the wheel based on the leg motion planning result; and (4) based on the rotation speed of the wheels and the leg motion planning result, a whole-body control algorithm is adopted to calculate and obtain moment instructions of leg joints and the wheels. According to the invention, the state space dimension is reduced, and the wheel-ground contact constraint is simplified.

The invention relates to the technical field of numerical control machine tool precision detection and compensation, in particular to a coordinated motion planning control method and related products in the five-axis machining process, and the method comprises the steps: building a machine tool digital twin model, and combining constraint conditions; calculating a dynamic load rate and adjusting an error weight according to the load rate; an optimal cooperative motion instruction is obtained through multi-objective optimization; obtaining a feed-forward compensation amount and a feedback compensation amount; fusing the feedforward compensation amount and the feedback compensation amount to obtain a total compensation amount, and correcting the optimal cooperative motion instruction in real time to generate a final control signal; according to the method, dynamic constraints of different dimensions of a shaft are unified, the real-time dynamic load rate of the shaft is calculated, and weight coefficients of a translation shaft error and a rotation shaft error in an objective function are dynamically adjusted and optimized according to the difference; and constructing a multi-objective optimization function, solving an optimal cooperative motion instruction according to the dynamic weight, and integrating feed-forward compensation and feedback compensation to generate a final control signal so as to realize accurate control.

The invention discloses an IsaacSim-based humanoid robot simulation method for a steel inspection and test laboratory, and relates to the technical field of humanoid robot simulation methods, and the IsaacSim-based humanoid robot simulation method for the steel inspection and test laboratory comprises the following steps: step 1, carrying out refined modeling on environmental elements of the steel inspection and test laboratory; step 2, integrating a humanoid robot model in an IsaacSim simulation environment; 3, configuring workflows of different tasks of the robot in the simulation scene; 4, collecting and analyzing data when the robot executes the task in the virtual environment; and step 5, feeding back an adjustment algorithm or a control parameter based on a simulation result so as to improve the adaptability and the stability of the robot in a complex laboratory environment. Through the method, a user can perform comprehensive algorithm verification in a simulation environment, including a series of key algorithms such as a control algorithm, a visual identification algorithm and a motion planning algorithm of the humanoid robot, so that the stability and effectiveness of the humanoid robot in an actual environment are ensured.

The embodiment of the invention discloses a vector control method and device for a steering wheel chassis. The method at least comprises the steps that the target speed and the target angular speed of a steering wheel chassis at the next moment are obtained; the target speed comprises a target transverse speed and a target longitudinal speed; obtaining pre-stored size parameters of each steering wheel; the size parameters at least include the distance between the mass center of the steering wheel chassis and each steering wheel in the coordinate system of the steering wheel chassis in the coordinate axis direction, and the original point of the coordinate system of the steering wheel chassis is located at the mass center of the steering wheel chassis; and calculating a control angle of a steering motor and a control speed of a driving motor corresponding to each steering wheel at the next moment based on a pre-constructed motion planning control strategy according to the target speed, the target angular speed and the size parameters of each steering wheel. By applying the scheme provided by the embodiment of the invention, the steering wheel chassis can be flexibly controlled.

The present disclosure provides an example method that includes: (a) obtaining context data descriptive of an environment surrounding an autonomous vehicle, the context data based on map data and perception data; (b) generating, by a proposer and based on the context data: (i) a plurality of candidate trajectories, and (ii) a plurality of actor forecasts for a plurality of actors in the environment; (c) generating, by a ranker and based on the context data, the plurality of candidate trajectories, and the plurality of actor forecasts, a ranking of the plurality of candidate trajectories; and (d) controlling a motion of the autonomous vehicle based on a candidate trajectory selected based on the ranking of the plurality of candidate trajectories, wherein the proposer comprises a first machine-learned model and the ranker comprises a second machine-learned model, and wherein the first machine-learned model and the second machine-learned model use a common backbone architecture.

The invention discloses a space floating mechanical arm motion planning method and system based on zero reaction space and reinforcement learning, and relates to the field of mechanical arm motion planning. The invention aims to solve the problem of base attitude disturbance caused by movement of an existing space floating mechanical arm. According to the technical scheme, the motion space of the space mechanical arm is designed to be the zero-reaction space, it is ensured that the motion of the mechanical arm does not generate unexpected disturbance on the base, and therefore the stability of the system and the task execution precision are improved. In addition, a reinforcement learning algorithm is combined, and the motion space is set as a zero-reaction motion space. The efficient search and optimization capability of reinforcement learning enables the mechanical arm to find an optimal motion strategy in a complex task space, and meanwhile, disturbance to the base is effectively avoided. The strategy can be continuously adjusted through multiple rounds of training and reinforcement learning so as to adapt to different task requirements, and it is guaranteed that when the mechanical arm executes space grabbing, maintenance and other tasks, high-precision operation can be kept, and it can be guaranteed that the base is stable. Experimental results show that by means of the method, deviation of the attitude of the base can be remarkably reduced, the stability and efficiency of task execution are improved, and a feasible solution is provided for on-orbit service tasks of the space mechanical arm in the future.

In various examples, policy prediction-based motion planner systems and methods for autonomous and semi-autonomous systems and applications are provided. A scenario tree structure may be generated that represents potential behaviors of one or more peripheral agents based on perception data of a scene within which an ego vehicle operates. A joint MPC algorithm may optimize the motion of an ego vehicle within the context of the scenario tree structure to produce a policy tree structure. An MPC policy prediction model may be trained to predict the policy tree structures that a joint MPC algorithm would produce, given a set of environmental perception data. An ego vehicle may comprise a trained MPC policy prediction model that receives perception data, and based on that input predicts a policy tree structure that may be used to define a motion policy for navigating the ego vehicle through the scene.

The invention discloses a quadruped robot reinforcement learning motion planning method and system based on depth vision, and relates to the technical field of quadruped robot motion planning. The method comprises the steps that a simulation environment is built, modeling is carried out on stair terrains with different gradients and robot kinetic parameters, and a stair terrain model comprises stair terrains with different difficulty differences; the robot model is loaded to move in the stair terrain model, the movement condition of the robot is tracked, the speed tracking effect is obtained, and the speed tracking effect is obtained by conducting strategy evaluation on the joint torque value output by the teacher strategy network and the corresponding movement linear speed in the movement process of the robot; the speed tracking effect is evaluated, and environment parameters in the stair terrain model are adaptively adjusted according to the evaluation result. Through the reinforcement learning framework based on the depth visual information, the irregular stair terrain can be effectively learned, and the quadruped robot can stably work in a complex environment in a self-adaptive mode.

Systems and methods for generating attention masks are provided. In particular, a computing system can access sensor data and map data for an area around an autonomous vehicle. The computing system can generate a voxel grid representation of the sensor data and map data. The computing system can generate an attention mask based on the voxel grid representation. The computing system can generate, by using the voxel grid representation and the attention mask as input to a machine-learned model, an attention weighted feature map. The computing system can determine using the attention weighted feature map, a planning cost volume for an area around the autonomous vehicle. The computing system can select a trajectory for the autonomous vehicle based, at least in part, on the planning cost volume.