Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

796 results about "Motion planning" patented technology

Motion planning (also known as the navigation problem or the piano mover's problem) is a term used in robotics is to find a sequence of valid configurations that moves the robot from the source to destination.

Manipulator grabbing planning system and method based on visual identification

The invention discloses a manipulator grabbing planning system and method based on visual identification, and relates to the technical field of robots, the manipulator grabbing planning system comprises a visual perception module, a coordinate conversion module, a motion planning module, a control execution module, a tail end perception module and a system integration and communication module; the visual perception module realizes multi-view target detection, semantic segmentation and three-dimensional pose estimation through a binocular depth camera; the coordinate conversion module is used for converting a target pose under a camera coordinate system into a world coordinate under a mechanical arm base coordinate system; the motion planning module is responsible for generating a mechanical arm grabbing path, optimizing a strategy and supporting generalization migration of a multi-form mechanical arm; the control execution module drives the six-axis cooperative mechanical arm to complete the grabbing action, and vision-force mixed feedback closed-loop control is achieved. The tail end sensing module monitors the grabbing state in real time through a touch sensor and dynamically adjusts a grabbing strategy; and the system integration and communication module is used for realizing real-time data interaction and cooperative work among the modules of the system.
Owner:SHANGHAI AOTEBOG TECH DEV CO LTD

Motion control method and system for intelligent robot

The invention provides a motion control method and system for an intelligent robot, and the method comprises the steps: collecting environment and state data through an intelligent sensor group of a humanoid robot, inputting the environment and state data into a pre-training first neural network model, and obtaining motion prediction data and an environment analysis result; constructing a motion planning model, and performing energy consumption-stability multi-objective optimization on the joint motion track by adopting a preset first algorithm; the central controller generates a joint position, speed and torque reference trajectory based on an optimization result; and the local second controller of each joint locally adjusts the reference trajectory within the prediction time domain according to the real-time feedback. According to the invention, multiple sensors are combined with the mixed attention neural network to realize environment and self state intelligent perception, and the problem of multi-sensor data fusion time sequence dependence is solved; through energy consumption-stability multi-objective optimization, the complex environment movement efficiency is remarkably improved; the central controller and the local controller work cooperatively, and in combination with an edge computing architecture, the communication delay is reduced, and the system response speed is improved.
Owner:SHENZHEN ZHANDIAN SMART TECH CO LTD

Multi-mode body-equipped intelligent robot control method and device

The invention relates to the technical field of body-equipped intelligent robots, in particular to a multi-mode body-equipped intelligent robot control method and device, and the method comprises the steps: synchronously collecting visual, auditory, tactile, force sense and body perception information, and unifying the information to the same time-space reference through a cross-mode time-space stamp alignment mechanism; hierarchical feature extraction and fusion are carried out on the multi-modal information, and unified multi-modal scene state representation is generated; reasoning a decision based on the representation by using a body agent framework, and outputting a control instruction; motion planning and control, visual servo tracking in a non-contact stage and dynamic parameter correction in a contact stage are executed according to instructions; optimizing the multi-modal strategy network through an incremental strategy distillation mechanism based on the interactive data flow; the problem of space-time asynchronization of multi-modal sensing information is solved through a cross-modal space-time stamp alignment mechanism.
Owner:CHONGQING IND INTELLIGENCE TECHNOLOGY RESEARCH INSTITUTE

Mechanical arm natural language instruction control system and method based on large language model

The invention discloses a mechanical arm natural language instruction control system and method based on a large language model, and belongs to the field of intelligent manufacturing. Aiming at the limitation that traditional mechanical arm control depends on pre-programming and a static rule library, a dynamic mapping mode from a natural language instruction to an atomic action sequence is designed, an atomic skill library including detection, grabbing, moving, placement and other operations is constructed, and semantic analysis and a multi-mode cooperation technology are combined, so that the atomic action sequence is obtained. And support is provided for man-machine cooperation of a flexible assembly task. The method specifically comprises the steps that a DeepSeek-Distil-Llam-8B large model and a LoRA fine tuning technology are adopted, and a natural language instruction is converted into an executable atomic action sequence; based on a transfer learning optimized YOLOv8 target detection technology and a binocular vision positioning technology, a sensing module adaptive to an assembly scene is constructed and is fused with a mechanical arm motion planning module, and positioning grabbing of parts and tools is achieved. And an interactive interface is built by combining a voice-to-text large model and a Gradio front-end framework, so that the convenience of man-machine interaction is improved. By optimizing large model reasoning and motion planning cooperation efficiency, response delay from instructions to execution is reduced, and an efficient and extensible solution is provided for man-machine cooperation in intelligent manufacturing.
Owner:BEIJING INST OF TECH

Automatic welding robot based on artificial intelligence and welding system thereof

The invention relates to the technical field of artificial intelligence welding, and discloses an automatic welding robot based on artificial intelligence and a welding system thereof. The system comprises a welding feature acquisition module, a process parameter generation module, a real-time regulation and control module and a motion planning module. The welding characteristic acquisition module is used for acquiring three-dimensional contour data, material component information and welding seam geometric parameters of a workpiece to be welded, and generating a welding characteristic spectrum through characteristic fusion; a process parameter generation module retrieves a matching template from a welding knowledge base according to the parameters, and outputs a reference welding process parameter combination in combination with an environment temperature and humidity compensation coefficient; the real-time regulation and control module dynamically corrects the reference parameters to generate an optimization instruction according to the molten pool form, thermal radiation distribution and electric arc voiceprint characteristics in the welding process; and the motion planning module calculates a motion track, attitude parameters and a speed curve of the welding execution mechanism according to the optimization instruction to form a robot control instruction set. The system can improve the intelligent level of welding, guarantees stable welding quality, and is suitable for welding scenes of multiple manufacturing industries.
Owner:湖北金石炼化建设有限公司

Operation and maintenance manipulator intelligent control method and system based on visual identification

The invention discloses an operation and maintenance manipulator intelligent control method and system based on visual identification, and relates to the technical field of intelligent manipulator control, and the method comprises the steps: collecting RGB image data and depth image data of an operation and maintenance operation area, and obtaining a standardized image matrix and a mapping relation matrix; inputting the standardized image matrix into an improved ResNet residual network model, generating a comprehensive feature descriptor, and calculating a spatial position coordinate and an attitude angle of the target equipment based on the mapping relation matrix; based on the current joint angle state of the manipulator, an improved Jacobian matrix inverse kinematics algorithm is used for solving a target angle sequence of each joint, a preset operation mode library is matched based on the comprehensive feature descriptor, and a grabbing force parameter and a motion speed parameter are determined; and converting the target angle sequence into a control instruction, and sending the control instruction to each joint driver of the manipulator to drive the manipulator to complete action planning. According to the invention, full-process automation from environment perception to task execution is realized.
Owner:AOWEI TECH (NANJING) CO LTD

Autonomous vehicle motion planning

The present disclosure provides an example method that includes: (a) obtaining context data descriptive of an environment surrounding an autonomous vehicle, the context data based on map data and perception data; (b) generating, by a proposer and based on the context data: (i) a plurality of candidate trajectories, and (ii) a plurality of actor forecasts for a plurality of actors in the environment; (c) generating, by a ranker and based on the context data, the plurality of candidate trajectories, and the plurality of actor forecasts, a ranking of the plurality of candidate trajectories; and (d) controlling a motion of the autonomous vehicle based on a candidate trajectory selected based on the ranking of the plurality of candidate trajectories, wherein the proposer comprises a first machine-learned model and the ranker comprises a second machine-learned model, and wherein the first machine-learned model and the second machine-learned model use a common backbone architecture.
Owner:AURORA OPERATIONS INC

Training a Motion Planning System for an Autonomous Vehicle

The present disclosure provides an example method for obtaining labeled trajectories. The example method can include obtaining log data describing a trajectory of a vehicle traveling through an environment. The example method can include determining a suboptimal condition associated with the trajectory. The example method can include generating label data that characterizes the suboptimal condition along one or more constraint dimensions of a motion planner of the autonomous vehicle control system. The example method can include generating a training example for training the one or more machine-learned models of the autonomous vehicle control system to decrease a probability of the autonomous vehicle control system inducing the suboptimal condition.
Owner:AURORA OPERATIONS INC

Motion planning and control for robots in shared workspace employing look ahead planning

The structures and algorithms described herein employ look ahead motion planning, in which motion planning for a least two goals is performed before a robot executes the resulting motion plans, and the ability to transition between the preceding one of the motion plans to a subsequent (e.g. following) one of the motions plans is assessed. Thus, the system can determine whether a robot will get trapped (e.g., blocked by another robot) at the end of a first motion plan, preventing, limiting or delaying execution of a second motion plan. Detection of such a condition can cause one or more remedial actions can be taken, for example generating a new, revised or replacement first motion plan. Other remedial action can moving another robot, performing motion planning for the other robot, to alleviate a blocking condition and / or determining a new order for the goals.
Owner:REALTIME ROBOTICS INC

Mechanical arm control method and system based on image point cloud cross-modal fusion and action block Transformer

The invention belongs to the field of intelligent equipment, and relates to a mechanical arm control method and system based on image point cloud cross-modal fusion and action block Transformer, and the method comprises the steps: constructing a scene point cloud according to a multi-view RGB image and a corresponding depth map; a virtual view of multi-view fusion image-point cloud information is generated around the working space of the mechanical arm through the scene point cloud; inputting the virtual view of the multi-view fusion image-point cloud information into a feature extraction network to obtain a feature map; obtaining each joint state and a historical action sequence in the mechanical arm, and projecting the joint states; inputting the feature map, the projected joint state and the historical action sequence into an encoder to obtain encoding features; the coding features are input into a decoder, and a mechanical arm action sequence is predicted based on an action partitioning strategy; the mechanical arm is controlled according to the mechanical arm action sequence; according to the method, the multi-modal sensing data of the point cloud image are integrated into uniform feature representation, and intelligent action planning of the mechanical arm is realized in combination with the action block Transform, so that the global understanding and decision-making capability of the robot on complex tasks is improved.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Rolling-walking compound motion control method and equipment for double-wheel-foot robot

The invention belongs to the related technical field of mobile robots, and discloses a rolling-walking composite motion control method and device for a double-wheel-foot robot, and the method comprises the steps: (1) simplifying the rolling motion of wheels of the double-wheel-foot robot into the sliding motion of a sliding block; (2) constructing a system kinetic equation of the double-wheel-foot robot based on the simplified kinetic model; constructing a constraint equation based on the contact state of the two legs and the ground; discretizing the system kinetic equation, the cost function and the constraint equation, and converting model prediction control problems corresponding to the system kinetic equation, the cost function and the constraint equation into nonlinear problems for numerical solution so as to complete leg motion planning; (3) calculating the rotation speed of the wheel based on the leg motion planning result; and (4) based on the rotation speed of the wheels and the leg motion planning result, a whole-body control algorithm is adopted to calculate and obtain moment instructions of leg joints and the wheels. According to the invention, the state space dimension is reduced, and the wheel-ground contact constraint is simplified.
Owner:HUAZHONG UNIV OF SCI & TECH

Coordinated motion planning control method in five-axis machining process and related product

The invention relates to the technical field of numerical control machine tool precision detection and compensation, in particular to a coordinated motion planning control method and related products in the five-axis machining process, and the method comprises the steps: building a machine tool digital twin model, and combining constraint conditions; calculating a dynamic load rate and adjusting an error weight according to the load rate; an optimal cooperative motion instruction is obtained through multi-objective optimization; obtaining a feed-forward compensation amount and a feedback compensation amount; fusing the feedforward compensation amount and the feedback compensation amount to obtain a total compensation amount, and correcting the optimal cooperative motion instruction in real time to generate a final control signal; according to the method, dynamic constraints of different dimensions of a shaft are unified, the real-time dynamic load rate of the shaft is calculated, and weight coefficients of a translation shaft error and a rotation shaft error in an objective function are dynamically adjusted and optimized according to the difference; and constructing a multi-objective optimization function, solving an optimal cooperative motion instruction according to the dynamic weight, and integrating feed-forward compensation and feedback compensation to generate a final control signal so as to realize accurate control.
Owner:DEYANG JIECHUANG TECH CO LTD

Autonomous Vehicle Motion Planning

The present disclosure provides an example method that includes: (a) obtaining context data descriptive of an environment surrounding an autonomous vehicle, the context data based on map data and perception data; (b) generating, by a proposer and based on the context data: (i) a plurality of candidate trajectories, and (ii) a plurality of actor forecasts for a plurality of actors in the environment; (c) generating, by a ranker and based on the context data, the plurality of candidate trajectories, and the plurality of actor forecasts, a ranking of the plurality of candidate trajectories; and (d) controlling a motion of the autonomous vehicle based on a candidate trajectory selected based on the ranking of the plurality of candidate trajectories, wherein the proposer comprises a first machine-learned model and the ranker comprises a second machine-learned model, and wherein the first machine-learned model and the second machine-learned model use a common backbone architecture.
Owner:AURORA OPERATIONS INC

Policy prediction-based motion planner for autonomous systems and applications

In various examples, policy prediction-based motion planner systems and methods for autonomous and semi-autonomous systems and applications are provided. A scenario tree structure may be generated that represents potential behaviors of one or more peripheral agents based on perception data of a scene within which an ego vehicle operates. A joint MPC algorithm may optimize the motion of an ego vehicle within the context of the scenario tree structure to produce a policy tree structure. An MPC policy prediction model may be trained to predict the policy tree structures that a joint MPC algorithm would produce, given a set of environmental perception data. An ego vehicle may comprise a trained MPC policy prediction model that receives perception data, and based on that input predicts a policy tree structure that may be used to define a motion policy for navigating the ego vehicle through the scene.
Owner:NVIDIA CORP

Systems and Methods for Using Attention Masks to Improve Motion Planning

Systems and methods for generating attention masks are provided. In particular, a computing system can access sensor data and map data for an area around an autonomous vehicle. The computing system can generate a voxel grid representation of the sensor data and map data. The computing system can generate an attention mask based on the voxel grid representation. The computing system can generate, by using the voxel grid representation and the attention mask as input to a machine-learned model, an attention weighted feature map. The computing system can determine using the attention weighted feature map, a planning cost volume for an area around the autonomous vehicle. The computing system can select a trajectory for the autonomous vehicle based, at least in part, on the planning cost volume.
Owner:AURORA OPERATIONS INC

Autonomous aerial vehicle hardware configuration

An introduced autonomous aerial vehicle can include multiple cameras for capturing images of a surrounding physical environment that are utilized for motion planning by an autonomous navigation system. In some embodiments, the cameras can be integrated into one or more rotor assemblies that house powered rotors to free up space within the body of the aerial vehicle. In an example embodiment, an aerial vehicle includes multiple upward-facing cameras and multiple downward-facing cameras with overlapping fields of view to enable stereoscopic computer vision in a plurality of directions around the aerial vehicle. Similar camera arrangements can also be implemented in fixed-wing aerial vehicles.
Owner:SKYDIO INC

Motion planning for multiple robots in shared workspace

Collision detection useful in motion planning for robotics advantageously represents planned motions of each of a plurality of robots as obstacles when performing motion planning for any given robot in the plurality of robots that operate in a shared workspace, including taking into account the planned motions during collision assessment. Edges of a motion planning graph are assigned cost values, based at least in part on the collision assessment. Obstacles may be pruned as corresponding motions are completed. Motion planning requests may be queued, and some robots skipped, for example in response to an error or blocked condition.
Owner:REALTIME ROBOTICS INC

Task-motion planning for safe and efficient urban driving

Autonomous vehicles plan at a task level to compute a sequence of symbolic actions to fulfill service requests, where efficiency is the main concern. The vehicle computes continuous trajectories to perform actions at the motion level, where safety is important. Task-motion planning in autonomous driving faces the problem of maximizing task-level efficiency while ensuring motion-level safety. Task-Motion Planning for Urban Driving (TMPUD) enables the task and motion planners to communicate about the safety level of driving behaviors. The motion planner incrementally advances the vehicle toward a goal with an associated incremental utility, based on at least a safety of motion trajectories. The task planner defines the goal and a sequence of the actions to advance the vehicle toward the goal, dependent an optimization of aggregate prospective utility of the task and the safety of the motion trajectories.
Owner:THE RES FOUNDATION FOR THE STATE UNIV OF NEW YORK

Multi-modal motion control system, method and equipment for cleaning robot and medium

The invention provides a cleaning robot multi-mode motion control system, method and equipment and a medium, and belongs to the technical field of intelligent robot control. The system comprises a multi-modal data acquisition module used for performing multi-source synchronous acquisition on an environment image, distance information, a voice signal and a collision signal and outputting original sensing data; the data fusion and processing module is used for receiving the original sensing data, sequentially executing space-time alignment correction, multi-source feature fusion and instruction semantic analysis, and outputting an environment sensing result and a structured control instruction; the motion planning and decision-making module is used for executing global path planning, dynamic obstacle trajectory prediction and motion parameter decision-making based on the environment perception result and the structured control instruction, and outputting a motion control instruction containing a motion mode, a speed and an angle; and the motion control execution module is used for receiving the motion control instruction, converting the motion control instruction into an execution mechanism driving signal, realizing closed-loop control through real-time state feedback and outputting a motion execution result.
Owner:SHANDONG NEW GENERATION INFORMATION IND TECH RES INST CO LTD

Welding robot intelligent process redundancy driven obstacle avoidance motion planning method and system and computer equipment

The invention discloses an obstacle avoidance motion planning method and system for redundant drive of an intelligent process of a welding robot and computer equipment. According to the method, firstly, a motion path of a robot is planned based on welding process redundancy, then a sampling strategy that the position and posture of a welding gun are separated is adopted, then the sampled posture is mapped to a planar two-dimensional space, and the relevance between an obstacle collision boundary and the obstacle avoidance posture of the welding gun is constructed; an incremental extreme gradient lifting model is adopted to carry out probability modeling on the collision boundary and obstacle avoidance attitude relevance, nearest neighbor search and new planning node expansion are carried out based on target cost to obtain new planning nodes, collision detection is carried out on the new planning nodes, and the new planning nodes passing the collision detection are used for incremental learning of an obstacle avoidance model; and finally, an optimization strategy of spherical linear interpolation and cosine slow motion time mapping is adopted to ensure continuity and smoothness of a motion planning path trace. The problem that in the prior art, it is difficult to quickly calculate and generate a collision-free welding track is solved.
Owner:SOUTH CHINA UNIV OF TECH

Large model-based medical double-arm robot task and motion planning method and system

The invention discloses a medical double-arm robot task and motion planning method and system based on a large model, and the method comprises the steps: obtaining the pose information of a to-be-operated object of a double-arm robot through an RGB-D camera and an AprilTag marking system according to a given medical preparation experiment scene; based on the pre-trained vision-language large model GPT-4V and the designed environment understanding prompt project, the relative position relation of the two-arm to-be-operated object is obtained; environment understanding information, user task requirements and designed task planning prompt projects are input into the large model to generate a specific action sequence of the double-arm robot; analyzing information of a motion starting point and a target point corresponding to the motion sequence on the basis of a large model and a designed motion planning analysis prompt project; and calling a robot open source motion library to plan a feasible path from a motion starting point to a target point, obtaining a motion track of the double-arm robot, and performing safe execution. And the flexibility and the operation efficiency of the medical double-arm robot in a complex dispensing environment are improved.
Owner:HUNAN UNIV

Man-machine collaborative decision planning system and method based on environment adaptive trajectory optimization

A man-machine collaborative decision-making planning system and method based on environment adaptive trajectory optimization comprises a man-machine interaction module, a rule-based motion planning module and a learning-based parameter generation module, the man-machine interaction module performs data processing such as environment perception and state estimation according to data collected by an onboard sensor and a remote controller, and the parameter generation module generates parameter parameters according to the data. Input information required by the motion planning and parameter generation module is obtained; the motion planning module performs topological path search, visibility detection and trajectory optimization processing according to the local map and the user instruction information to obtain a planned trajectory of unmanned aerial vehicle tracking control; and the parameter generation module performs environment adaptive strategy processing according to the distribution information of the trajectory in the environment to obtain a speed constraint parameter for adjusting motion planning. According to the method, semantic information and depth information in environmental perception are considered, the speed parameters of the planner are automatically and dynamically adjusted, and the adaptability of the planned trajectory to different scenes is effectively improved, so that the cognitive and operation burden of a pilot in a complex environment is reduced, and the navigation efficiency and the system safety are improved.
Owner:SHANGHAI JIAOTONG UNIV

Musculoskeletal virtual dexterous hand shape control method and system based on layering strategy

The invention discloses a musculoskeletal virtual dexterous hand shape control method and a musculoskeletal virtual dexterous hand shape control system based on a layering strategy. The core of the method is to decouple a complex muscle control task into two layers of high-layer kinematics planning and bottom-layer dynamics mapping. Performing motion planning in a low-dimensional joint space by adopting a deep reinforcement learning algorithm in a high-level hand shape simulation layer to generate a target joint angle sequence; in the muscle control layer of the bottom layer, through a neural network subjected to supervised learning training, joint instructions planned on the upper layer are efficiently and accurately mapped into activation signals for driving multiple paths of redundant muscles. Through the hierarchical decoupling strategy, the high-level controller avoids the problem of direct exploration in the native high-dimensional muscle space, and the bottom-level controller is specially responsible for solving the problem of complex nonlinear mapping from a moving target to redundant muscle driving. According to the method, the learning efficiency and stability of the control strategy are remarkably improved, and the final precision of hand shape simulation is improved.
Owner:SOUTHEAST UNIV

Cluster adaptive path planning method based on geometric PDE and PINN

The invention discloses a cluster adaptive path planning method based on geometric PDE and PINN. The method comprises the following steps: firstly, modeling a dynamic model of a fault cluster with time-varying topology; secondly, through Riemannian manifold and gradient evolution based on heat flow, a fault cluster is constructed based on a parabolic PDE system, and a reference trajectory of a cluster system in an obstacle environment is obtained; and learning a steady-state solution of the cluster based on a parabolic PDE system by using a physical information neural network (PINN), and obtaining a fault-tolerant optimal control law of the cluster system based on geometric PDE and PINN under the conditions of time-varying topology, actuator fault and obstacle operation. According to the method, it is ensured that the intelligent agent navigates to the target position from the initial position along the dynamic reference trajectory, meanwhile, endogenous compensation of the fault effect is achieved, and unified solution of fault compensation and motion planning is achieved.
Owner:HANGZHOU DIANZI UNIV +2

Industrial robot motion planning method and device and storage medium

The invention relates to the field of artificial intelligence, and provides an industrial robot motion planning method which comprises the following steps: constructing a digital environment model containing a spatial topological relation by collecting real-time dynamic data of a working environment of an industrial robot; constructing a mixed deep learning model containing a graph convolutional network and a visual Transform module; generating a dynamic cost map containing safe passing area and obstacle area identifiers; based on the dynamic cost map, adopting a reinforcement learning decision algorithm to generate an initial motion path, and performing real-time path adjustment through a local planning algorithm; kinematics smoothing processing is carried out on the adjusted path, and a continuous motion track of each joint of the industrial robot is generated; and in the execution process, closed-loop correction is carried out on the continuous movement track of each joint of the industrial robot through feedback of the sensor, and the digital environment model is updated based on the correction result. According to the technical scheme, high-precision real-time motion planning of the robot in a dynamic environment can be realized.
Owner:SHENZHEN DACHUAN SOFTWARE CO LTD

Double-arm collaborative robot pipeline butt joint method based on relative Jacobi

The invention relates to the technical field of two-arm robot pipeline butt joint cooperative motion planning and control in a complex environment, in particular to a two-arm cooperative robot pipeline butt joint method based on relative Jacobi. According to the method, a double-arm kinematics equation relative to a Jacobian matrix is constructed, constraint conditions are constructed in combination with pipeline butt joint requirements, pose errors and integral information are introduced to design an anti-noise motion planning scheme, an end effector obstacle avoidance strategy is designed, the anti-noise scheme is converted into a quadratic optimization problem, and the quadratic optimization problem is solved through a numerical algorithm. According to the method, the problems that an existing scheme is insufficient in precision, inaccurate in constraint condition and poor in anti-noise and obstacle avoidance capacity are solved, the uniformity and accuracy of butt joint of the two-arm pipelines are improved, the influence of environmental noise is effectively restrained, butt joint safety is guaranteed, butt joint efficiency and reliability are improved, and the high-precision pipeline butt joint requirement under the complex environment is met.
Owner:HAINAN UNIV

Mechanical arm motion planning method based on vision-language multi-modal data

The invention provides a mechanical arm motion planning method based on vision-language multi-modal data. The method comprises the following steps: acquiring a human language instruction, and controlling an image acquisition device of the robot to acquire an environment image; generating multi-modal data based on the human language instruction and the environment image by utilizing a multi-modal data generation module; encoding the human language instruction by utilizing a language model to obtain a text language vector; performing absolute position coding on the environment image to obtain a visual coding vector; fusing the text language vector and the visual coding vector to obtain multi-modal data; and inputting the multi-modal data into the trained multi-modal motion control model to obtain a motion planning track of the mechanical arm. A target track generated by the mechanical arm under manual control is used for reinforcement learning to train a multi-modal motion control model, the motion planning path and strategy of the mechanical arm are optimized, and the mechanical arm can achieve the motion effect approaching human beings.
Owner:TONGLIAO HUOLINHE KENGKOU POWER GENERATION CO LTD

Intelligent oil containment boom mechanical arm device and control method thereof

The invention relates to an intelligent oil containment boom mechanical arm device and a control method thereof. The intelligent oil containment boom mechanical arm device comprises a mechanical arm, a hydraulic driving system, a sensing system and a control unit. The mechanical arm comprises a transmission base, a steering stand column, a large arm, a small arm, an overturning head and a tail end grabbing hook assembly. The hydraulic driving system comprises a hydraulic pump station, a proportional reversing electromagnetic valve group and a plurality of execution oil cylinders; the sensing system comprises a gyroscope array and a camera array to form a distributed attitude sensing network and a distributed visual sensing network; the control unit identifies and positions an oil containment boom target based on a sea surface image acquired by the distributed visual perception network, performs motion planning on the mechanical arm, and controls the mechanical arm to move and grab the target; in the target positioning process, posture changes of all parts of the mechanical arm are sensed in real time based on the distributed posture sensing network, and dynamic compensation is conducted on the posture error of the mechanical arm. According to the device, the oil containment boom can be quickly, accurately, reliably and intelligently grabbed, and the laying efficiency and safety of the oil containment boom are improved.
Owner:GUANGZHOU COSCO SHIPPING JINGHAI ENVIRONMENTAL PROTECTION TECHNOLOGY CO LTD +1

Mechanical arm automatic control method and system based on target recognition

The invention relates to the technical field of mechanical arm automatic control, and discloses a mechanical arm automatic control method and system based on target recognition. According to the method, position, speed and mechanical arm joint angle data of a target object are collected through a vision sensor, a speed sensor, an angle sensor and the like, a control moment prediction value is obtained through a prediction model, and the prediction value is corrected in combination with the data matching degree and the correlation degree; clustering the three-dimensional data set, and analyzing similarity under different sequence lengths to obtain a historical predicted value; and according to the corrected predicted value, the historical predicted value and the data similarity, a mechanical arm track predicted value is obtained, and then motion planning control is conducted. The system comprises a memory, a processor and related programs. According to the method, the trajectory prediction accuracy and self-adaptability of the mechanical arm are improved, the dynamic working environment can be effectively handled, and the control precision and efficiency are improved.
Owner:XIAN DASHENG TECH CO LTD

Industrial robot motion planning system and method based on artificial intelligence

The invention discloses an industrial robot motion planning system and method based on artificial intelligence, and belongs to the field of industrial robot intelligent obstacle avoidance. Based on a visual sensor, a depth sensor and a material priori library, feature extraction and attribute judgment are performed on an obstacle, and the risk is preliminarily evaluated; based on a force control sensor at the tail end of the robot and low-speed contact motion, contact force and displacement curve changes are obtained, the deformable characteristic of an obstacle is evaluated, and based on a deformation potential value, contact force and displacement curve analysis, contact stress calculation and obstacle energy absorption calculation, the damage risk of the obstacle is evaluated. The contact impact energy and the damage coefficient of the robot are evaluated based on the quality, the running speed and the anti-seismic coefficient of the industrial robot, the danger of the robot facing the obstacle is comprehensively evaluated based on the obstacle initial judgment risk, the damage probability and the robot damage coefficient, and the working efficiency and safety of the industrial robot are improved.
Owner:NANCHANG HANGTIAN GUANGXIN TECH CO LTD +1