Robot joint control method and device, electronic equipment and storage medium
Through machine vision, scene information is obtained and control command parameters are generated. Combined with feedforward control technology, the problem of response delay of bionic robots in unknown scenarios is solved, and dynamic response speed and motion stability are improved.
Patent Information
- Application Number
- CN202411198272.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The joints of bionic robots have delays in action responses in unknown application scenarios, making it difficult to quickly adjust to achieve flexible and stable motion, especially in complex and uncertain environments.
Using machine vision-based working condition prediction method, the control command parameters of joint motors are generated by obtaining scene road conditions and operation scene images, and feedforward control technology is used to superimpose these parameters on the joint motor control system to optimize the robot in unknown scenarios. dynamic response.
It improves the dynamic response speed of the robot in unknown scenarios, enhances the stability and flexibility of motion, and reduces the response delay caused by insufficient familiarity with the scene.
Smart Images

Figure CN118952205A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bionic robots, and in particular to a control method, device, electronic equipment and storage medium for robot joints. Background Art
[0002] The electric joints of bionic robots need to be controlled and adjusted in real time according to the application scenario conditions to achieve synchronization and coordination of flexible movements of the whole body. Due to the complexity and uncertainty of the application scenario conditions, the joint system needs to have the ability to respond and adjust quickly.
[0003] The core components of robot joints generally include motors, motor drivers, reducers and sensors. A technical solution to improve the response speed of robot joints is to increase the motor time constant, adopt faster signal acquisition circuits and main control circuits, and optimize the motor control algorithm during the design of robot joints. Although this method can improve the dynamic response speed of the robot, due to the uncertainty of the environment in which the robot is located, the robot's action response will still have a certain delay in unknown application scenarios, such as sudden changes in road conditions and loads. The present invention proposes a robot joint position and torque feedforward control system based on machine vision working condition prediction, which can effectively optimize and solve the above problems. Summary of the invention
[0004] The embodiments of the present invention provide a control method, device, electronic device and storage medium for a robot joint, which improve the dynamic response speed of the robot joint when operating in unknown scenarios and enhance the flexibility and stability of the robot operation.
[0005] According to one aspect of an embodiment of the present invention, a control method for a robot joint is provided, comprising: obtaining a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot, and an operating scene image of the scene in which the robot is currently located; performing path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; generating control instruction parameters for the robot joint motor based on the road condition status and the operating scene image, wherein the control instruction parameters are used to control the robot's joints to drive the robot to run from the current position to the working position and perform the operating task, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters; using a feedforward control method to superimpose the control instruction parameters on the instructions of the robot joint motor control system to control the robot's joints to drive the robot to move along the operating path and perform the operating task.
[0006] Optionally, control instruction parameters of the robot joint motors are generated based on road conditions and work scene images, including: using a first neural network model to identify a target object in the work scene image to obtain an object type and an object position of the target object, wherein the target object is an object of a work task to be performed by the robot; determining the robot's work load parameters based on the object type; determining the robot's work loading time based on the object position; and generating control instruction parameters of the robot's joint motors based on the work load parameters, the work loading time and road conditions.
[0007] Optionally, control instruction parameters of the robot joint motors are generated based on the job load parameters, job loading duration and road conditions, including: using a second neural network model to generate control instruction parameters of the robot joint motors based on the job load parameters, job loading duration and road conditions.
[0008] Optionally, the control instruction parameters are superimposed on the instructions of the robot joint motor control system by a feedforward control method, and the robot joints are controlled to drive the robot to move along the running path and perform the working task, including: the control instruction parameters are superimposed on the instructions of the robot joint motor control system by a feedforward control method, and the robot joints are controlled by the joint motor control system to drive the robot to move along the running path and perform the working task, wherein the joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.
[0009] Optionally, the control instruction parameters include: joint motor position control instruction parameters, joint motor torque control instruction parameters, and the control instruction parameters are superimposed on the instructions of the robot joint motor control system by feedforward control. The joint motor control system controls the robot's joints to drive the robot to move along the running path and perform work tasks, including: adding the joint motor position control instruction parameters to the joint motor position control loop to obtain the joint motor position control instruction, and adding the joint motor torque control instruction parameters to the joint motor current control loop to obtain the joint motor torque current control instruction, and controlling the robot's joints to drive the robot to move along the running path and perform work tasks through the joint motor control system.
[0010] Optionally, a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path and the road condition status of the running path, including: performing path planning on the robot based on the current position and the working position to obtain the running path; identifying the road environment image of the running path in the scene road condition image to obtain the road condition status.
[0011] Optionally, the method further includes: training model parameters of the first initial neural network model based on a sample database of target object types in job scenarios to obtain a first neural network model.
[0012] Optionally, the method further includes: training model parameters of a second initial neural network model based on robot motion control historical data and simulation data to obtain a second neural network model.
[0013] According to another aspect of an embodiment of the present invention, a control device for a robot is also provided, including: an acquisition module, used to acquire a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and the working scene image of the scene in which the robot is currently located; a planning module, used to perform path planning analysis on the robot based on the current position, the working position and the scene road condition image, and obtain the robot's running path and the road condition status of the running path; a generation module, used to generate control instruction parameters of the robot's joint motors based on the road condition status and the working scene image, wherein the control instruction parameters are used to control the robot's joints to drive the robot to run from the current position to the working position and perform the working task, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters; an execution module, used to superimpose the control instruction parameters on the instructions of the robot's joint motor control system in a feedforward control manner, and control the robot's joints to drive the robot to move along the running path and perform the working task.
[0014] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an operating scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; control instruction parameters of the robot's joint motors are generated based on the road condition status and the operating scene image; and the control instruction parameters are superimposed on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The robot's proper action characteristics in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby achieving the purpose of improving the robot's joint response speed in unknown scenes, and achieving the technical effect of improving the robot's dynamic response speed of operation and enhancing the stability and flexibility of the robot's movement. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0016] Figure 1 is a flow chart of a method for controlling a robot joint according to an embodiment of the present invention;
[0017] Figure 2 is a schematic diagram of an optional robot joint motor control system according to an embodiment of the present invention;
[0018] Figure 3 is a schematic diagram of an optional robot joint workload estimation unit according to an embodiment of the present invention;
[0019] Figure 4 is a schematic diagram of an optional robot joint position and torque control instruction parameter calculation unit generating control instruction parameters according to an embodiment of the present invention;
[0020] Figure 5 is a schematic diagram of generating robot joint control instruction parameters based on a path road condition and robot workload according to an embodiment of the present invention;
[0021] Figure 6 is a schematic diagram of a robot joint control device according to an embodiment of the present invention;
[0022] Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0025] Example 1
[0026] According to an embodiment of the present invention, an embodiment of a control method for a robot joint is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0027] Figure 1 is a flow chart of a method for controlling a robot joint according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0028] Step S102, obtaining a scene road condition image of the scene where the robot is currently located, the current position of the robot, the working position of the robot, and the working scene image of the scene where the robot is currently located.
[0029] The scene road condition image in the above steps refers to the environment image between the current position of the robot and the working position, including elements such as roads, traffic signs, vehicles, pedestrians, etc., which are used to navigate the robot or help the robot make decisions. These images can be obtained through cameras or other sensors, and then analyzed and processed through computer vision technology to help the robot identify and understand the current road conditions, but the method of obtaining images and the technology used to process images are not limited to this. Acquiring scene road condition images can enable the robot to understand the environment in advance, which is conducive to improving the speed at which the robot performs its work tasks.
[0030] The current position of the robot in the above steps refers to the precise coordinates or specific location of the robot. The current position of the robot can be determined by a sensor or a global positioning system (GPS), but the method for determining the current position of the robot is not limited to this. The current position of the robot can be used to guide the robot to complete a specific task or navigate to a specific location.
[0031] The work position in the above steps is the position where the robot is to perform the work task, which can be a specific workstation, a position on the production line, or a specific area, but is not limited to this. When performing the work task, the robot needs to be able to accurately identify and reach this position in order to perform the pre-set work task.
[0032] The working scene image in the above steps is an environmental image of the robot's working position and an image of the working object, which can represent the environment in which the robot performs the work, the volume, weight and spatial position of the working object.
[0033] In an optional embodiment, the vision and perception system installed inside the robot can collect the scene road image of the scene the robot is currently in, the current position of the robot, the working position of the robot, and the working scene image of the scene the robot is currently in. When obtaining the current position of the robot, the robot vision system and simultaneous localization and mapping (SLAM) technology can be used to determine the current position of the robot.
[0034] When obtaining the scene road condition image of the scene currently located by the robot, the camera equipped in the robot vision system can be used to capture the road condition image of the current scene in real time, and then the robot vision and perception system uses image processing technology to analyze and identify the road condition image to obtain the scene road condition image of the scene currently located by the robot. The laser radar sensor in the robot vision system can also be used to scan the surrounding environment and generate point cloud data. The robot vision system obtains the point cloud data and constructs a three-dimensional model of the environment, thereby obtaining the scene road condition image of the scene currently located by the robot. However, the method of using the robot vision and perception system to obtain the scene road condition image of the scene currently located by the robot is not limited to this.
[0035] When using the robot vision and perception system to obtain the work position and work scene image, the camera or sensor in the robot vision and perception system can be used to scan the work area, obtain the work scene image of the work area, and identify the features in the work area, such as markings, colors, shapes, etc., in order to determine the robot's working position so that the robot can perform the work task at the accurate working position, but the method of using the robot vision and perception system to obtain the work scene image and work position is not limited to this.
[0036] When obtaining the current position of the robot, the historical scan data generated by the laser radar sensor in the robot's vision and perception system that previously scanned the surrounding environment can be obtained, and then feature extraction can be performed to extract landmark points from the laser radar data, convert the extracted landmark points into map data, and build an environmental map based on the map data. Using the constructed environmental map and the real-time scan data of the laser radar sensor, the robot is positioned using the SLAM algorithm, that is, the robot's position in the map is determined.
[0037] In another optional embodiment, when obtaining the scene road condition image of the scene currently located by the robot, an ultrasonic sensor can be used to help the robot detect the shape of the road and the distance and position of the surrounding obstacles, thereby generating a road condition image and scene information, and then collecting the data obtained by the ultrasonic sensor, and converting the data into a scene road condition image through image processing algorithms and computer vision technology, and deep learning and neural network technology can be used to improve the accuracy and efficiency of image processing, thereby obtaining the scene road condition image of the scene currently located by the robot. However, the method of obtaining the scene road condition image of the scene currently located by the robot is not limited to this.
[0038] When obtaining the current position of the robot, an acceleration sensor or a velocity sensor can be used to track the movement of the robot to infer the position of the robot. An external positioning system, such as base station positioning, wireless positioning, etc., can also be used to obtain the current position information of the robot, but the method of obtaining the current position of the robot is not limited to this.
[0039] When obtaining the work position and the work scene image, the work position can be obtained from the work scheduling system, and then transmitted to the robot's control system through the network. The robot can obtain the position of the work task to be performed by receiving this information, but the method of obtaining the work position is not limited to this. Then, based on the work position, the ultrasonic sensor is used to detect the environmental characteristics of the work area, collect the data obtained by the ultrasonic sensor, and convert the data into the work scene image through image processing algorithms and computer vision technology.
[0040] Step S104, performing path planning analysis on the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path and the road condition status of the running path.
[0041] The running path in the above steps is the walking path of the robot from the current position to the working position. When planning the running path, factors such as distance and obstacles need to be considered. The robot needs to walk along the planned running path and move from the current position to the working position.
[0042] The road condition of the running path in the above steps is the road condition corresponding to the running path, and the road condition may include but is not limited to the road slope, step height, and road unevenness, etc. The robot's actions can be adjusted according to the road condition of the robot's running path to ensure safety and smooth completion of the task.
[0043] In an optional embodiment, the path planning unit in the robot joint position and torque control instruction parameter calculation unit can plan the path of the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path. Based on the current position and the working position, the area with a high probability of the robot's path is divided, and then the scene road condition image corresponding to the area is processed and recognized to identify information such as roads, obstacles, and traffic signs, and an environmental map of the area is constructed. The environmental map is processed using a path planning algorithm. The path planning algorithm can be a Dijkstra algorithm, a minimum spanning tree algorithm, or a two-way search algorithm, but is not limited thereto. By processing the environmental map with a path planning algorithm, the robot's running path can be obtained, so that the robot can reach the designated working position safely and efficiently.
[0044] The expected path condition analysis unit in the robot joint position and torque control instruction parameter calculation unit can analyze the robot's running path based on the scene road condition image to obtain the road condition state. The road condition state includes but is not limited to the road slope, step height, and road unevenness. Specifically, after determining the robot's running path, computer vision technology can be used to identify the road slope information in the image, or an image processing algorithm can be used to detect the horizontal and vertical lines in the image, and then the road slope is calculated according to the position and angle of these lines, but the way to obtain the road slope in the road condition state is not limited to this. In addition, the step edge in the image can be detected by an image processing algorithm, and then the height of the step is calculated according to the position and length of the edge, so as to obtain the step height in the road condition state. The sensor carried by the robot can be used to detect the unevenness of the road surface. For example, a laser radar sensor can be used to scan the road surface, and then the expected path condition analysis unit in the robot joint expected torque and current control instruction calculation unit obtains the scan data and analyzes the scan data to obtain the unevenness of the road surface in the road condition state. Through the above method, the road condition status of the robot's running path can be obtained. By obtaining the road condition status of the robot's running path, the changes in the robot's motion control working conditions can be estimated in advance, so as to control the robot in advance to make actions that meet the road conditions and improve the robot's response speed.
[0045] Step S106, generating control instruction parameters for the robot joint motors based on the road condition and the working scene image, wherein the control instruction parameters are used to control the robot joints to drive the robot from the current position to the working position and perform the working task, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters.
[0046] The robot joint motors in the above steps are used to control the robot to operate and perform tasks.
[0047] The control instruction parameters in the above steps are parameters used to control the movement, behavior and function of the robot joints. They can control the robot joints to drive the robot along the running path and perform the task, and make the robot behave in a manner suitable for the current road condition according to the scene road condition image and the existing road condition. The control instruction parameters can be adjusted according to the specific task and environment to achieve the robot's autonomous behavior and interactive capabilities.
[0048] The joint motor position control instruction parameters in the above steps are used to describe the target motion position of the motor of each action joint of the robot.
[0049] The joint motor torque control instruction parameters in the above steps are used to control and adjust the torque output of each joint of the robot to achieve precise motion control and force regulation.
[0050] In an optional embodiment, existing road condition data and work scene image data are collected as historical road condition data and historical work scene image data, and the historical road condition data and historical work scene image data are preprocessed, including image denoising, feature extraction and other operations, so as to be used as input for the machine learning model. The preprocessed historical road condition data and historical scene road condition image data are input into the machine learning model to train the machine learning model. The trained machine learning model is used to establish a mapping model between the road condition state and the control instruction parameters, and the established mapping model is used to generate the control instruction parameters of the robot in real time according to the current road condition state.
[0051] In another optional embodiment, a pre-trained neural network model is used to process the work scene image to identify the object that the robot needs to work on. The features of the object are extracted, and the time and load required for the robot to perform the work are estimated based on the object features. The control instruction parameters are generated based on the time and load required to perform the work and the work scene image.
[0052] The control instruction parameters include joint motor position control instruction parameters and joint motor torque control instruction parameters. The joint motor position control instruction parameters are used to determine the target position of the joint motor for each action during the robot's motion process. The joint motor torque control instruction parameters are used to control the joint motor torque output of the robot when performing each action. The joint motor position control instruction parameters and the joint motor torque control instruction parameters are used to jointly control the robot's joint movements, thereby driving the robot to run from the current position to the working position and perform the working task.
[0053] Step S108, using a feedforward control method to superimpose the control instruction parameters onto the instructions of the robot joint motor control system, controlling the robot joints to drive the robot to move along the running path and perform the task.
[0054] The feedforward control in the above steps is a control system design method. It generates control command parameters of the joint motor in advance according to the predicted changes in the robot joint working conditions, superimposes them with the existing position, speed, and current commands in the motor control system, and feeds back the existing joint motor position, speed, and current to realize the control function of the robot joint motor control system. The feedforward control method can enable the robot joint motor control system to respond quickly when disturbed or changed, and achieve better control performance.
[0055] The robot joint motor control system in the above steps is a system that controls the robot's joints to drive the robot to move along the running path and perform the working task.
[0056] The work tasks in the above steps are the work tasks that the robot needs to perform, which can be security tasks such as patrolling, monitoring, emergency rescue, etc., or medical care such as surgical assistance, rehabilitation training, and elderly care, or cleaning and maintenance tasks such as cleaning the floor, cleaning the glass curtain wall, cleaning the aquarium, and cleaning the pond, but the work tasks that the robot can complete are not limited to these.
[0057] In an optional embodiment, the robot joint motor control system consists of position control, speed control and current control loops, and the joint motor position control instruction parameters and the joint motor torque control instruction parameters can be superimposed on the instructions of the robot joint motor control system through feedforward control.
[0058] Figure 2 is a schematic diagram of an optional robot joint motor control system according to an embodiment of the present invention, such as Figure 2 As shown, the robot joint motor control system 200 includes a motor, an encoder, a torque control sensor and a reduction gear, wherein the encoder can perform speed and position measurement, and obtain joint motor position feedback and speed feedback after decoding. The current reference position control instruction is subtracted from the position feedback through a subtractor, and the motor position control is realized as a position control input, and the position control output is subtracted from the speed feedback through a subtractor, and the motor speed control is realized as a speed control input. The current in the motor abc coordinate system is converted into the current in the dq coordinate system, and then the d-axis and q-axis current feedback are obtained. Among them, the abc coordinate system is a stationary coordinate system based on a three-phase motor, which is used to describe the current and voltage vectors of the motor in a three-phase circuit; the dq coordinate system is a rectangular coordinate system, and the d-axis and q-axis are two axes in the rectangular coordinate system, which are usually used to describe the current and voltage rotating vectors in the motor control system. The output of the speed control is subtracted from the current feedback through a subtractor, and the motor current control is realized as a current control input. Then the generation and power drive of pulse width modulation are realized. The current position control instruction and the joint motor position control instruction parameter can be superimposed by an adder to realize position feedforward control. The torque control command parameter is theoretically converted into the torque current control command parameter and then superimposed with the speed control output through an adder to realize torque feedforward control.
[0059] By using feedforward control to superimpose control command parameters onto the robot joint motor control system commands, the position and current control loops can be adjusted to improve the robot's torque control dynamic response. By applying specific current and voltage to the motor to achieve the required speed and torque, the robot can move along the operating path and perform tasks.
[0060] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an working scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; joint motor control instruction parameters of the robot at the working position are generated based on the road condition status and the scene road condition image, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters; the control instruction parameters are superimposed on the instructions of the robot's joint motor control system in a feedforward control manner to control the robot's joints to drive the robot to move along the operating path and perform the working task. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The action characteristics of the robot joints in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby realizing the purpose of improving the response speed of the robot joints in unknown scenes, and achieving the technical effect of improving the dynamic response speed of the robot operation and enhancing the stability and flexibility of the robot movement.
[0061] Optionally, control instruction parameters of the robot joint motors are generated based on road conditions and work scene images, including: using a first neural network model to identify a target object in the work scene image to obtain an object type and an object position of the target object, wherein the target object is an object of a work task to be performed by the robot; determining the robot's work load parameters based on the object type; determining the robot's work loading time based on the object position; and generating control instruction parameters of the robot's joint motors based on the work load parameters, the work loading time and road conditions.
[0062] In the above steps, the target object is the object that the robot needs to operate when performing the task. For example, when the robot performs the security patrol task, the target object is an area, and when the robot performs the task of carrying objects, the target object is one or more objects. However, the above target objects are only examples, and the actual target objects are not limited to these.
[0063] The object type in the above steps is the type of object that the robot needs to operate when performing the task, which can characterize the type characteristics of the target object and can be used to determine the robot's operation load parameters. For example, when the robot performs the task of carrying items, the target object is an item, and the object type is a rectangular item with a length of 0.5 meters, a width of 0.3 meters, and a height of 0.3 meters. The load of the robot to complete this operation can be determined based on the characteristics of the object type. It should be noted that the above examples are only examples, and the actual target objects and object types are not limited to this.
[0064] The object position in the above steps is the position of the object that the robot needs to operate when performing the task, which can characterize the position characteristics of the target object and can be used to determine the robot's task loading time.
[0065] The workload parameters in the above steps may include but are not limited to weight, volume, etc., which can be used to determine the load that the robot needs to bear to complete the preset workload.
[0066] The job loading duration in the above steps is the time required for the robot to move from the current position to the target object position along the running path. The time required for the robot to move from the current position to the target object position can be determined by the current position of the robot, the position of the target object, and the running path of the robot. Since the robot is in a job loading state during the movement on the running path, this time is the job loading duration.
[0067] The first neural network model in the above steps is a model that can recognize objects based on video images, and is used to identify the object type and object position of the target object, and can obtain the robot's job load parameters and job loading time based on the object type and object position.
[0068] In an optional instance, the video image object recognition function of a convolutional neural network model can be trained using video image data of multiple object samples. The video image data of multiple object samples may include video frames of objects of multiple categories and label information associated with each video frame. The trained convolutional neural network model is used as the first neural network model.
[0069] The robot workload estimation unit may generate control instruction parameters using the first neural network model. Figure 3 is a schematic diagram of an optional robot joint workload estimation unit according to an embodiment of the present invention, such as Figure 3 As shown, the first neural network model in the robot workload estimation unit, namely Figure 3The large workload model in the robot obtains the operating scene image data from the robot vision and perception system, processes the operating scene image data, identifies the target object to be operated by the robot, and parses the target object to obtain the object matching result and object position, that is, the object type and object position of the target object.
[0070] The object matching result, that is, the object type of the target object, is analyzed, and the target object that processes the same object type is found from the historical data, and the expected workload volume and weight are calculated, thereby determining the workload parameters of the robot.
[0071] The object position, that is, the object position of the target object, is analyzed to calculate the expected job load loading time, thereby determining the job loading time of the robot.
[0072] According to the above process, the robot workload estimation unit can determine the expected workload and loading time of the robot, that is, the workload parameters of the robot and the workload loading duration of the robot. A mapping model between workload parameters, workload loading duration, road conditions and robot control instruction parameters is established through a regression analysis model. Using the established mapping model, the robot control instruction parameters, such as joint motor position control instruction parameters, joint motor torque control instruction parameters, etc., are generated in real time according to the current workload parameters, workload loading duration and road conditions.
[0073] In another optional embodiment, Figure 4 is a schematic diagram of an optional robot joint position and torque control instruction parameter calculation unit generating control instruction parameters according to an embodiment of the present invention, such as Figure 4 As shown, the application scenario, that is, the scene road condition image mentioned above, the current position of the robot, the robot's operating position, and the operating scene image are input into the robot joint position and torque control instruction parameter calculation unit 400. The robot vision and perception system in the robot joint position and torque control instruction parameter calculation unit 400 performs path planning based on the application scenario to obtain the expected path condition, that is, the road condition state mentioned above. At the same time, the robot vision and perception system also uses the first neural network model to identify the operating scene image in the application scenario, estimate the robot's operating load, and obtain the operating load parameters and the operating loading duration. According to the operating load parameters and the operating loading duration obtained by estimating the expected path condition and the robot's operating load, the joint position and torque control instruction parameters under the expected condition are calculated to obtain the control instruction parameters of the robot joint motor, and the control instruction parameters of the robot joint motor are applied to the joint motor feedforward and feedback control.
[0074] Optionally, control instruction parameters of the robot joint motors are generated based on the job load parameters, job loading duration and road conditions, including: using a second neural network model to generate control instruction parameters of the robot joint motors based on the job load parameters, job loading duration and road conditions.
[0075] The second neural network model in the above steps can calculate the control instruction parameters of the robot according to the road condition, the expected load parameters of the operation and the loading time. The control instruction parameters can be the expected torque control instruction parameters of the motors of each joint of the robot, or the expected position control instruction parameters of the motors of each joint of the robot, but are not limited thereto. The second neural network model can be obtained by training based on the deep neural network model through historical or simulation data.
[0076] In an optional embodiment, a second neural network model may be used to generate control instruction parameters based on job load parameters, job loading duration, and road conditions. Figure 5 is a schematic diagram of generating control instruction parameters based on a road condition and a workload according to an embodiment of the present invention. Figure 5 As shown, the robot motion control large model is used as the second neural network model, the robot's job load parameters and job loading duration are input into the robot motion control large model, and the road condition status information including slope, step height, and road surface unevenness is also input into the robot motion control large model. The robot motion control large model predicts the control instruction parameters to obtain the joint motor position control instructions and the joint motor torque control instructions, that is, the control instruction parameters.
[0077] Optionally, the control instruction parameters are superimposed on the robot joint motor control system instructions by feedforward control, and the robot's joints are controlled to drive the robot to move along the running path and perform the working task, including: the control instruction parameters are superimposed on the robot joint motor control system instructions by feedforward control, and the robot's joints are controlled by the joint motor control system to drive the robot to move along the running path and perform the working task, wherein the joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.
[0078] The joint motor position control loop in the above steps is a loop for controlling the target position of the robot joint motor. The joint motor speed control loop is a loop for controlling the speed of the robot shutdown motor. The joint motor current control loop is a loop for controlling the torque current of the joint motor during the robot movement.
[0079] In an optional embodiment, the joint motor position control loop, the joint motor speed control loop, and the joint motor current control loop all adopt a feedforward control mechanism, and the target position, speed, and current information in the joint motor control system are combined with the control instruction parameters in the form of position feedback, speed feedback, and current feedback, and the existing target position, speed, and current output are adjusted, so that the robot moves according to the adjusted target position, speed, and current output, thereby realizing control of the robot's movement.
[0080] Optionally, the control instruction parameters include: joint motor position control instruction parameters, joint motor torque control instruction parameters, and the control instruction parameters are superimposed on the instructions of the robot joint motor control system by feedforward control. The joint motor control system controls the robot's joints to drive the robot to move along the running path and perform work tasks, including: adding the joint motor position control instruction parameters to the joint motor position control loop to obtain the joint motor position control instruction, and adding the joint motor torque control instruction parameters to the joint motor current control loop to obtain the joint motor torque current control instruction, and controlling the robot's joints to drive the robot to move along the running path and perform work tasks through the joint motor control system.
[0081] The joint motor position control instruction in the above steps is obtained by adding the joint motor position control instruction parameters to the joint motor position control loop, and is used to control the target position of the robot joint motion.
[0082] The joint motor torque control instruction in the above steps is obtained by adding the joint motor torque control instruction parameters to the joint motor current control loop, and is used to control the output torque instruction of the robot joint motion.
[0083] In an optional embodiment, the joint motor position control instruction parameter is superimposed with the reference position instruction in the joint motor control system, and added to the joint motor position control loop in a feedforward control manner to obtain the joint motor position control instruction. The joint motor torque control instruction parameter is converted into a motor torque current control instruction parameter, and superimposed with the speed control output in the joint motor control system, and added to the joint motor current control loop in a feedforward control manner to obtain the joint motor torque current control instruction. The joint motor control system controls the robot's joint drive robot to move forward along the running path and perform the task according to the joint motor position control instruction and the joint motor torque current control instruction.
[0084] Optionally, a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path and the road condition status of the running path, including: performing path planning on the robot based on the current position and the working position to obtain the running path; identifying the road environment image of the running path in the scene road condition image to obtain the road condition status.
[0085] In an optional embodiment, the sensor data is used to map and locate the robot's surrounding environment to obtain accurate location information of the robot's current location and working location, as well as map information between the current location and the working location. Simultaneous localization and mapping SLAM technology can be used to plan a path based on the map information between the robot's current location and the working location. When executing the path planning algorithm, factors such as avoiding obstacles and optimizing the path length are considered to plan the robot's running path.
[0086] When identifying the road surface image of the running path in the road condition image, computer vision technology can be used to segment and extract features of the image to identify various road conditions, such as flat road surface, potholes, obstacles, etc. Deep learning technology can also be used to train the model to achieve automatic recognition and classification of road conditions to obtain the road condition state, but the method of identifying the road surface image of the running path in the scene road condition image to obtain the road condition state is not limited to this.
[0087] Optionally, the method further includes: training model parameters of the first initial neural network model based on a sample database of target object types in job scenarios to obtain a first neural network model.
[0088] The work scene target object type sample database in the above steps is a sample database containing target objects, wherein the target objects are located in the work scene image.
[0089] The first initial neural network model in the above steps is an untrained neural network model and can be used to train to obtain a first neural network model.
[0090] The model parameters in the above steps are the weights and biases that need to be learned and adjusted during the training process. The model parameters determine how the neural network model maps input data to output data.
[0091] In an optional embodiment, a sample database of target object types of work scenes including work scene images and target objects is obtained, and the first initial neural network model is trained to recognize target objects from work scene images. Data is extracted from the sample database of target object types of work scenes as input data, and preprocessing operations such as cleaning, standardization, and normalization are performed on the data to ensure the quality and availability of the data. The input data is input into the pre-selected first initial neural network model for training, and the model parameters are continuously updated through the back propagation algorithm until the loss function of the first initial neural network model converges, thereby obtaining the first neural network model.
[0092] Optionally, the method further includes: training model parameters of a second initial neural network model based on robot motion control historical data and simulation data to obtain a second neural network model.
[0093] The robot motion control history data in the above steps are data generated by the robot in the process of performing previous operations, and may be control instruction parameters of the robot joint motor corresponding to various operation load parameters, operation loading duration, and road conditions.
[0094] The simulation data in the above steps are control instruction parameters of the robot joint motors corresponding to various job load parameters, job loading durations, and road conditions obtained through simulation.
[0095] The second initial neural network model in the above steps is an untrained neural network model and can be used to train a second neural network model.
[0096] In an optional embodiment, the robot motion control historical data can be obtained from a database, or simulation data can be obtained based on the results of a simulation experiment. The robot motion control historical data and simulation data are used as input data to train the second initial neural network model to predict the control instruction parameters of the robot joint motor according to the job load parameters, job loading time and road conditions.
[0097] The data is cleaned, standardized, normalized and other preprocessing operations are performed to ensure the quality and availability of the data. The input data is input into the pre-selected second initial neural network model for training, and the model parameters are continuously updated through the back propagation algorithm until the loss function of the second initial neural network model converges to obtain the second neural network model.
[0098] Example 2
[0099] According to an embodiment of the present invention, an embodiment of a robot joint control device is provided, which can execute the robot control method provided in the above-mentioned embodiment 1. The specific implementation method and preferred application scenario are the same as those of the above-mentioned embodiment 1, and will not be repeated here.
[0100] Figure 6 is a schematic diagram of a robot joint control device according to an embodiment of the present invention. Figure 6 As shown:
[0101] The acquisition module 60 is used to acquire the scene road condition image of the scene currently located by the robot, the current position of the robot, the working position of the robot and the working scene image of the scene currently located by the robot. Among them, the working position is the position of the robot to perform the working task.
[0102] The planning module 62 is used to perform path planning analysis on the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path and the road condition status of the running path.
[0103] The generation module 64 is used to generate control instruction parameters for the joint motors of the robot to move from the current position to the working position and during the operation based on the path road condition and the working scene image.
[0104] The execution module 66 is used to superimpose the control instruction parameters generated in advance onto the robot joint motor control system instructions based on a feedforward control method to control the robot's joint motors, so as to drive the robot to move forward along the running path and perform the working task.
[0105] Optionally, the planning module includes: a planning unit, which is used to plan the path of the robot based on the current position and the working position to obtain the running path; and a second recognition unit, which is used to recognize the road surface image of the running path in the scene road condition image to obtain the road condition status.
[0106] Optionally, the generation module includes: a first recognition unit, used to use a first neural network model to identify the target object in the scene road condition image, and obtain the object type and object position of the target object, wherein the target object is the object of the robot to perform a working task; a first determination unit, used to determine the robot's operating load parameters based on the object type; a second determination unit, used to determine the robot's operating loading time based on the object position; a generation unit, used to generate control instruction parameters based on the operating load parameters, operating loading time and road condition status.
[0107] Optionally, the generation unit is also used to obtain control instruction parameters by calculating using a second neural network model based on road conditions, expected workload parameters and loading duration.
[0108] Optionally, the generation unit is also used to obtain the sample scene road condition image and the marked object type and the marked object position of the marked object in the sample scene road condition image, wherein the marked object is the object of the work task to be executed that is pre-marked in the sample scene road condition image; use the initial first neural network model to identify the marked objects in the sample scene road condition to obtain the predicted object type and predicted object position of the marked object.
[0109] Optionally, the execution module includes: a control unit, for responding to receiving control instructions according to the robot joints, controlling the robot's joint control system based on position control instructions and torque control instruction parameters to drive the robot to perform work tasks along a running path, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.
[0110] Optionally, the control unit also subtracts the current reference position control instruction from the position feedback through a subtractor, and uses it as a position control input to realize the motor position control. The position control output is subtracted from the speed feedback through a subtractor, and used as a speed control input to realize the motor speed control. The speed control output is subtracted from the current feedback through a subtractor, and used as a current control input to realize the motor current control, and then the pulse width modulation generation and power drive are realized to control the robot joint motor to drive the robot to move along the running path and perform the task.
[0111] Example 3
[0112] According to an embodiment of the present invention, there is also provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method for controlling a robot joint in Embodiment 1 is executed when the program is running.
[0113] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0114] Optionally, in this embodiment, the above-mentioned processor can be configured to perform the following steps through a computer program: obtain a scene road condition image of the scene in which the robot is currently located, the current position of the robot, the working position of the robot and the working scene image of the scene in which the robot is currently located; perform path planning analysis on the robot based on the current position, the working position and the scene road condition image to obtain the robot's running path and the road condition status of the running path; generate control instruction parameters of the robot's joint motors based on the road condition status and the working scene image; and use a feedforward control method to superimpose the control instruction parameters on the instructions of the robot's joint motor control system to control the robot's joints to drive the robot to move along the running path and perform working tasks.
[0115] Optionally, the processor can also be configured to perform the following steps through a computer program: using a first neural network model to identify a target object in a scene road condition image, and obtaining an object type and an object position of the target object, wherein the target object is an object of a working task to be performed by the robot; determining a working load parameter of the robot based on the object type; determining a working loading time of the robot based on the object position; and generating control instruction parameters based on the working load parameter, the working loading time and the road condition status.
[0116] Optionally, the processor can also be configured to perform the following steps through a computer program: plan the path of the robot based on the current position and the working position to obtain the running path; identify the road surface image of the running path in the scene road condition image to obtain the road condition status.
[0117] Optionally, the processor may also be configured to execute the following steps through a computer program: based on the job load parameters, the job loading duration, and the road condition, a control instruction parameter is obtained using a second neural network model.
[0118] Optionally, the above-mentioned processor can also be configured to perform the following steps through a computer program: obtaining the sample scene road condition image and the annotated object type and annotated object position of the annotated object in the sample scene road condition image, wherein the annotated object is an object of the work task to be performed that is pre-annotated in the sample scene road condition image; using the initial first neural network model to identify the annotated object in the sample scene road condition, and obtain the predicted object type and predicted object position of the annotated object.
[0119] Optionally, the above-mentioned processor can also be configured to perform the following steps through a computer program: in response to receiving a control instruction based on the robot joint, the control instruction parameters are superimposed on the instructions of the robot joint motor control system in a feedforward control manner, and the robot joints are controlled to drive the robot along the running path and perform the working task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.
[0120] Optionally, the processor can also be configured to perform the following steps through a computer program: subtract the current reference position control instruction from the position feedback through a subtractor, and use it as a position control input to implement motor position control, and subtract the position control output from the speed feedback through a subtractor, and use it as a speed control input to implement motor speed control. Subtract the output of the speed control from the current feedback through a subtractor, and use it as a current control input to implement motor current control, and then realize the generation of pulse width modulation and power drive to control the robot joint motor to drive the robot to move along the running path and perform the task.
[0121] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an operating scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; control instruction parameters of the robot's joint motors are generated based on the road condition status and the operating scene image; and the control instruction parameters are superimposed on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The robot's proper action characteristics in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby achieving the purpose of improving the robot's joint response speed in unknown scenes, and achieving the technical effect of improving the robot's dynamic response speed of operation and enhancing the stability and flexibility of the robot's movement.
[0122] Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present disclosure. Figure 7 As shown, the electronic device 700 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0123] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processor 710, at least one memory 720, a bus 730 connecting different system components (including the memory 720 and the processor 710), and a display 740.
[0124] The memory 720 stores program codes, which can be executed by the processor 710, so that the processor 710 executes the steps described in the method section of the embodiment of the present application according to various exemplary embodiments of the present disclosure.
[0125] The memory 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202, and may further include a read-only memory unit (ROM) 7203, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
[0126] In some examples, the memory 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include the implementation of a network environment. The memory 720 may further include a memory remotely disposed relative to the processor 710, and these remote memories may be connected to the electronic device 700 via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0127] Bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a local bus of processor 710, or any of a variety of bus architectures.
[0128] The display 740 may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 700 .
[0129] Optionally, the electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 760. Figure 7 As shown, the network adapter 760 communicates with other modules of the electronic device 700 via the bus 730. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0130] The electronic device 700 may further include: a keyboard, a cursor control device (such as a mouse), an input / output interface (I / O interface), a network interface, a power supply and / or a camera.
[0131] It can be understood by those skilled in the art that Figure 7The structure shown is for illustration only and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 The memory 720 may be used to store computer programs and corresponding data, such as the computer programs and corresponding data corresponding to the robot control method in the embodiment of the present disclosure. The processor 710 executes various functional applications and data processing by running the computer programs stored in the memory 720, that is, implementing the robot control method described above.
[0132] Example 4
[0133] An embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium may be used to store program codes executed by the robot control method provided in the above embodiment.
[0134] Optionally, in this embodiment, the above-mentioned computer-readable storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
[0135] Optionally, in this embodiment, the above storage medium may be located in any electronic device in a group of electronic devices in a computer network, or in any mobile terminal in a group of mobile terminals.
[0136] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining a scene road condition image of the scene in which the robot is currently located, the current position of the robot, the working position of the robot, and an operating scene image of the scene in which the robot is currently located; performing path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; generating control instruction parameters for the robot's joint motors based on the road condition status and the operating scene image; and superimposing the control instruction parameters on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks.
[0137] Optionally, the computer-readable storage medium is also configured to store program codes for executing the following steps: using a first neural network model to identify a target object in a scene road condition image to obtain an object type and an object position of the target object, wherein the target object is an object of a working task to be performed by the robot; determining a working load parameter of the robot based on the object type; determining a working loading time of the robot based on the object position; and generating control instruction parameters based on the working load parameter, the working loading time and the road condition status.
[0138] Optionally, the computer-readable storage medium is also configured to store program codes for executing the following steps: planning the path of the robot based on the current position and the working position to obtain the running path; identifying the road surface image of the running path in the scene road condition image to obtain the road condition status.
[0139] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: based on the job load parameters, the job loading duration, and the road condition, using the second neural network model to obtain control instruction parameters.
[0140] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: obtaining a sample scene road condition image and a labeled object type and a labeled object position of an labeled object in the sample scene road condition image, wherein the labeled object is an object of a work task to be performed that is pre-annotated in the sample scene road condition image; using an initial first neural network model to identify the labeled objects in the sample scene road condition to obtain a predicted object type and a predicted object position of the labeled objects.
[0141] Optionally, the computer-readable storage medium is also configured to store program code for executing the following steps: in response to receiving a control instruction based on the robot joint, superimposing the control instruction parameters on the instructions of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot along a running path and perform a work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.
[0142] Optionally, the computer-readable storage medium is further configured to store program codes for executing the following steps: subtracting the current reference position control instruction from the position feedback through a subtractor, using the subtractor as the position control input to implement the motor position control, subtracting the position control output from the speed feedback through a subtractor, using the speed control input to implement the motor speed control. Subtracting the speed control output from the current feedback through a subtractor, using the speed control input to implement the motor current control, and then implementing the generation of pulse width modulation and power drive to control the robot joint motor to drive the robot to move along the running path and perform the task.
[0143] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an operating scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; control instruction parameters of the robot's joint motors are generated based on the road condition status and the operating scene image; and the control instruction parameters are superimposed on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The robot's proper action characteristics in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby achieving the purpose of improving the robot's joint response speed in unknown scenes, and achieving the technical effect of improving the robot's dynamic response speed of operation and enhancing the stability and flexibility of the robot's movement.
[0144] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0145] Example 5
[0146] An embodiment of the present application further provides a computer program product, including a computer program, which implements the robot control method in each embodiment of the present invention when executed by a processor.
[0147] Optionally, the computer program can be configured to perform the following steps: obtaining a scene road condition image of the scene in which the robot is currently located, the current position of the robot, the working position of the robot, and an operating scene image of the scene in which the robot is currently located; performing path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; generating control instruction parameters for the robot's joint motors based on the road condition status and the operating scene image; and superimposing the control instruction parameters on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks.
[0148] Optionally, the computer program can be configured to perform the following steps: using a first neural network model to identify a target object in a scene road condition image to obtain an object type and an object position of the target object, wherein the target object is an object of a working task to be performed by the robot; determining a robot's operating load parameter based on the object type; determining a robot's operating loading time based on the object position; and generating control instruction parameters based on the operating load parameter, the operating loading time and the road condition status.
[0149] Optionally, the computer program can be configured to perform the following steps: plan a path for the robot based on the current position and the working position to obtain an operating path; and identify a road surface image of the operating path in a scene road condition image to obtain a road condition status.
[0150] Optionally, the computer program may be configured to perform the following steps: based on the job load parameters, the job loading duration, and the road condition, obtaining control instruction parameters using a second neural network model.
[0151] Optionally, the computer program can be configured to perform the following steps: obtaining a sample scene road condition image and annotated object types and annotated object positions of annotated objects in the sample scene road condition image, wherein the annotated objects are objects of work tasks to be performed that are pre-annotated in the sample scene road condition image; and using an initial first neural network model to identify the annotated objects in the sample scene road condition to obtain predicted object types and predicted object positions of the annotated objects.
[0152] Optionally, the computer program can be configured to perform the following steps: in response to receiving control instructions based on the robot joints, superimposing the control instruction parameters onto the instructions of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot along a running path and perform a work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.
[0153] Optionally, the computer program can be configured to perform the following steps: subtract the current reference position control instruction from the position feedback through a subtractor, use it as the position control input to implement the motor position control, subtract the position control output from the speed feedback through a subtractor, use it as the speed control input to implement the motor speed control. Subtract the speed control output from the current feedback through a subtractor, use it as the current control input to implement the motor current control, and then implement the generation of pulse width modulation and power drive to control the robot joint motor to drive the robot to move along the running path and perform the task.
[0154] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an operating scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; control instruction parameters of the robot's joint motors are generated based on the road condition status and the operating scene image; and the control instruction parameters are superimposed on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The robot's proper action characteristics in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby achieving the purpose of improving the robot's joint response speed in unknown scenes, and achieving the technical effect of improving the robot's dynamic response speed of operation and enhancing the stability and flexibility of the robot's movement.
[0155] Example 6
[0156] An embodiment of the present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the robot control method in each embodiment of the present invention is implemented.
[0157] It should be noted that the program code contained in the non-volatile computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, etc., or any appropriate combination of the above.
[0158] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: obtaining a scene road condition image of the scene in which the robot is currently located, the current position of the robot, the working position of the robot, and an operating scene image of the scene in which the robot is currently located; performing path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; generating control instruction parameters for the robot's joint motors based on the road condition status and the operating scene image; and superimposing the control instruction parameters on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks.
[0159] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: using a first neural network model to identify a target object in a scene road condition image to obtain an object type and an object position of the target object, wherein the target object is an object of a working task to be performed by the robot; determining a job load parameter of the robot based on the object type; determining a job loading time of the robot based on the object position; and generating control instruction parameters based on the job load parameter, the job loading time and the road condition status.
[0160] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: planning a path for the robot based on a current position and a working position to obtain an operating path; and identifying a road surface image of the operating path in a scene road condition image to obtain a road condition status.
[0161] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: based on the job load parameters, the job loading duration, and the road condition, using a second neural network model to obtain control instruction parameters.
[0162] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: obtaining a sample scene road condition image and a labeled object type and a labeled object position of an labeled object in the sample scene road condition image, wherein the labeled object is an object of a work task to be performed that is pre-labeled in the sample scene road condition image; using an initial first neural network model to identify the labeled objects in the sample scene road condition to obtain a predicted object type and a predicted object position of the labeled objects.
[0163] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: in response to receiving a control instruction based on a robot joint, superimposing the control instruction parameters onto the instructions of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot along a running path and perform a work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.
[0164] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: subtracting the current reference position control instruction from the position feedback through a subtractor, using the subtractor as the position control input to implement the motor position control, subtracting the position control output from the speed feedback through a subtractor, using the speed control input to implement the motor speed control. Subtracting the speed control output from the current feedback through a subtractor, using the current control input to implement the motor current control, and then implementing the generation of pulse width modulation and power drive to control the robot joint motor to drive the robot to move along the running path and perform the task.
[0165] In an embodiment of the present invention, a scene road condition image of a scene in which the robot is currently located, the current position of the robot, the working position of the robot and an operating scene image of the scene in which the robot is currently located are obtained; a path planning analysis is performed on the robot based on the current position, the working position and the scene road condition image to obtain the robot's operating path and the road condition status of the operating path; control instruction parameters of the robot's joint motors are generated based on the road condition status and the operating scene image; and the control instruction parameters are superimposed on the instructions of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the operating path and perform operating tasks. It is easy to notice that the robot's running path and the road condition status of the running path can be obtained according to the current position, working position and the scene road condition image of the current scene, so that the situation of the current scene can be understood in advance, and the control instruction parameters can be generated based on the road condition status and the working scene image. The robot's proper action characteristics in the current scene can be obtained according to the situation of the current scene. The robot is controlled to perform the working task along the running path based on the control instruction parameters, which can avoid the response delay caused by the robot's insufficient familiarity with the current scene, and enable the robot to perform the task under the premise of being familiar with the environmental characteristics, thereby achieving the purpose of improving the robot's joint response speed in unknown scenes, and achieving the technical effect of improving the robot's dynamic response speed of operation and enhancing the stability and flexibility of the robot's movement.
[0166] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0167] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0168] In the several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in a manner where there is a delay in the dynamic response of the robot in other unknown scenarios. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0169] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0170] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0172] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for controlling a robot joint, characterized in that: include: Acquire a scene road condition image of a scene currently located by the robot, a current position of the robot, an operation position of the robot, and an operation scene image of the scene currently located by the robot; Performing path planning analysis on the robot based on the current position, the working position and the scene road condition image to obtain an operation path of the robot and a road condition state of the operation path; Generate control instruction parameters of the joint motors of the robot based on the road condition and the working scene image, wherein the control instruction parameters are used to control the joints of the robot to drive the robot to run from the current position to the working position and perform the working task, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters; The control instruction parameters are superimposed on the instructions of the robot joint motor control system in a feedforward control manner, and the robot joints are controlled to drive the robot to move forward along the running path and perform the working task.
2. The method according to claim 1, characterized in that Generating control instruction parameters of the robot joint motor based on the road condition and the working scene image includes: Using a first neural network model to identify a target object in the working scene image, and obtaining an object type and an object position of the target object, wherein the target object is an object for which the robot is to perform the working task; determining a workload parameter of the robot based on the object type; Determining a loading duration of a job of the robot based on the position of the object; The control instruction parameters of the robot joint motor are generated based on the workload parameters, the workload loading duration and the road condition.
3. The method according to claim 2, characterized in that Generating control instruction parameters of the robot joint motor based on the job load parameter, the job loading duration and the road condition includes: A second neural network model is used to generate control instruction parameters for the robot joint motor based on the job load parameters, the job loading duration, and the road condition.
4. The method according to claim 1, characterized in that: The control instruction parameters are superimposed on the instructions of the robot joint motor control system in a feedforward control manner, and the joints of the robot are controlled to drive the robot to move forward along the running path and perform the working task, including: The control instruction parameters are superimposed on the instructions of the robot joint motor control system by adopting a feedforward control method, and the joints of the robot are controlled by the joint motor control system to drive the robot to move along the running path and perform the working task, wherein the joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.
5. The method according to claim 4, characterized in that The control instruction parameters include: joint motor position control instruction parameters, joint motor torque control instruction parameters, and the control instruction parameters are superimposed on the instructions of the robot joint motor control system in a feedforward control manner, and the joints of the robot are controlled by the joint motor control system to drive the robot to move forward along the running path and perform the operation task, including: The joint motor position control instruction parameters are added to the joint motor position control loop to obtain the joint motor position control instruction, and the joint motor torque control instruction parameters are added to the joint motor current control loop to obtain the joint motor torque current control instruction. The joints of the robot are controlled by the joint motor control system to drive the robot to move along the running path and perform the working task.
6. The method according to claim 1, characterized in that Performing path planning analysis on the robot based on the current position, the working position and the scene road condition image to obtain the running path of the robot and the road condition state of the running path includes: Performing path planning for the robot based on the current position and the working position to obtain the operation path; The road surface environment image of the running path in the scene road condition image is identified to obtain the road condition state.
7. The method according to claim 2, characterized in that The method further comprises: Based on a sample database of target object types in job scenarios, model parameters of a first initial neural network model are trained to obtain the first neural network model.
8. The method according to claim 3, characterized in that The method further comprises: Based on the robot motion control historical data and simulation data, the model parameters of the second initial neural network model are trained to obtain the second neural network model.
9. A control device for a robot joint, characterized in that: include: An acquisition module, used to acquire a scene road condition image of a scene currently located by the robot, a current position of the robot, an operation position of the robot, and an operation scene image of the scene currently located by the robot; A planning module, configured to perform path planning analysis on the robot based on the current position, the working position and the scene road condition image, and obtain an operation path of the robot and a road condition state of the operation path; A generating module, for generating control instruction parameters of the joint motors of the robot based on the road condition and the working scene image, wherein the control instruction parameters are used to control the joints of the robot to drive the robot to move from the current position to the working position and perform the working task, wherein the control instruction parameters include at least one of the following: joint motor position control instruction parameters, joint motor torque control instruction parameters; The execution module is used to superimpose the control instruction parameters onto the instructions of the robot joint motor control system in a feedforward control manner, so as to control the joints of the robot to drive the robot to move forward along the running path and perform the working task.
10. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 8 when running.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.
12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Robotic arm control method based on man-machine fusion
CN111152220A
Robot control method, device and equipment and computer readable storage medium
CN113021340A
Industrial robot teaching control method, electronic equipment and storage medium
CN114425770A
Robot joint constraint trajectory planning method and device, equipment and medium
CN115625712A
Collaborative robot control system and method
CN116175543A
