Control method and device of robot joint, electronic equipment and storage medium

Abstract

This invention discloses a control method, device, electronic device, and storage medium for robot joints. The method includes: acquiring a scene road condition image of the robot's current location, the robot's current position, the robot's working position, and a working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot joint motor control system using a feedforward control method to control the robot's joints to drive the robot forward along the running path and perform the work task. This invention improves the dynamic response speed of the robot's working joints in unknown scenarios and enhances the stability and flexibility of the robot's movement.

Description

Technical Field

[0001] This invention relates to the field of biomimetic robots, and more specifically, to a method, device, electronic device, and storage medium for controlling robot joints. Background Technology

[0002] The electric joints of bionic robots need to be controlled and adjusted in real time according to the application scenario and working conditions to achieve synchronized and coordinated flexible movements of the whole body. Due to the complexity and variability of the application scenario and working conditions, the joint system needs to have the ability to respond quickly and adjust rapidly.

[0003] The core components of a robot joint generally include a motor, motor driver, reducer, and sensors. One technical solution to improve the response speed of robot joints involves increasing the motor time constant, using faster signal acquisition and main control circuits, and optimizing motor control algorithms during the joint design process. While this method can improve the robot's dynamic response speed, the uncertainty of the robot's environment means that in unknown application scenarios, such as sudden changes in road conditions or load, the robot's action response will still exhibit a certain delay. This invention proposes a robot joint position and torque feedforward control system based on machine vision-based condition prediction, which can effectively optimize and solve the above-mentioned problems. Summary of the Invention

[0004] This invention provides a method, device, electronic device, and storage medium for controlling robot joints, which improves the dynamic response speed of robot joints in unknown scenarios and enhances the flexibility and stability of robot operation.

[0005] According to one aspect of the present invention, a robot joint control method is provided, comprising: acquiring a scene road condition image of the robot's current location, the robot's current position, the robot's working position, and a working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot joint motors based on the road condition status and the working scene image, wherein the control command parameters are used to control the robot's joints to drive the robot from the current position to the working position and perform a work task, wherein the control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters; superimposing the control command parameters onto the commands of the robot joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task.

[0006] Optionally, generating control command parameters for the robot joint motors based on road conditions and work scene images includes: using a first neural network model to identify target objects in the work scene images to obtain the object type and object position of the target objects, wherein the target objects are objects for the robot to perform work tasks; determining the robot's work load parameters based on the object type; determining the robot's work loading time based on the object position; and generating control command parameters for the robot joint motors based on the work load parameters, work loading time, and road conditions.

[0007] Optionally, control command parameters for the robot joint motors are generated based on the workload parameters, workload duration, and road conditions, including: using a second neural network model to generate control command parameters for the robot joint motors based on the workload parameters, workload duration, and road conditions.

[0008] Optionally, a feedforward control method is used to superimpose control command parameters onto the commands of the robot joint motor control system, controlling the robot's joints to drive the robot to move along the running path and perform the task. This includes: using a feedforward control method to superimpose control command parameters onto the commands of the robot joint motor control system, and controlling the robot's joints to drive the robot to move along the running path and perform the task through the joint motor control system. The joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.

[0009] Optionally, the control command parameters include: joint motor position control command parameters and joint motor torque control command parameters. The control command parameters are superimposed on the commands of the robot's joint motor control system using a feedforward control method. The joint motor control system controls the robot's joints to move along the running path and perform the work task. This includes: adding the joint motor position control command parameters to the joint motor position control loop to obtain the joint motor position control command, and adding the joint motor torque control command parameters to the joint motor current control loop to obtain the joint motor torque current control command. The joint motor control system then controls the robot's joints to move along the running path and perform the work task.

[0010] Optionally, path planning analysis is performed on the robot based on the current location, the work location, and the scene road condition image to obtain the robot's running path and the road condition status of the running path, including: performing path planning on the robot based on the current location and the work location to obtain the running path; and identifying the road surface environment image of the running path in the scene road condition image to obtain the road condition status.

[0011] Optionally, the method further includes: training the model parameters of the first initial neural network model based on a sample database of target object types in the work scenario to obtain the first neural network model.

[0012] Optionally, the method further includes: training the model parameters of the second initial neural network model based on historical data and simulation data of robot motion control to obtain the second neural network model.

[0013] According to another aspect of the present invention, a robot control device is also provided, comprising: an acquisition module, configured to acquire a scene road condition image of the robot's current location, the robot's current position, the robot's working position, and a working scene image of the robot's current location; a planning module, configured to perform path planning analysis on the robot based on the current position, the working position, and the scene road condition image to obtain the robot's running path and the road condition status of the running path; a generation module, configured to generate control command parameters for the robot's joint motors based on the road condition status and the working scene image, wherein the control command parameters are used to control the robot's joints to drive the robot from the current position to the working position and perform a work task, wherein the control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters; and an execution module, configured to superimpose the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform a work task.

[0014] In this embodiment of the invention, the following steps are taken: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path are obtained based on the current location, the work location, and the road condition image of the current scene. This allows for advance understanding of the current scene. Control command parameters are generated based on the road condition and the work scene image. The robot's expected action characteristics in the current scene can be determined based on the current scene conditions. By controlling the robot to perform the work task along the running path based on the control command parameters, response delays caused by insufficient familiarity with the current scene can be avoided. This allows the robot to perform tasks under the premise of familiarity with the environmental characteristics, thereby achieving the goal of improving the robot's joint response speed in unknown scenes. This achieves the technical effect of improving the robot's dynamic response speed and enhancing the stability and flexibility of robot movement. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0016] Figure 1 This is a flowchart of a robot joint control method according to an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram of an optional robot joint motor control system according to an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of an optional robot joint workload estimation unit according to an embodiment of the present invention;

[0019] Figure 4 This is a schematic diagram of an optional robot joint position and torque control command parameter calculation unit generating control command parameters according to an embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of an optional method for generating robot joint control command parameters based on path conditions and robot workload according to an embodiment of the present invention.

[0021] Figure 6 This is a schematic diagram of a robot joint control device according to an embodiment of the present invention;

[0022] Figure 7 This is a schematic diagram of an optional electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Example 1

[0026] According to an embodiment of the present invention, an embodiment of a robot joint control method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] Figure 1 This is a flowchart of a robot joint control method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0028] Step S102: Obtain the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location.

[0029] The scene road condition images mentioned above refer to the environmental images from the robot's current position to the work position, including elements such as roads, traffic signs, vehicles, and pedestrians. These images are used for robot navigation or to assist in decision-making. These images can be acquired through cameras or other sensors and then analyzed and processed using computer vision technology to help the robot identify and understand the current road conditions. However, the methods of acquiring and processing images are not limited to these. Acquiring scene road condition images allows the robot to understand its environment in advance, which helps improve the speed at which the robot performs its tasks.

[0030] The robot's current position in the above steps refers to the precise coordinates or specific location of the robot. The robot's current position can be determined by sensors or the Global Positioning System (GPS), but the method of determining the robot's current position is not limited to these. The robot's current position can be used to guide the robot to complete specific tasks or navigate to specific locations.

[0031] The work location mentioned in the above steps refers to the position where the robot will perform the task. This location can be a specific workstation, a position on the production line, or a specific area, but it is not limited to these. When performing the task, the robot needs to be able to accurately identify and reach this location in order to execute the pre-set task.

[0032] The work scene images in the above steps are environmental images of the robot's work location and images of the work objects, which can represent the environment in which the robot performs the work, the volume, weight, and spatial location of the work objects.

[0033] In one alternative embodiment, a vision and perception system installed inside the robot can acquire images of the road conditions in the robot's current environment, the robot's current position, the robot's work position, and images of the work environment in which the robot is currently located. When acquiring the robot's current position, a robot vision system and Simultaneous Localization and Mapping (SLAM) technology can be used to determine the robot's current location.

[0034] When acquiring images of the road conditions in the robot's current environment, the robot's vision system can capture these images in real time using cameras. The robot's vision and perception system then uses image processing technology to analyze and identify the road conditions, thus obtaining an image of the road conditions in the robot's current environment. Alternatively, the robot's vision system can use a LiDAR sensor to scan the surrounding environment and generate point cloud data. The robot's vision system then acquires this point cloud data and constructs a 3D model of the environment, thereby obtaining an image of the road conditions in the robot's current environment. However, the methods for acquiring images of the road conditions in the robot's current environment using the robot's vision and perception system are not limited to these.

[0035] When using a robot vision and perception system to acquire images of the work location and work scene, cameras or sensors in the robot vision and perception system can be used to scan the work area, acquire images of the work scene, and identify features in the work area, such as markings, colors, and shapes, in order to determine the robot's work location so that the robot can perform its work tasks at the accurate work location. However, the methods of using a robot vision and perception system to acquire images of the work scene and work location are not limited to this.

[0036] To determine the robot's current position, historical scan data generated by the LiDAR sensors in the robot's vision and perception system can be acquired. Feature extraction is then performed, extracting landmarks from the LiDAR data. These landmarks are converted into map data, and an environmental map is constructed based on this map. Using the constructed environmental map and the real-time LiDAR sensor scan data, the SLAM algorithm is employed to localize the robot, determining its position on the map.

[0037] In another alternative embodiment, when acquiring a scene road condition image of the robot's current location, ultrasonic sensors can be used to help the robot detect the shape of the road and the distance and position of surrounding obstacles, thereby generating a road condition image and scene information. Then, the data acquired by the ultrasonic sensors is collected, and through image processing algorithms and computer vision technology, the data is converted into a scene road condition image. Deep learning and neural network technologies can be used to improve the accuracy and efficiency of image processing, thus obtaining a scene road condition image of the robot's current location. However, the method of acquiring a scene road condition image of the robot's current location is not limited to this.

[0038] When obtaining the robot's current position, accelerometers or velocity sensors can be used to track the robot's movement and infer its position. Alternatively, external positioning systems, such as base station positioning or wireless positioning, can be used to obtain the robot's current position information. However, the methods for obtaining the robot's current position are not limited to these.

[0039] When acquiring the work location and work scene image, the work location can be obtained from the work scheduling system and then transmitted to the robot's control system via the network. The robot can then receive this information to determine the location of the task to be performed, but the method of acquiring the work location is not limited to this. Based on the work location, ultrasonic sensors are then used to detect environmental features of the work area, collecting the data acquired by the ultrasonic sensors. Image processing algorithms and computer vision technology are then used to convert the data into a work scene image.

[0040] Step S104: Perform path planning analysis on the robot based on the current location, work location, and scene road condition images to obtain the robot's running path and the road condition status of the running path.

[0041] The running path in the above steps is the walking path of the robot from the current position to the work position. When planning the running path, factors such as distance and obstacles need to be considered. The robot needs to walk according to the planned running path to move from the current position to the work position.

[0042] The road condition status of the running path mentioned above refers to the road conditions corresponding to the running path. Road conditions may include, but are not limited to, road surface slope, step height, and unevenness. The robot's actions can be adjusted according to the road condition status of the robot's running path to ensure safe and successful task completion.

[0043] In one optional embodiment, the path planning unit in the robot joint position and torque control command parameter calculation unit can perform path planning for the robot based on the current position, the work position, and the scene road condition image to obtain the robot's running path. Based on the current position and the work position, areas with a high probability of robot path segmentation are defined. Then, image processing and recognition are performed on the scene road condition image corresponding to the area to identify information such as roads, obstacles, and traffic signs, constructing an environmental map of the area. A path planning algorithm is then used to process the environmental map. The path planning algorithm can be Dijkstra's algorithm, a minimum spanning tree algorithm, or a bidirectional search algorithm, but is not limited to these. By processing the environmental map through the path planning algorithm, the robot's running path can be obtained, enabling the robot to safely and efficiently reach the designated work position.

[0044] The expected path condition analysis unit in the robot joint position and torque control command parameter calculation unit can analyze the robot's running path based on scene road condition images to obtain the road condition status. Road condition status includes, but is not limited to, road surface slope, step height, and road surface unevenness. Specifically, after determining the robot's running path, computer vision technology can be used to identify road surface slope information in the image, or image processing algorithms can be used to detect horizontal and vertical lines in the image, and then the road surface slope can be calculated based on the position and angle of these lines. However, the method of obtaining the road surface slope in the road condition status is not limited to these methods. Furthermore, image processing algorithms can be used to detect step edges in the image, and then the step height can be calculated based on the position and length of the edges, thus obtaining the step height in the road condition status. Sensors on the robot can be used to detect road surface unevenness; for example, a LiDAR sensor can be used to scan the road surface. Then, the expected path condition analysis unit in the robot joint expected torque and current control command calculation unit acquires the scan data and analyzes it to obtain the road surface unevenness in the road condition status. The above method can be used to obtain the road conditions of the robot's running path. By obtaining the road conditions of the robot's running path, the changes in the robot's motion control conditions can be estimated in advance, thereby controlling the robot to make actions that conform to the road conditions and improving the robot's response speed.

[0045] Step S106: Generate control command parameters for the robot joint motors based on the road condition and work scene image. The control command parameters are used to control the robot's joints to drive the robot from the current position to the work position and perform the work task. The control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters.

[0046] The robot joint motors in the above steps are used to control the robot's operation and task execution.

[0047] The control command parameters in the above steps are used to control the movement, behavior, and function of the robot's joints. They control the robot joints to drive it along the operating path and perform tasks, and enable the robot to act appropriately based on the scene's road condition image and the current road conditions. The control command parameters can be adjusted according to specific tasks and environments to achieve autonomous behavior and interactive capabilities for the robot.

[0048] The joint motor position control command parameters in the above steps are used to describe the target motion position of the joint motors for each robot action.

[0049] The joint motor torque control command parameters in the above steps are used to control and adjust the torque output of each joint of the robot in order to achieve precise motion control and force adjustment.

[0050] In one optional embodiment, existing road condition data and operational scene image data are collected as historical road condition data and historical operational scene image data. These historical road condition data and historical operational scene image data are preprocessed, including image denoising and feature extraction, to be used as input for a machine learning model. The preprocessed historical road condition data and historical operational scene image data are then input into the machine learning model to train it. The trained machine learning model is used to establish a mapping model between road condition states and control command parameters. Based on the current road condition state, the established mapping model is used to generate the robot's control command parameters in real time.

[0051] In another alternative embodiment, a pre-trained neural network model is used to process the work scene image to identify the object on which the robot needs to perform the task. Features of the object are extracted, and the time and load required for the robot to perform the task are estimated based on these features. Control command parameters are then generated based on the required time and load, as well as the work scene image.

[0052] The control command parameters include joint motor position control command parameters and joint motor torque control command parameters. The joint motor position control command parameters are used to determine the target position of the joint motors for each action during the robot's movement. The joint motor torque control command parameters are used to control the joint motor torque output of the robot when performing each action. The joint motor position control command parameters and joint motor torque control command parameters are used together to control the robot's joint movements, thereby driving the robot to move from the current position to the working position and perform the work task.

[0053] In step S108, the control command parameters are superimposed onto the commands of the robot joint motor control system using a feedforward control method, thereby controlling the robot's joints to drive the robot to move along the running path and perform the task.

[0054] The feedforward control described above is a control system design method that generates control command parameters for the joint motors in advance based on predicted changes in the robot's joint operating conditions. These parameters are then superimposed on the existing position, velocity, and current commands in the motor control system. Simultaneously, the existing joint motor position, velocity, and current are fed back to the system, thus realizing the control function of the robot's joint motor control system. Feedforward control allows the robot's joint motor control system to respond quickly to disturbances or changes, achieving better control performance.

[0055] The robot joint motor control system mentioned above is a system that controls the robot's joints to drive the robot forward along the running path and perform tasks.

[0056] The tasks mentioned above are the tasks that the robot needs to perform. These can be security tasks such as patrolling, monitoring, and emergency rescue; medical care tasks such as surgical assistance, rehabilitation training, and elderly care; or cleaning and maintenance tasks such as cleaning floors, glass curtain walls, aquariums, and ponds. However, the tasks that the robot can perform are not limited to these.

[0057] In one optional embodiment, the robot joint motor control system consists of position control, speed control, and current control loops. The joint motor position control command parameters and joint motor torque control command parameters can be superimposed on the robot joint motor control system commands through feedforward control.

[0058] Figure 2 This is a schematic diagram of an optional robot joint motor control system according to an embodiment of the present invention, such as... Figure 2 As shown, the robot joint motor control system 200 includes a motor, encoder, torque control sensor, and reduction gear. The encoder measures speed and position, and decodes the data to obtain joint motor position and speed feedback. The current reference position control command is subtracted from the position feedback by a subtractor, serving as the position control input for motor position control. The position control output is subtracted from the speed feedback by a subtractor, serving as the speed control input for motor speed control. The motor current in the abc coordinate system is converted to the current in the dq coordinate system, resulting in d-axis and q-axis current feedback. The abc coordinate system is a stationary coordinate system based on a three-phase motor, used to describe the current and voltage vectors of the motor in a three-phase circuit. The dq coordinate system is a Cartesian coordinate system, with the d and q axes being two axes typically used to describe the current and voltage rotation vectors in a motor control system. The speed control output is subtracted from the current feedback by a subtractor, serving as the current control input for motor current control. Pulse width modulation (PWM) generation and power drive are then implemented. An adder can superimpose the current position control command with the joint motor position control command parameters to achieve position feedforward control. After the torque control command parameters are theoretically converted into torque current control command parameters, they are superimposed with the speed control output through an adder to achieve torque feedforward control.

[0059] By using feedforward control to superimpose control command parameters onto the robot joint motor control system commands, the position and current control loops can be adjusted, improving the robot's torque control dynamic response. By applying specific current and voltage to the motor, the required speed and torque can be achieved, enabling the robot to move along the running path and perform tasks.

[0060] In this embodiment of the invention, the following methods are employed: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating joint motor control command parameters for the robot at the working position based on the road condition status and scene road condition image, wherein the control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path can be obtained from the current location, the work location, and the road condition image of the current scene. This allows for a prior understanding of the current scene. Based on the road condition and the work scene image, control command parameters can be generated. The robot's joints can be determined according to the current scene conditions. Based on the control command parameters, the robot can be controlled to perform the work task along the running path. This avoids response delays caused by insufficient familiarity with the current scene, allowing the robot to perform tasks under the premise of familiarity with the environmental characteristics. This achieves the goal of improving the response speed of the robot joints in unknown scenes, and achieves the technical effects of improving the dynamic response speed of the robot's operation and enhancing the stability and flexibility of the robot's movement.

[0061] Optionally, generating control command parameters for the robot joint motors based on road conditions and work scene images includes: using a first neural network model to identify target objects in the work scene images to obtain the object type and object position of the target objects, wherein the target objects are objects for the robot to perform work tasks; determining the robot's work load parameters based on the object type; determining the robot's work loading time based on the object position; and generating control command parameters for the robot joint motors based on the work load parameters, work loading time, and road conditions.

[0062] In the steps described above, the target object refers to the object that the robot needs to manipulate when performing its tasks. For example, when the robot performs a security patrol task, the target object is an area; when the robot performs a task of moving items, the target object is one or more items. However, the target objects described above are merely examples, and the actual target objects are not limited to these.

[0063] The object type mentioned above refers to the type of object that the robot needs to manipulate when performing a task. It characterizes the type features of the target object and can be used to determine the robot's workload parameters. For example, when the robot performs a task of moving an item, the target object is an item, specifically a rectangular item with a length of 0.5 meters, a width of 0.3 meters, and a height of 0.3 meters. The workload for the robot to complete this task can be determined based on the characteristics of the object type. It should be noted that the above example is for illustrative purposes only, and the actual target object and object type are not limited to this.

[0064] The object position in the above steps refers to the position of the object that the robot needs to operate on when performing the task. It can characterize the positional features of the target object and can be used to determine the robot's task loading time.

[0065] The workload parameters in the above steps may include, but are not limited to, weight and volume, and can be used to determine the load that the robot needs to withstand to complete the preset task.

[0066] The task loading time in the above steps is the time required for the robot to move from its current position to the target object position along the running path. The time required for the robot to move from its current position to the target object position can be determined by the robot's current position, the target object position, and the robot's running path. Since the robot is in a task loading state while moving along the running path, this time is the task loading time.

[0067] The first neural network model in the above steps is a model that can identify objects based on video images. It is used to identify the object type and object location of the target object, and can obtain the robot's workload parameters and task loading time based on the object type and object location.

[0068] In one alternative instance, a convolutional neural network model for object recognition can be trained using video image data of multiple object samples. The video image data of multiple object samples may include video frames of multiple categories of objects, as well as label information associated with each video frame. The trained convolutional neural network model is used as the first neural network model.

[0069] The robot's workload estimation unit can use a first neural network model to generate control command parameters. Figure 3 This is a schematic diagram of an optional robot joint workload estimation unit according to an embodiment of the present invention, such as... Figure 3 As shown, the first neural network model in the robot's workload estimation unit, namely... Figure 3The large workload model in the system acquires image data of the work scene from the robot's vision and perception system, processes the image data of the work scene, identifies the target object that the robot needs to operate on, and parses the target object to obtain the object matching result and the object position, that is, the object type and object position of the target object.

[0070] The object matching results, i.e. the object type of the target object, are analyzed. Target objects of the same type are searched from historical data to calculate the expected workload volume and weight, thereby determining the robot's workload parameters.

[0071] The position of the object, i.e. the position of the target object, is analyzed to calculate the expected loading time of the task load, thereby determining the loading time of the robot's task load.

[0072] Based on the above process, the robot's expected workload estimation unit can determine the robot's workload and loading time, i.e., the robot's workload parameters and loading duration. A regression analysis model is used to establish a mapping model between workload parameters, loading duration, road conditions, and robot control command parameters. Using this established mapping model, based on the current workload parameters, loading duration, and road conditions, the robot's control command parameters, such as joint motor position control parameters and joint motor torque control parameters, are generated in real time.

[0073] In another alternative embodiment, Figure 4 This is a schematic diagram illustrating the generation of control command parameters by an optional robot joint position and torque control command parameter calculation unit according to an embodiment of the present invention, as shown below. Figure 4 As shown, the application scenario, namely the scene road condition image mentioned above, the robot's current position, the robot's working position, and the working scene image, are input into the robot joint position and torque control command parameter calculation unit 400. The robot vision and perception system in the robot joint position and torque control command parameter calculation unit 400 performs path planning based on the application scenario to obtain the expected path conditions, namely the road condition status mentioned above. Simultaneously, the robot vision and perception system also uses a first neural network model to identify the working scene image in the application scenario, perform robot workload estimation, and obtain workload parameters and workload loading time. Based on the expected path conditions and the workload parameters and workload loading time obtained from the robot workload estimation, the joint position and torque control command parameters under the expected conditions are calculated to obtain the control command parameters for the robot joint motors. These control command parameters are then applied to the joint motor feedforward and feedback control.

[0074] Optionally, control command parameters for the robot joint motors are generated based on the workload parameters, workload duration, and road conditions, including: using a second neural network model to generate control command parameters for the robot joint motors based on the workload parameters, workload duration, and road conditions.

[0075] The second neural network model in the above steps can calculate the robot's control command parameters based on road conditions, expected load parameters, and loading time. These control command parameters can be the expected torque control parameters for each joint motor of the robot, or the expected position control parameters for each joint motor, but are not limited to these. The second neural network model can be trained based on a deep neural network model using historical or simulation data.

[0076] In an alternative embodiment, a second neural network model can be used to generate control command parameters based on job load parameters, job loading duration, and road condition status. Figure 5 This is a schematic diagram illustrating an optional method for generating control command parameters based on road conditions and workload, according to an embodiment of the present invention. Figure 5 As shown, the robot motion control model is used as the second neural network model. The robot's workload parameters and workload duration are input into the robot motion control model. At the same time, road condition information, including slope, step height, and road unevenness, is also input into the robot motion control model. The robot motion control model predicts the control command parameters to obtain the joint motor position control command and the joint motor torque control command, which are the control command parameters.

[0077] Optionally, a feedforward control method is used to superimpose control command parameters onto the robot joint motor control system commands, controlling the robot's joints to drive the robot to move along the running path and perform the task. This includes: using a feedforward control method to superimpose control command parameters onto the robot joint motor control system commands, and controlling the robot's joints to drive the robot to move along the running path and perform the task through the joint motor control system. The joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.

[0078] The joint motor position control loop in the above steps controls the target position of the robot's joint motors. The joint motor speed control loop controls the speed of the robot's motors when they are turned off. The joint motor current control loop controls the torque current of the joint motors during the robot's movement.

[0079] In one optional embodiment, the joint motor position control loop, joint motor speed control loop, and joint motor current control loop all adopt a feedforward control mechanism. The target position, speed, and current information in the joint motor control system are combined with the control command parameters in the form of position feedback, speed feedback, and current feedback to adjust the existing target position, speed, and current output, so that the robot moves according to the adjusted target position, speed, and current output, thereby realizing the control of the robot's motion.

[0080] Optionally, the control command parameters include: joint motor position control command parameters and joint motor torque control command parameters. The control command parameters are superimposed on the commands of the robot's joint motor control system using a feedforward control method. The joint motor control system controls the robot's joints to move along the running path and perform the work task. This includes: adding the joint motor position control command parameters to the joint motor position control loop to obtain the joint motor position control command, and adding the joint motor torque control command parameters to the joint motor current control loop to obtain the joint motor torque current control command. The joint motor control system then controls the robot's joints to move along the running path and perform the work task.

[0081] The joint motor position control command in the above steps is obtained by adding the joint motor position control command parameters to the joint motor position control loop, and is used to control the target position of the robot joint movement.

[0082] The joint motor torque control command in the above steps is obtained by adding the joint motor torque control command parameters to the joint motor current control loop, and is used to control the output torque of the robot joint movement.

[0083] In one optional embodiment, the joint motor position control command parameters and the reference position command in the joint motor control system are superimposed and added to the joint motor position control loop using a feedforward control method to obtain the joint motor position control command. The joint motor torque control command parameters are converted into motor torque current control command parameters, superimposed with the speed control output in the joint motor control system, and added to the joint motor current control loop using a feedforward control method to obtain the joint motor torque current control command. The joint motor control system controls the robot's joint-driven robot to move along the running path and perform the task according to the joint motor position control command and the joint motor torque current control command.

[0084] Optionally, path planning analysis is performed on the robot based on the current location, the work location, and the scene road condition image to obtain the robot's running path and the road condition status of the running path, including: performing path planning on the robot based on the current location and the work location to obtain the running path; and identifying the road surface environment image of the running path in the scene road condition image to obtain the road condition status.

[0085] In one optional embodiment, sensor data is used to map and locate the robot's surrounding environment, obtaining precise location information of the robot's current position and working position, as well as map information between the current position and the working position. Simultaneous Localization and Mapping (SLAM) technology can be used to plan a path based on the map information between the robot's current position and the working position. When executing the path planning algorithm, factors such as avoiding obstacles and optimizing path length are considered to plan the robot's running path.

[0086] When identifying road surface images of a travel path in road condition images, computer vision techniques can be used to segment and extract features from the images to identify various road surface conditions, such as smooth surfaces, potholes, and obstacles. Deep learning techniques can also be used to train models to automatically identify and classify road surface conditions, thereby obtaining the road condition status. However, the methods for identifying road surface images of a travel path in a scene's road condition images to obtain the road condition status are not limited to these.

[0087] Optionally, the method further includes: training the model parameters of the first initial neural network model based on a sample database of target object types in the work scenario to obtain the first neural network model.

[0088] The sample database of target object types in the above steps is a sample database containing target objects, where the target objects are located in the image of the work scene.

[0089] The first initial neural network model in the above steps is an untrained neural network model, which can be used to train and obtain the first neural network model.

[0090] The model parameters mentioned above are the weights and biases that need to be learned and adjusted during the training process. The model parameters determine how the neural network model maps input data to output data.

[0091] In one optional embodiment, a sample database of target object types in a work scene, containing images of the work scene and target objects, is acquired. This database is then used to train a first initial neural network model to identify target objects from the work scene images. Data is extracted from the sample database of target object types in the work scene as input data. The data undergoes preprocessing operations such as cleaning, standardization, and normalization to ensure data quality and usability. The input data is then fed into a pre-selected first initial neural network model for training. The model parameters are continuously updated using a backpropagation algorithm until the loss function of the first initial neural network model converges, resulting in the first neural network model.

[0092] Optionally, the method further includes: training the model parameters of the second initial neural network model based on historical data and simulation data of robot motion control to obtain the second neural network model.

[0093] The robot motion control history data in the above steps is data generated by the robot in the past when performing tasks. It can be the control command parameters of the robot joint motors corresponding to various task load parameters, task loading time, and road conditions.

[0094] The simulation data in the above steps are the control command parameters of the robot joint motors obtained through simulation under various work load parameters, work loading time, and road conditions.

[0095] The second initial neural network model in the above steps is an untrained neural network model, which can be used to train and obtain the second neural network model.

[0096] In one optional embodiment, historical robot motion control data can be obtained from a database, or simulation data can be obtained based on the results of simulation experiments. The historical robot motion control data and simulation data are used as input data to train a second initial neural network model to predict the control command parameters of the robot's joint motors based on the workload parameters, workload duration, and road conditions.

[0097] Data undergoes preprocessing operations such as cleaning, standardization, and normalization to ensure data quality and usability. The input data is then fed into a pre-selected second initial neural network model for training. The model parameters are continuously updated using the backpropagation algorithm until the loss function of the second initial neural network model converges, resulting in the second neural network model.

[0098] Example 2

[0099] According to an embodiment of the present invention, a robot joint control device is provided. This device can execute the robot control method provided in Embodiment 1 above. The specific implementation method and preferred application scenario are the same as those in Embodiment 1 above, and will not be repeated here.

[0100] Figure 6 This is a schematic diagram of a robot joint control device according to an embodiment of the present invention, such as... Figure 6 As shown:

[0101] The acquisition module 60 is used to acquire the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location. Among them, the working position is the location where the robot is to perform the task.

[0102] The planning module 62 is used to perform path planning analysis on the robot based on the current location, the work location, and the scene road condition image to obtain the robot's running path and the road condition status of the running path.

[0103] The generation module 64 is used to generate control command parameters for the robot to move from its current position to the working position and for the joint motors during the working process, based on the path condition and the image of the working scene.

[0104] The execution module 66 is used to superimpose pre-generated control command parameters onto the robot joint motor control system commands based on feedforward control to control the robot's joint motors, thereby driving the robot to move along the running path and perform the task.

[0105] Optionally, the planning module includes: a planning unit for planning the robot's path based on the current position and the work position to obtain the running path; and a second recognition unit for recognizing the road surface image of the running path in the scene road condition image to obtain the road condition status.

[0106] Optionally, the generation module includes: a first identification unit, used to identify target objects in the scene road condition image using a first neural network model, and obtain the object type and object location of the target object, wherein the target object is the object of the task to be performed by the robot; a first determination unit, used to determine the robot's task load parameters based on the object type; a second determination unit, used to determine the robot's task loading time based on the object location; and a generation unit, used to generate control command parameters based on the task load parameters, task loading time, and road condition status.

[0107] Optionally, the generation unit is also used to calculate control command parameters based on road conditions, expected workload parameters, and loading duration using a second neural network model.

[0108] Optionally, the generation unit is also used to obtain the sample scene road condition image and the annotation object type and annotation object location of the annotated objects in the sample scene road condition image, wherein the annotated objects are objects of the task to be performed that have been pre-annotated in the sample scene road condition image; and to use the initial first neural network model to identify the annotated objects in the sample scene road condition to obtain the predicted object type and predicted object location of the annotated objects.

[0109] Optionally, the execution module includes: a control unit, configured to control the robot's joint control system based on position control command and torque control command parameters in response to receiving control commands from the robot joints, so as to drive the robot to perform a task along the running path, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.

[0110] Optionally, the control unit also subtracts the current reference position control command from the position feedback using a subtractor, using this subtraction as the position control input to achieve motor position control. The position control output is subtracted from the speed feedback using a subtractor, using this subtraction as the speed control input to achieve motor speed control. The speed control output is subtracted from the current feedback using a subtractor, using this subtraction as the current control input to achieve motor current control. Then, pulse width modulation is generated and power is driven to control the robot's joint motors, driving the robot to move along the running path and perform the task.

[0111] Example 3

[0112] According to an embodiment of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the robot joint control method of Embodiment 1 during runtime.

[0113] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0114] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: acquire the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; perform path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generate control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimpose the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task.

[0115] Optionally, the processor may also be configured to perform the following steps via a computer program: using a first neural network model to identify target objects in a scene road condition image, and obtaining the object type and object location of the target objects, wherein the target objects are objects for which the robot is to perform a task; determining the robot's task load parameters based on the object type; determining the robot's task loading time based on the object location; and generating control command parameters based on the task load parameters, task loading time, and road condition status.

[0116] Optionally, the processor may also be configured to perform the following steps via a computer program: perform path planning for the robot based on the current position and the work position to obtain the running path; and identify the road surface image of the running path in the scene road condition image to obtain the road condition status.

[0117] Optionally, the processor may also be configured to perform the following steps via a computer program: obtaining control command parameters using a second neural network model based on job load parameters, job loading duration, and road condition status.

[0118] Optionally, the processor may also be configured to perform the following steps via a computer program: obtaining a sample scene road condition image and the annotation object type and annotation object location of the annotated objects in the sample scene road condition image, wherein the annotated objects are objects of the task to be performed that have been pre-annotated in the sample scene road condition image; using an initial first neural network model to identify the annotated objects in the sample scene road condition, and obtaining the predicted object type and predicted object location of the annotated objects.

[0119] Optionally, the processor may also be configured to perform the following steps via a computer program: in response to receiving control commands based on the robot joints, superimposing control command parameters onto the commands of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot to move along the running path and perform the work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.

[0120] Optionally, the processor can also be configured to perform the following steps via a computer program: subtracting the current reference position control command from the position feedback using a subtractor, and using the result as the position control input to achieve motor position control; subtracting the position control output from the speed feedback using a subtractor, and using the result as the speed control input to achieve motor speed control; subtracting the speed control output from the current feedback using a subtractor, and using the result as the current control input to achieve motor current control; and then generating pulse width modulation and power driving to control the robot's joint motors to drive the robot along the running path and perform the task.

[0121] In this embodiment of the invention, the following steps are taken: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path are obtained based on the current location, the work location, and the road condition image of the current scene. This allows for advance understanding of the current scene. Control command parameters are generated based on the road condition and the work scene image. The robot's expected action characteristics in the current scene can be determined based on the current scene conditions. By controlling the robot to perform the work task along the running path based on the control command parameters, response delays caused by insufficient familiarity with the current scene can be avoided. This allows the robot to perform tasks under the premise of familiarity with the environmental characteristics, thereby achieving the goal of improving the robot's joint response speed in unknown scenes. This achieves the technical effect of improving the robot's dynamic response speed and enhancing the stability and flexibility of robot movement.

[0122] Figure 7 This is a schematic diagram of an optional electronic device according to an embodiment of the present disclosure. (As shown) Figure 7 As shown, the electronic device 700 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0123] like Figure 7 As shown, the electronic device 700 is presented in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processor 710, at least one memory 720, a bus 730 connecting different system components (including memory 720 and processor 710), and a display 740.

[0124] The memory 720 stores program code that can be executed by the processor 710, causing the processor 710 to perform the steps described in the method section of the embodiments of this application according to various exemplary implementations of this disclosure.

[0125] The memory 720 may include a readable medium in the form of volatile memory cells, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include read-only memory (ROM) 7203, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0126] In some instances, memory 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Memory 720 may further include memory remotely located relative to processor 710, which can be connected to electronic device 700 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0127] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, peripheral bus, graphics acceleration port, processor 710, or a local bus using any of the various bus structures.

[0128] The display 740 may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of the electronic device 700.

[0129] Optionally, the electronic device 700 can also communicate with one or more external devices 800 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via the input / output (I / O) interface 750. Furthermore, the electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 760. Figure 7 As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although... Figure 7 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] The aforementioned electronic device 700 may also include: a keyboard, a cursor control device (such as a mouse), an input / output interface (I / O interface), a network interface, a power supply, and / or a camera.

[0131] Those skilled in the art will understand that Figure 7The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 700 may also include components that are more... Figure 7 The more or fewer components shown, or having the same Figure 7 Different configurations are shown. The memory 720 can be used to store computer programs and corresponding data, such as the computer program and corresponding data corresponding to the robot control method in this embodiment. The processor 710 executes various functional applications and data processing by running the computer program stored in the memory 720, thereby realizing the aforementioned robot control method.

[0132] Example 4

[0133] The embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the robot control method provided in the above embodiments.

[0134] Optionally, in this embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0135] Optionally, in this embodiment, the storage medium may be located in any one of the electronic devices in the group of electronic devices in the computer network, or in any one of the mobile terminals in the group of mobile terminals.

[0136] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: acquiring a scene road condition image of the robot's current location, the robot's current position, the robot's working position, and a working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task.

[0137] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: identifying target objects in a scene road condition image using a first neural network model to obtain the object type and object location of the target objects, wherein the target objects are objects for the robot to perform a task; determining the robot's workload parameters based on the object type; determining the robot's task loading time based on the object location; and generating control command parameters based on the workload parameters, task loading time, and road condition status.

[0138] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: performing path planning for the robot based on the current position and the work position to obtain the running path; and recognizing the road surface image of the running path in the scene road condition image to obtain the road condition status.

[0139] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: obtaining control command parameters using a second neural network model based on job load parameters, job loading duration, and road condition status.

[0140] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: obtaining a sample scene road condition image and the labeled object type and labeled object location of labeled objects in the sample scene road condition image, wherein the labeled objects are objects of the task to be performed that have been pre-labeled in the sample scene road condition image; and using an initial first neural network model to identify the labeled objects in the sample scene road condition to obtain the predicted object type and predicted object location of the labeled objects.

[0141] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: in response to receiving control commands based on robot joints, superimposing control command parameters onto the commands of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot to move along the running path and perform the work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.

[0142] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: subtracting the current reference position control command from the position feedback via a subtractor, and using this as a position control input to achieve motor position control; subtracting the position control output from the speed feedback via a subtractor, and using this as a speed control input to achieve motor speed control; subtracting the speed control output from the current feedback via a subtractor, and using this as a current control input to achieve motor current control; and then generating pulse width modulation and power driving to control the robot's joint motors to drive the robot along the running path and perform the task.

[0143] In this embodiment of the invention, the following steps are taken: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path are obtained based on the current location, the work location, and the road condition image of the current scene. This allows for advance understanding of the current scene. Control command parameters are generated based on the road condition and the work scene image. The robot's expected action characteristics in the current scene can be determined based on the current scene conditions. By controlling the robot to perform the work task along the running path based on the control command parameters, response delays caused by insufficient familiarity with the current scene can be avoided. This allows the robot to perform tasks under the premise of familiarity with the environmental characteristics, thereby achieving the goal of improving the robot's joint response speed in unknown scenes. This achieves the technical effect of improving the robot's dynamic response speed and enhancing the stability and flexibility of robot movement.

[0144] The above description of the embodiments makes it easy for those skilled in the art to understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0145] Example 5

[0146] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the robot control method of various embodiments of the present invention.

[0147] Optionally, the computer program can be configured to perform the following steps: acquire the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; perform path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generate control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimpose the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task.

[0148] Optionally, the computer program can be configured to perform the following steps: using a first neural network model to identify target objects in the scene road condition image, obtaining the object type and object location of the target objects, wherein the target objects are objects for the robot to perform the task; determining the robot's task load parameters based on the object type; determining the robot's task loading time based on the object location; and generating control command parameters based on the task load parameters, task loading time, and road condition status.

[0149] Optionally, the computer program can be configured to perform the following steps: plan the robot's path based on the current position and the work position to obtain the running path; identify the road surface image of the running path in the scene road condition image to obtain the road condition status.

[0150] Optionally, the computer program can be configured to perform the following steps: obtaining control command parameters using a second neural network model based on the job load parameters, job loading duration, and road condition status.

[0151] Optionally, the computer program can be configured to perform the following steps: obtain a sample scene road condition image and the labeled object type and labeled object location of the labeled objects in the sample scene road condition image, wherein the labeled objects are objects of the task to be performed that have been pre-labeled in the sample scene road condition image; and use an initial first neural network model to identify the labeled objects in the sample scene road condition to obtain the predicted object type and predicted object location of the labeled objects.

[0152] Optionally, the computer program can be configured to perform the following steps: in response to receiving control commands based on the robot joints, superimposing control command parameters onto the commands of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot to move along the running path and perform the work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.

[0153] Optionally, the computer program can be configured to perform the following steps: subtracting the current reference position control command from the position feedback using a subtractor, and using this subtraction as the position control input to achieve motor position control; subtracting the position control output from the speed feedback using a subtractor, and using this subtraction as the speed control input to achieve motor speed control; subtracting the speed control output from the current feedback using a subtractor, and using this subtraction as the current control input to achieve motor current control; and then generating pulse width modulation and power driving to control the robot's joint motors to drive the robot along the running path and perform the task.

[0154] In this embodiment of the invention, the following steps are taken: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path are obtained based on the current location, the work location, and the road condition image of the current scene. This allows for advance understanding of the current scene. Control command parameters are generated based on the road condition and the work scene image. The robot's expected action characteristics in the current scene can be determined based on the current scene conditions. By controlling the robot to perform the work task along the running path based on the control command parameters, response delays caused by insufficient familiarity with the current scene can be avoided. This allows the robot to perform tasks under the premise of familiarity with the environmental characteristics, thereby achieving the goal of improving the robot's joint response speed in unknown scenes. This achieves the technical effect of improving the robot's dynamic response speed and enhancing the stability and flexibility of robot movement.

[0155] Example 6

[0156] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the robot control method in various embodiments of the present invention.

[0157] It should be noted that program code contained on a non-volatile computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, or any suitable combination thereof.

[0158] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: acquiring a scene road condition image of the robot's current location, the robot's current position, the robot's working position, and a working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task.

[0159] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: identifying target objects in a scene road condition image using a first neural network model to obtain the object type and object location of the target objects, wherein the target objects are objects for the robot to perform a task; determining the robot's workload parameters based on the object type; determining the robot's task loading duration based on the object location; and generating control command parameters based on the workload parameters, task loading duration, and road condition status.

[0160] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: planning a path for the robot based on the current position and the work position to obtain a running path; and recognizing the road surface image of the running path in the scene road condition image to obtain the road condition status.

[0161] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: obtaining control command parameters using a second neural network model based on job load parameters, job loading duration, and road condition status.

[0162] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: obtaining a sample scene road condition image and the labeled object type and labeled object location of labeled objects in the sample scene road condition image, wherein the labeled objects are objects of the task to be performed that have been pre-labeled in the sample scene road condition image; and using an initial first neural network model to identify the labeled objects in the sample scene road condition to obtain the predicted object type and predicted object location of the labeled objects.

[0163] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: in response to receiving control commands based on robot joints, superimposing control command parameters onto the commands of the robot joint motor control system using a feedforward control method, controlling the robot joints to drive the robot to move along the running path and perform a work task, wherein the joint control system includes a position control loop, a speed control loop, and a current control loop.

[0164] Optionally, the non-volatile computer-readable storage medium is configured to store a computer program for performing the following steps: subtracting the current reference position control command from the position feedback via a subtractor, and using this as a position control input to achieve motor position control; subtracting the position control output from the speed feedback via a subtractor, and using this as a speed control input to achieve motor speed control; subtracting the speed control output from the current feedback via a subtractor, and using this as a current control input to achieve motor current control; and then generating pulse width modulation and power driving to control the robot's joint motors to drive the robot along the running path and perform the task.

[0165] In this embodiment of the invention, the following steps are taken: acquiring the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; performing path planning analysis on the robot based on the current position, working position, and scene road condition image to obtain the robot's running path and the road condition status of the running path; generating control command parameters for the robot's joint motors based on the road condition status and the working scene image; and superimposing the control command parameters onto the commands of the robot's joint motor control system using a feedforward control method to control the robot's joints to drive the robot to move along the running path and perform the work task. It is noteworthy that the robot's running path and the road condition of the running path are obtained based on the current location, the work location, and the road condition image of the current scene. This allows for advance understanding of the current scene. Control command parameters are generated based on the road condition and the work scene image. The robot's expected action characteristics in the current scene can be determined based on the current scene conditions. By controlling the robot to perform the work task along the running path based on the control command parameters, response delays caused by insufficient familiarity with the current scene can be avoided. This allows the robot to perform tasks under the premise of familiarity with the environmental characteristics, thereby achieving the goal of improving the robot's joint response speed in unknown scenes. This achieves the technical effect of improving the robot's dynamic response speed and enhancing the stability and flexibility of robot movement.

[0166] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0167] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented through methods where the robot's dynamic response has a delay in other unknown scenarios. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces; the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0172] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for controlling robot joints, characterized in that, include: Acquire the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location; Based on the current location, the work location, and the scene road condition image, the robot is subjected to path planning analysis to obtain the robot's running path and the road condition status of the running path; Based on the road condition and the work scene image, control command parameters for the robot joint motors are generated. The control command parameters are used to control the robot's joints to drive the robot from the current position to the work position and perform the work task. The control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters. The control command parameters are superimposed onto the commands of the robot joint motor control system using a feedforward control method, thereby controlling the robot joints to drive the robot to move along the running path and perform the task. The control command parameters for the robot joint motors, generated based on the road conditions and the work scene image, include: The first neural network model is used to identify the target object in the work scene image to obtain the object type and object location of the target object, wherein the target object is the object to which the robot is to perform the work task; The robot's workload parameters are determined based on the object type; The robot's loading time is determined based on the object's location; The second neural network model is used to generate control command parameters for the robot joint motors based on the task load parameters, the task loading duration, and the road condition.

2. The method according to claim 1, characterized in that, The control command parameters are superimposed onto the commands of the robot joint motor control system using a feedforward control method to control the robot's joints to drive the robot forward along the running path and perform work tasks, including: The control command parameters are superimposed onto the commands of the robot joint motor control system using a feedforward control method. The joint motor control system controls the robot's joints to drive the robot forward along the running path and perform the task. The joint motor control system includes a joint motor position control loop, a joint motor speed control loop, and a joint motor current control loop.

3. The method according to claim 2, characterized in that, The control command parameters include: joint motor position control command parameters and joint motor torque control command parameters. These control command parameters are superimposed onto the commands of the robot's joint motor control system using a feedforward control method. The joint motor control system then controls the robot's joints to drive the robot along the running path and execute the task, including: The joint motor position control command parameters are added to the joint motor position control loop to obtain the joint motor position control command, and the joint motor torque control command parameters are added to the joint motor current control loop to obtain the joint motor torque current control command. The joint motor control system controls the robot's joints to drive the robot to move along the running path and perform the work task.

4. The method according to claim 1, characterized in that, Based on the current location, the work location, and the scene road condition image, the robot undergoes path planning analysis to obtain the robot's running path and the road condition status of the running path, including: The robot's path is planned based on its current location and its working location to obtain the running path; The road surface environment image of the running path in the scene road condition image is identified to obtain the road condition status.

5. The method according to claim 1, characterized in that, The method further includes: Based on a sample database of target object types in the work scenario, the model parameters of the first initial neural network model are trained to obtain the first neural network model.

6. The method according to claim 1, characterized in that, The method further includes: Based on historical and simulation data of robot motion control, the model parameters of the second initial neural network model are trained to obtain the second neural network model.

7. A control device for a robot joint, characterized in that, include: The acquisition module is used to acquire the scene road condition image of the robot's current location, the robot's current position, the robot's working position, and the working scene image of the robot's current location. The planning module is used to perform path planning analysis on the robot based on the current position, the work position and the scene road condition image, so as to obtain the robot's running path and the road condition status of the running path; The generation module is used to generate control command parameters for the robot joint motors based on the road condition and the work scene image. The control command parameters are used to control the robot's joints to drive the robot from the current position to the work position and perform the work task. The control command parameters include at least one of the following: joint motor position control command parameters and joint motor torque control command parameters. The execution module is used to superimpose the control command parameters onto the commands of the robot joint motor control system using a feedforward control method, thereby controlling the robot's joints to drive the robot to move along the running path and perform the work task; The generation module is used to identify target objects in the work scene image using a first neural network model, and obtain the object type and object location of the target objects, wherein the target objects are the objects for which the robot is to perform the work task; determine the robot's work load parameters based on the object type; determine the robot's work loading time based on the object location; and generate control command parameters for the robot's joint motors using a second neural network model based on the work load parameters, the work loading time, and the road condition status.

8. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images