Optimization method and device for mechanical arm control, electronic equipment and readable medium

Abstract

The application relates to an optimization method and device for mechanical arm control, electronic equipment and a readable medium, comprising: acquiring a preset model and a distance value of a robot from a first elevator button in a current scene, wherein the training data of the preset model is a data set generated by the robot interacting with a lift control system; dynamically optimizing joint weight parameters of the preset model through the distance value to obtain an optimized preset model, wherein the joint weight parameters are used to represent the importance of different mechanical arm joints of the robot when action prediction is performed; and using the optimized preset model to control the mechanical arm of the robot, so that when each joint of the mechanical arm of the robot performs a first pressing action according to first action parameters output by the optimized preset model, the first elevator button is triggered. The problem that the weights of different joints of the mechanical arm are not differentiated and controlled according to the actual scene is solved.

Description

Technical Field

[0001] This application relates to the field of intelligent robot technology, and in particular to an optimization method, apparatus, electronic device and readable medium for controlling a robotic arm. Background Technology

[0002] Service robots and food delivery robots are becoming increasingly common in hotels, office buildings, and other similar settings. One of their key functions is to autonomously use elevators to provide services across floors. For elevators equipped with remote control systems, robots can remotely call the elevator and select floors through integration with the elevator control system, eliminating the need for physical operation. However, in many elevators without remote control, robots must rely on robotic arms to simulate human actions by pressing elevator buttons.

[0003] Existing robotic arm control models often employ a uniform joint drive strategy when executing button presses, failing to differentiate the weights of different joints for specific scenarios. This coarse motion prediction approach is prone to positioning errors and trigger failures in practical applications, reducing the robot's adaptability and success rate in non-standard elevator environments.

[0004] Therefore, how to differentiate the weights of different joints of a robotic arm for specific real-world scenarios has become a pressing technical problem.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides an optimization method, apparatus, electronic device, and readable medium for robotic arm control to solve the aforementioned technical problem of "not performing differentiated control of the weights of different joints of the robotic arm for actual scenarios".

[0007] According to one aspect of the embodiments of this application, this application provides an optimization method for robotic arm control, comprising: acquiring a preset model and acquiring the distance value between the robot and a first elevator button in the current scene, wherein the training data of the preset model is a dataset generated by the robot interacting with an elevator control system; dynamically optimizing the joint weight parameters of the preset model through the distance value to obtain an optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints of the robot in motion prediction; and using the optimized preset model to control the robotic arm of the robot so that when each joint of the robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, the first elevator button is triggered.

[0008] Optionally, before obtaining the preset model, the method further includes training the preset model in the following manner: obtaining a dataset generated by the interaction between the robot and the elevator control system; adding joint weight parameters to the initial loss function of the initial model to obtain a target loss function; and iteratively training the initial model using the target loss function and the dataset to obtain the preset model.

[0009] Optionally, the dataset generated by the interaction between the robot and the elevator control system includes: sending control commands to the robot to control the robot to perform a second pressing action, wherein the second pressing action is used to drive the robot's robotic arm to press the second elevator button; acquiring the second action parameters corresponding to the second pressing action, and collecting image features of the second elevator button, wherein the second action parameters include pressing force, pressing angle, and pressing duration; receiving the arrival floor returned by the elevator control system, and identifying whether the arrival floor matches the control command, generating a result label; mapping and storing the control command, second action parameters, image features, and result labels to obtain candidate data; and integrating the result labels into successful candidate data to obtain the dataset.

[0010] Optionally, joint weight parameters are added to the initial loss function of the initial model to obtain the target loss function, including: constructing the initial model and obtaining the joint weight parameters; constructing a first loss function for the joint angle prediction subtask and a second loss function for the button trigger state prediction subtask based on the joint weight parameters; and weighting and fusing the first loss function and the second loss function to obtain the target loss function.

[0011] Optionally, the initial model is iteratively trained using the target loss function and the dataset to obtain a preset model, including: using image features, second action parameters, and control commands from the dataset as input data and inputting the input data into the initial model; obtaining the joint angles and button trigger states output by the initial model; extracting the target results corresponding to the input data from the dataset, and determining the first prediction error and the second prediction error based on the joint angles, button trigger states, and target results respectively; substituting the first prediction error and the second prediction error into the target loss function to obtain the actual loss function value; using the actual loss function value to backpropagate the model parameters and joint weight parameters to obtain gradient values; updating the model parameters and joint weight parameters based on the gradient values; and continuing to train the model using the updated model parameters and joint weight parameters until the target loss function converges to obtain the preset model.

[0012] Optionally, obtaining the distance between the robot and the first elevator button in the current scene includes: obtaining a region image of the first elevator button, and obtaining the current joint information of the robotic arm and the posture information of the end effector, wherein the current joint information is used to characterize the current internal configuration state of the robotic arm; extracting features from the region image, the current joint information, and the posture information respectively to obtain a first feature, a second feature, and a third feature; fusing the first feature, the second feature, and the third feature to obtain a fused feature vector; compressing the fused feature vector to map the fused feature vector to a scalar; and determining the scalar as the distance between the robot's end effector and the first elevator button.

[0013] Optionally, the joint weight parameters of the preset model are dynamically optimized using distance values, including: obtaining the initial weight values ​​of each joint from the joint weight parameters; comparing the distance values ​​with a first threshold and a second threshold respectively, and optimizing the initial weight values ​​of each joint based on the comparison results, wherein the first threshold is greater than the second threshold.

[0014] Optionally, the initial weight values ​​of each joint are optimized based on the comparison results, including: if the distance value is greater than or equal to a first threshold, the initial weight value of the base joint is increased to a first target value, and the initial weight value of the wrist joint is decreased to a second target value; or if the distance value is less than the first threshold and greater than the second threshold, the initial weight value of the base joint is adjusted to a third target value, and the initial weight value of the wrist joint is adjusted to a fourth target value, wherein the third target value is less than the first target value and the fourth target value is greater than the second target value; or if the distance value is less than or equal to the second threshold, the initial weight value of the base joint is decreased to a fifth target value, and the initial weight value of the wrist joint is increased to a sixth target value, wherein the fifth target value is less than the third target value and the sixth target value is greater than the fourth target value.

[0015] According to another aspect of the embodiments of this application, this application provides an optimization device for robotic arm control, comprising: an acquisition module, configured to acquire a preset model and acquire the distance value between the robot and a first elevator button in the current scene, wherein the training data of the preset model is a dataset generated by the robot interacting with an elevator control system; an optimization module, configured to dynamically optimize the joint weight parameters of the preset model using the distance value to obtain an optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints of the robot in motion prediction; and a control module, configured to control the robotic arm of the robot using the optimized preset model, so that when each joint of the robotic arm of the robot performs a first pressing action according to the first action parameters output by the optimized preset model, the first elevator button is triggered.

[0016] According to another aspect of the embodiments of this application, this application provides an electronic device, including a memory, a processor, a communication interface and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. When the processor executes the computer program, it implements the steps of the above method.

[0017] According to another aspect of the embodiments of this application, this application also provides a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the above-described method.

[0018] Compared with related technologies, the technical solutions provided in this application have the following advantages: This application provides an optimization method for robotic arm control, comprising: acquiring a preset model and acquiring the distance value between the robot and a first elevator button in the current scenario, wherein the training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system; dynamically optimizing the joint weight parameters of the preset model using the distance value to obtain an optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints in motion prediction; and using the optimized preset model to control the robotic arm so that when each joint of the robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, the first elevator button is triggered. By dynamically optimizing the joint weights based on the actual distance between the robot and the elevator button during the model execution phase, the importance of different joints in different operating scenarios is differentiated. This solves the problem of not differentiating the weights of different joints of the robotic arm for actual scenarios. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the hardware environment for an optional optimized method for robotic arm control provided according to an embodiment of this application; Figure 2 A flowchart illustrating an optional optimized method for robotic arm control according to an embodiment of this application; Figure 3This is a schematic diagram of the composition of an optional optimized system for robotic arm control according to an embodiment of this application; Figure 4 This is a block diagram of an optional optimized device for robotic arm control according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0024] Service robots and food delivery robots are becoming increasingly common in hotels, office buildings, and other similar settings. One of their key functions is to autonomously use elevators to provide services across floors. For elevators equipped with remote control systems, robots can remotely call the elevator and select floors through integration with the elevator control system, eliminating the need for physical operation. However, in many elevators without remote control, robots must rely on robotic arms to simulate human actions by pressing elevator buttons.

[0025] Existing robotic arm control models often employ a uniform joint drive strategy when executing button presses, failing to differentiate the weights of different joints for specific scenarios. This coarse motion prediction approach is prone to positioning errors and trigger failures in practical applications, reducing the robot's adaptability and success rate in non-standard elevator environments.

[0026] Therefore, how to differentiate the weights of different joints of a robotic arm for specific real-world scenarios has become a pressing technical problem.

[0027] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of an optimized method for controlling a robotic arm is provided.

[0028] Optionally, in the embodiments of this application, the above-described optimized method for robotic arm control can be applied to, for example... Figure 1 The hardware environment shown consists of terminal 101 and server 103. Figure 1As shown, server 103 is connected to terminal 101 via a network and can be used to provide services to the terminal or clients installed on the terminal. Database 105 can be set up on the server or independently of the server to provide data storage services for server 103. The network mentioned above includes, but is not limited to, wide area network, metropolitan area network or local area network. Terminal 101 includes, but is not limited to, PC, mobile phone, tablet computer, etc.

[0029] An optimized method for robotic arm control in this embodiment can be executed by server 103, or it can be executed jointly by server 103 and terminal 101, such as... Figure 2 As shown, it includes: Step 201: Obtain a preset model and obtain the distance value between the robot and the first elevator button in the current scene. The training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system. Step 202: Dynamically optimize the joint weight parameters of the preset model using distance values ​​to obtain the optimized preset model. The joint weight parameters are used to characterize the importance of different robotic arm joints in motion prediction. Step 203: Use the optimized preset model to control the robot's robotic arm so that when each joint of the robot's robotic arm performs the first pressing action according to the first action parameters output by the optimized preset model, it triggers the first elevator button.

[0030] The optimized control method for joint-level robotic arm control in this application mainly includes: acquiring a preset model containing joint weight parameters, optimizing the joint weights during the model usage phase in actual scenarios, and enabling the robotic arm to perform differentiated control of different joints according to actual scenarios through differential learning during model training and dynamic parameter optimization during model application, thereby improving the accuracy and success rate of elevator button pressing and enhancing the robot's adaptability in non-standard elevator environments.

[0031] During the model usage phase in real-world scenarios, the joint weights in the preset model are dynamically adjusted based on the distance between the robot's end effector and the target elevator button. For example, when the distance is greater, the weight of the base joint is increased to achieve rapid positioning, while when the distance is less, the weight of the wrist joint is increased to ensure accurate pressing.

[0032] It should be noted that although in the embodiments of this application, "current scenario" refers to a scenario of non-standard elevator system, which can be simply understood as a scenario where the elevator cannot be called directly by command but can only be pressed by pressing the button, this application can still be applied to the scenario of standard elevator system. It is not only applicable to hotel scenarios, but can also be extended to diverse environments such as parks, hospitals, and office buildings.

[0033] As an optional embodiment, before obtaining the preset model, the method further includes training the preset model in the following manner: obtaining a dataset generated by the interaction between the robot and the elevator control system; adding joint weight parameters to the initial loss function of the initial model to obtain a target loss function; and iteratively training the initial model using the target loss function and the dataset to obtain the preset model.

[0034] During the model training phase, the interaction dataset between the robot and the ladder control system is used as training data. Joint weight parameters are added to the loss function of the initial model, enabling the model to identify and learn the importance of different joints in action prediction. Thus, the initial model can gradually learn how to output fine-grained joint-level control strategies during the training process.

[0035] The training steps of the model will be explained in detail below through specific examples.

[0036] As an optional embodiment, acquiring the dataset generated by the interaction between the robot and the elevator control system includes: sending a control command to the robot to control the robot to perform a second pressing action, wherein the second pressing action is used to drive the robot's robotic arm to press a second elevator button; acquiring second action parameters corresponding to the second pressing action, and collecting image features of the second elevator button, wherein the second action parameters include pressing force, pressing angle, and pressing duration; receiving the arrival floor returned by the elevator control system, and identifying whether the arrival floor matches the control command, generating a result label; mapping and storing the control command, second action parameters, image features, and result labels to obtain candidate data; and integrating the result labels into successful candidate data to obtain the dataset.

[0037] The dataset in this application was generated by collecting interaction data between the robot and the elevator control system in different elevator control scenarios. Based on diverse hotel elevator data with elevator control systems, a more adaptable robotic arm control model was constructed and trained, enabling it to cope with different types of elevators and complex scenarios.

[0038] Control commands are sent to the robot to drive the robotic arm to perform the second pressing action. The control commands will carry the target floor (and if it is a call for an elevator, it will also carry the robot's current floor information). Second action parameters related to this action are collected, including multi-dimensional data such as pressing force, angle, and duration, and image features of the second elevator button are collected simultaneously.

[0039] It receives the arrival floor information returned by the elevator control system, compares it with the control command, and uses this information to determine whether the second pressing action was successful.

[0040] The comparison results are automatically used to generate result labels, which are then mapped and stored with the second action parameters, image features, and control commands to form candidate data.

[0041] In this embodiment, only candidate data corresponding to successful labels are integrated as valid training samples, and invalid action data (i.e., actions that cannot be completed to call the elevator or reach the specified floor) are removed to achieve data cleaning, thereby constructing a high-quality dataset.

[0042] For example, recording the motion parameters of a robotic arm when simulating human operation. (including pressure) ,angle Image features of time series t and elevator buttons (Shape, material, layout, etc.). Valid data can be represented as: in, Elevator control commands, The comparison results are based on the actual floors reached. For robotic arm motion parameters , For the image features of the button, This represents the number of data samples.

[0043] By utilizing the work data of hotel robots with elevator control systems that are directly integrated with elevator systems, highly accurate training data can be automatically generated, replacing traditional manual annotation and significantly reducing the time and economic cost of data preparation.

[0044] In the embodiments of this application, the acquisition of training data is automatically completed by the actual interaction between the robot and the elevator, which not only avoids the tedious process of manually labeling elevator buttons and operation results one by one, but also enables automatic collection and high-quality screening of training data.

[0045] As an optional embodiment, joint weight parameters are added to the initial loss function of the initial model to obtain the target loss function, including: constructing the initial model and obtaining the joint weight parameters; constructing a first loss function for the joint angle prediction subtask and a second loss function for the button trigger state prediction subtask based on the joint weight parameters; and performing weighted fusion of the first loss function and the second loss function to obtain the target loss function.

[0046] An initial model capable of predicting both joint angles and button triggers is constructed to describe the kinematics and task objectives of the robotic arm in an elevator operation scenario. The initial model provided in this application is a VLA (Vision-Language-Action) model. After the model is trained, it is deployed at the robot's robotic arm.

[0047] The initial values ​​of the joint weight parameters can be set according to the actual situation and will be optimized and adjusted according to changes in the scenario. The joint weight parameters can characterize the importance of different joints of the robotic arm in the motion prediction task. For example, the base joint is more inclined towards coarse positioning, while the wrist joint is more inclined towards fine manipulation.

[0048] For example, if it is divided into 6 joints, the motion output layer of the VLA model will output the motion of all 6 joints. Introducing a weight matrix Calculate the weighted output Weight This reflects the priority of joints in the task. For example, base joints (joints 1 and 2) have a higher weight (e.g., 1.0), facilitating rapid and wide-range movements; elbow joints (joints 3 and 4) have a medium weight (e.g., 0.5), balancing position adjustment and postural stability; wrist joints (joints 5 and 6) have a lower weight (e.g., 0.2), ensuring precise alignment and pressure. The weights satisfy normalization constraints (e.g., ...). or To avoid losing control of the range of motion.

[0049] The first loss function corresponds to the joint angle prediction subtask. It is derived by weighting the errors between the predicted and actual angles of each joint of the robotic arm according to the joint weight parameters. This first loss function ensures that the importance of different joints is differentiated in the error feedback.

[0050] For example, the weighted mean squared error loss function is used: ,in, For the target joint angle, To predict joint angles, weights The model is guided to prioritize optimizing the prediction accuracy of key joints.

[0051] The second loss function corresponds to the button trigger state prediction subtask. It establishes the state prediction loss for whether the robotic arm's pressing action triggers the elevator button, which can measure the accuracy of the model at the task result level.

[0052] For example, the overall optimization objective is to minimize the prediction error: ,in, This represents the button trigger state predicted by the model. This is the actual triggered state.

[0053] The first loss function and the second loss function are weighted and fused according to a certain ratio to obtain the target loss function. The specific weighting ratio can be set according to the actual situation. The target loss function not only constrains the accuracy at the action level, but also constrains the success rate of task execution.

[0054] This embodiment introduces joint weight parameters into the loss function and constructs a weighted target loss function, thereby achieving differentiated modeling and optimization of the importance of different joints, which improves the accuracy of motion prediction of the robotic arm in complex elevator environments and the success rate of button triggering.

[0055] As an optional embodiment, an initial model is iteratively trained using a target loss function and a dataset to obtain a preset model. This includes: using image features, second action parameters, and control commands from the dataset as input data, and inputting the input data into the initial model; obtaining the joint angles and button trigger states output by the initial model; extracting the target results corresponding to the input data from the dataset, and determining a first prediction error and a second prediction error based on the joint angles, button trigger states, and target results, respectively; substituting the first prediction error and the second prediction error into the target loss function to obtain the actual loss function value; using the actual loss function value to backpropagate the model parameters and joint weight parameters to obtain gradient values; updating the model parameters and joint weight parameters based on the gradient values; and continuing to train the model using the updated model parameters and joint weight parameters until the target loss function converges to obtain the preset model.

[0056] Training samples were selected from the constructed dataset, including image features of elevator buttons, second motion parameters related to the robotic arm pressing action, and control commands to guide the robotic arm to perform the action. The second motion parameters include pressing force, pressing angle, and pressing duration. This multimodal information was combined and used as input data for the initial model.

[0057] After receiving input data, the initial model outputs two types of prediction results: one is the predicted value of the angle of each joint of the robotic arm, which is used to describe the motion planning; the other is the predicted value of the button trigger state, which is used to reflect whether the robotic arm's motion can effectively trigger the elevator button.

[0058] Extract the target result corresponding to the input data from the dataset, including the real joint angle data and the real label of the button trigger. Then compare the prediction result with the target result to obtain the first prediction error and the second prediction error. The first prediction error is the deviation between the predicted joint angle value and the real angle, and the second prediction error is the difference between the predicted button trigger value and the real trigger state.

[0059] The first prediction error and the second prediction error are substituted into the target loss function. The first prediction error part also introduces joint weight parameters to distinguish the importance of different joints in action prediction. The second prediction error part reflects the success rate of task completion. The actual loss function value is obtained through weighted fusion to ensure that the model is optimized in both action accuracy and task performance.

[0060] Backpropagation is performed using the actual loss function value to calculate the gradient values ​​corresponding to the model parameters and joint weight parameters. Then, the parameters are updated based on the gradients, causing the model to gradually optimize towards loss convergence. This training process is repeated continuously, using the updated model parameters and joint weight parameters for iterative optimization, until the target loss function reaches the convergence condition, thus obtaining the final preset model.

[0061] Through this embodiment, the model not only optimizes the overall prediction capability during training, but also dynamically learns and adjusts the importance weights of each joint, achieving a synergistic improvement in motion prediction accuracy and button trigger success rate, and significantly improving the adaptability and operational reliability of the robotic arm in non-standard elevator environments.

[0062] As an optional embodiment, obtaining the distance between the robot and the first elevator button in the current scene includes: acquiring a region image of the first elevator button, and acquiring the current joint information of the robotic arm and the posture information of the end effector, wherein the current joint information is used to characterize the current internal configuration state of the robotic arm; extracting features from the region image, the current joint information, and the posture information respectively to obtain a first feature, a second feature, and a third feature; fusing the first feature, the second feature, and the third feature to obtain a fused feature vector; compressing the fused feature vector to map the fused feature vector to a scalar; and determining the scalar as the distance between the robot's end effector and the first elevator button.

[0063] During the deployment phase, the trained model is applied to hotel robots in elevators that do not support remote elevator control, optimizing the robotic arm's operational performance in non-standard elevator environments and adapting to different types of elevator buttons and complex scenarios (such as changes in lighting, button aging, etc.).

[0064] The output of the preset model is: ,in For the predicted robotic arm motion parameters, For the trained model, These are the model parameters.

[0065] The robotic arm provided in this application is a multi-degree-of-freedom system (such as 6DoF), where each joint corresponds to a rotational angle (joint angle) or linear displacement (sliding joint). The current joint information may include the joint angle (i.e., the current rotational position of the joint), the joint speed (the rotational speed of the joint), and the joint force / torque (i.e., the magnitude of the torque required to drive the joint). This information together describes the internal configuration state of the robotic arm and determines the overall shape of the robotic arm.

[0066] An end effector is the tool at the very tip of a robotic arm, such as a gripper or a probe for pressing a button. Attitude information includes position, orientation, velocity, and forces. Position represents the end effector's three-dimensional coordinates in space; orientation represents its facing direction in space, commonly expressed using Euler angles or quaternions; velocity represents the linear and angular velocities of the end effector; and forces represent the forces and torques exerted when the end effector contacts an object. Attitude information characterizes the robotic arm's actual performance in the external task space, i.e., "where the hand is, and where it points."

[0067] The feature dimensions corresponding to the posture information and the current joint information are relatively high, and it would be complicated to directly use them to calculate the distance between the end effector and the first elevator button. Therefore, the lightweight linear layer is used to fuse the multimodal features (i.e., the first feature, the second feature, and the third feature) and map them into a scalar. The scalar can be used to represent the distance between the end effector and the first elevator button.

[0068] After obtaining the distance value, it can be normalized to align with different camera calibrations, thereby eliminating numerical differences under different environments.

[0069] To ensure efficient and precise movements, the 6DoF robotic arm dynamically adjusts joint weights based on the distance (d) between the end effector and the button, using functions... accomplish.

[0070] For example, the feature extraction module of the VLA model (based on CNN or Vision Transformer) extracts high-dimensional feature vectors from video frames and joint poses, containing the relative position information of the button and the end effector; this feature vector is input into a linear layer and outputs a predicted distance. Weight adjustment function based on Dynamically adjust joint weights: when When the weight is larger, the weight of the base joints (joints 1 and 2) is increased (e.g., ), accelerate rough positioning; when When the wrist is smaller, the weight of the wrist joints (joints 5 and 6) is increased (e.g. ), optimize for precise pressing.

[0071] Weight adjustment function It can be based on a distance threshold or a continuous function (such as...) This ensures a smooth transition.

[0072] This embodiment accurately predicts the distance between the robot's end effector and the elevator button by fusing visual features with its own joint and posture features, thereby providing a quantitative basis for the dynamic adjustment of joint weights and improving the robot's positioning accuracy and pressing success rate in complex environments.

[0073] As an optional embodiment, the joint weight parameters of the preset model are dynamically optimized by means of distance values, including: obtaining the initial weight values ​​of each joint from the joint weight parameters; comparing the distance values ​​with a first threshold and a second threshold respectively, and optimizing the initial weight values ​​of each joint according to the comparison results, wherein the first threshold is greater than the second threshold.

[0074] The initial weights set during training can serve as a baseline for dynamic adjustment during deployment, maintaining consistency between training and deployment while allowing real-time adaptation to task requirements. Through high-quality training data and a weighted optimization mechanism, the model significantly improves efficiency in button recognition, activation status detection, and triggering, adapting to non-standardized elevator systems. Initial weights for each joint can be pre-set based on their functional roles within the robotic arm, such as a higher weight for the base joint and a lower weight for the wrist joint.

[0075] The joints in this application are divided into base joints, elbow joints, and wrist joints.

[0076] In the actual application stage of the model, the initial weight values ​​are used as a basis, and then dynamically adjusted according to the distance values ​​in the current scene to obtain the final joint weights.

[0077] The first threshold and the second threshold can be set according to the actual situation and are used to measure the distance. This application does not limit the specific values, as long as the first threshold is greater than the second threshold.

[0078] This application determines which distance interval a distance value belongs to by comparing it with a first threshold and a second threshold, and then adjusts the weights for each interval. The following section provides a detailed explanation of how the weights are adjusted for each interval.

[0079] As an optional embodiment, optimizing the initial weight values ​​of each joint based on the comparison results includes: if the distance value is greater than or equal to a first threshold, increasing the initial weight value of the base joint to a first target value and decreasing the initial weight value of the wrist joint to a second target value; or if the distance value is less than the first threshold and greater than the second threshold, adjusting the initial weight value of the base joint to a third target value and adjusting the initial weight value of the wrist joint to a fourth target value, wherein the third target value is less than the first target value and the fourth target value is greater than the second target value; or if the distance value is less than or equal to the second threshold, decreasing the initial weight value of the base joint to a fifth target value and increasing the initial weight value of the wrist joint to a sixth target value, wherein the fifth target value is less than the third target value and the sixth target value is greater than the fourth target value.

[0080] For ease of understanding, assume that the initial weight value of each joint is 0.5. Throughout the optimization process, the elbow joint maintains the baseline value (initial weight value), which is equivalent to stabilizing the arm length and spatial orientation.

[0081] If the distance value is greater than or equal to the first threshold, it is assumed that the robot's end effector is far away from the elevator button. In this case, the weight of the base joint is increased to the first target value (e.g., 1.0) to accelerate coarse localization, and the weight of the wrist joint is reduced to the second target value (e.g., 0.3) to avoid premature fine-tuning and wasting computing resources.

[0082] When the distance value is less than or equal to the second threshold, it is considered that the robot's end effector is close to the elevator button. Then, the weight of the wrist joint is increased to the sixth target value (e.g., 0.7) to ensure accurate alignment and pressing, and the weight of the base joint is reduced to the fifth target value (e.g., 0.4) to reduce large-scale movement interference.

[0083] If the distance value is less than the first threshold and greater than the second threshold, it is determined to be in the intermediate stage. At this time, the base weight drops back to the third target value (e.g., 0.7), and the wrist weight is adjusted to the fourth target value (0.4). The overall division of labor transitions from coarse positioning to fine operation, ensuring stable connection of movements.

[0084] This embodiment enables graded adjustment of the weights of the base joint and wrist joint based on different distance ranges between the end effector and the button, achieving progressive control from coarse positioning to fine pressing, thereby improving the flexibility and stability of the robotic arm's movements and ensuring a high success rate of triggering in complex elevator environments.

[0085] Figure 3 The figure shows a schematic diagram of an optimized robotic arm control system according to an embodiment of this application. The overall architecture includes four parts: data collection, model training, model deployment, and extended applications. Data collection includes: acquiring elevator button image features (different lighting, shapes, materials, and layouts); synchronously recording robotic arm motion parameters (posture, timing, etc.); receiving elevator control commands (call, target floor) via a communication module; and recording the control commands and actual floor information (Ground Truth). Model training includes the VLA algorithm, visual and robotic arm state inputs, different joint weights (initial settings), and training the model to predict the optimal trigger posture of the robotic arm. Model deployment includes adapting to non-standardized elevators (new / unseen systems), deploying the model to the robotic arm, visual and robotic arm state inputs, different joint weights (weight optimization), and outputting robotic arm behavior parameters. Extended applications include adapting to elevators in non-hotel scenarios (parks, hospitals, office buildings). Specific control methods based on this system are detailed above.

[0086] By continuously collecting and updating data, the system constantly optimizes model performance to adapt to new elevator systems and more complex operating scenarios. The robotic arm module is designed to be modularly installed on existing hotel service robots or food delivery robots, enabling elevator interaction without modifying the robot's original functions. Simultaneously, motion data and elevator interaction data are collected while the robot is performing routine tasks (such as food delivery and cleaning), ensuring seamless compatibility between data collection and normal robot operation.

[0087] This application significantly improves the elevator operation capabilities of service robots in hotels and diverse scenarios worldwide, reduces the error rate, and enhances the system's versatility and future expansion potential.

[0088] This application provides an optimization method for robotic arm control, comprising: acquiring a preset model and acquiring the distance value between the robot and a first elevator button in the current scenario, wherein the training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system; dynamically optimizing the joint weight parameters of the preset model using the distance value to obtain an optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints in motion prediction; and using the optimized preset model to control the robotic arm so that when each joint of the robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, the first elevator button is triggered. By dynamically optimizing the joint weights based on the actual distance between the robot and the elevator button during the model execution phase, the importance of different joints in different operating scenarios is differentiated. This solves the problem of not differentiating the weights of different joints of the robotic arm for actual scenarios.

[0089] According to another aspect of the embodiments of this application, this application provides an optimization device for robotic arm control, such as... Figure 4 As shown, it includes: The acquisition module 401 is used to acquire a preset model and acquire the distance value between the robot and the first elevator button in the current scene. The training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system. The optimization module 402 is used to dynamically optimize the joint weight parameters of the preset model through distance values ​​to obtain the optimized preset model. The joint weight parameters are used to characterize the importance of different robotic arm joints in motion prediction. The control module 403 is used to control the robot's robotic arm using an optimized preset model, so that when each joint of the robot's robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, it triggers the first elevator button.

[0090] It should be noted that the acquisition module 401 in this embodiment can be used to execute step 201 in this application embodiment, the optimization module 402 in this embodiment can be used to execute step 202 in this application embodiment, and the control module 403 in this embodiment can be used to execute step 203 in this application embodiment.

[0091] Optionally, the device further includes a training module for training the preset model in the following manner before acquiring the preset model: acquiring the dataset generated by the interaction between the robot and the elevator control system; adding joint weight parameters to the initial loss function of the initial model to obtain the target loss function; and iteratively training the initial model using the target loss function and the dataset to obtain the preset model.

[0092] Optionally, the training module is further configured to send control commands to the robot to control the robot to perform a second pressing action, wherein the second pressing action is used to drive the robot's robotic arm to press the second elevator button; acquire the second action parameters corresponding to the second pressing action, and collect image features of the second elevator button, wherein the second action parameters include pressing force, pressing angle, and pressing duration; receive the arrival floor returned by the elevator control system, and identify whether the arrival floor matches the control command, generating a result label; map and store the control command, second action parameters, image features, and result labels to obtain candidate data; and integrate the result labels into successful candidate data to obtain the dataset.

[0093] Optionally, the training module is further used to build an initial model and obtain joint weight parameters; to build a first loss function for the joint angle prediction subtask based on the joint weight parameters, and a second loss function for the button trigger state prediction subtask; and to perform weighted fusion of the first loss function and the second loss function to obtain the target loss function.

[0094] Optionally, the training module is further configured to take image features, second action parameters, and control commands from the dataset as input data and input the input data into the initial model; obtain the joint angles and button trigger states output by the initial model; extract the target results corresponding to the input data from the dataset, and determine the first prediction error and the second prediction error based on the joint angles, button trigger states, and target results respectively; substitute the first prediction error and the second prediction error into the target loss function to obtain the actual loss function value; use the actual loss function value to backpropagate the model parameters and joint weight parameters to obtain the gradient value; update the model parameters and joint weight parameters based on the gradient value; and continue to train the model using the updated model parameters and joint weight parameters until the target loss function converges to obtain the preset model.

[0095] Optionally, the acquisition module 401 is specifically used to acquire a region image of the first elevator button, and acquire the current joint information of the robotic arm and the posture information of the end effector, wherein the current joint information is used to characterize the current internal configuration state of the robotic arm; to extract features from the region image, the current joint information and the posture information respectively to obtain a first feature, a second feature and a third feature; to fuse the first feature, the second feature and the third feature to obtain a fused feature vector; to compress the fused feature vector to map the fused feature vector into a scalar; and to determine the scalar as the distance value between the robot's end effector and the first elevator button.

[0096] Optionally, the optimization module 402 includes: The `get` submodule is used to obtain the initial weight values ​​of each joint from the joint weight parameters. The comparison submodule is used to compare the distance value with the first threshold and the second threshold respectively, and optimize the initial weight value of each joint based on the comparison result, wherein the first threshold is greater than the second threshold.

[0097] Optionally, the comparison submodule is specifically configured to: if the distance value is greater than or equal to a first threshold, increase the initial weight value of the base joint to a first target value and decrease the initial weight value of the wrist joint to a second target value; or if the distance value is less than the first threshold and greater than the second threshold, adjust the initial weight value of the base joint to a third target value and adjust the initial weight value of the wrist joint to a fourth target value, wherein the third target value is less than the first target value and the fourth target value is greater than the second target value; or if the distance value is less than or equal to the second threshold, decrease the initial weight value of the base joint to a fifth target value and increase the initial weight value of the wrist joint to a sixth target value, wherein the fifth target value is less than the third target value and the sixth target value is greater than the fourth target value.

[0098] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown.

[0099] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 5 As shown, the system includes a memory 501, a processor 503, a communication interface 505, and a communication bus 507. The memory 501 stores a computer program that can run on the processor 503. The memory 501 and the processor 503 communicate through the communication interface 505 and the communication bus 507. When the processor 503 executes the computer program, it implements the steps of the above method.

[0100] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0101] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0102] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0103] According to another aspect of the embodiments of this application, a computer-readable medium having processor-executable non-volatile program code is also provided.

[0104] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0105] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0106] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0107] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An optimized method for controlling a robotic arm, characterized in that, include: Obtain a preset model and obtain the distance value between the robot and the first elevator button in the current scene, wherein the training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system; The joint weight parameters of the preset model are dynamically optimized by means of the distance value to obtain the optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints of the robot in motion prediction. The optimized preset model is used to control the robot's robotic arm so that when each joint of the robot's robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, the first elevator button is triggered. Before obtaining the preset model, the method further includes training the preset model in the following manner: obtaining the dataset generated by the interaction between the robot and the elevator control system; adding the joint weight parameters to the initial loss function of the initial model to obtain the target loss function; and iteratively training the initial model using the target loss function and the dataset to obtain the preset model. The step of obtaining the distance value between the robot and the first elevator button in the current scene includes: obtaining a region image of the first elevator button, and obtaining the current joint information of the robotic arm and the posture information of the end effector, wherein the current joint information is used to characterize the current internal configuration state of the robotic arm; performing feature extraction on the region image, the current joint information, and the posture information respectively to obtain a first feature, a second feature, and a third feature; fusing the first feature, the second feature, and the third feature to obtain a fused feature vector; compressing the fused feature vector to map the fused feature vector into a scalar; and determining the scalar as the distance value between the robot's end effector and the first elevator button.

2. The method according to claim 1, characterized in that, The process of acquiring the dataset generated by the interaction between the robot and the elevator control system includes: Send a control command to the robot to control the robot to perform a second pressing action, wherein the second pressing action is used to drive the robot's robotic arm to press a second elevator button; Acquire the second action parameters corresponding to the second pressing action, and collect the image features of the second elevator button, wherein the second action parameters include pressing force, pressing angle and pressing duration; Receive the arrival floor returned by the elevator control system, identify whether the arrival floor matches the control command, and generate a result label; The control commands, second action parameters, image features, and result labels are mapped and stored to obtain candidate data; The candidate data labeled as successful are integrated to obtain the dataset.

3. The method according to claim 2, characterized in that, The step of adding joint weight parameters to the initial loss function of the initial model to obtain the target loss function includes: Construct the initial model and obtain the joint weight parameters; Based on the joint weight parameters, a first loss function for the joint angle prediction subtask and a second loss function for the button trigger state prediction subtask are constructed. The first loss function and the second loss function are weighted and fused to obtain the target loss function.

4. The method according to claim 3, characterized in that, The step of iteratively training the initial model using the target loss function and the dataset to obtain a preset model includes: The image features, the second action parameters, and the control commands in the dataset are used as input data, and the input data is input into the initial model. Obtain the joint angles and button trigger states output by the initial model; Extract the target result corresponding to the input data from the dataset, and determine the first prediction error and the second prediction error based on the joint angle, the button trigger state and the target result, respectively. Substitute the first prediction error and the second prediction error into the target loss function to obtain the actual loss function value; The gradient values ​​are obtained by backpropagating the model parameters and the joint weight parameters using the actual loss function values. Update the model parameters and the joint weight parameters based on the gradient value; Continue training the model using the updated model parameters and joint weight parameters until the target loss function converges to obtain the preset model.

5. The method according to claim 1, characterized in that, The step of dynamically optimizing the joint weight parameters of the preset model using the distance value includes: Obtain the initial weight value of each joint from the joint weight parameters; The distance value is compared with a first threshold and a second threshold respectively, and the initial weight value of each joint is optimized based on the comparison result, wherein the first threshold is greater than the second threshold.

6. The method according to claim 5, characterized in that, The step of optimizing the initial weight values ​​of each joint based on the comparison results includes: If the distance value is greater than or equal to the first threshold, then the initial weight value of the base joint is increased to the first target value, and the initial weight value of the wrist joint is decreased to the second target value; or If the distance value is less than the first threshold and greater than the second threshold, then the initial weight value of the base joint is adjusted to a third target value, and the initial weight value of the wrist joint is adjusted to a fourth target value, wherein the third target value is less than the first target value, and the fourth target value is greater than the second target value; or If the distance value is less than or equal to the second threshold, the initial weight value of the base joint is reduced to the fifth target value, and the initial weight value of the wrist joint is increased to the sixth target value, wherein the fifth target value is less than the third target value, and the sixth target value is greater than the fourth target value.

7. An optimized control device for a robotic arm, characterized in that, include: The acquisition module is used to acquire a preset model and acquire the distance value between the robot and the first elevator button in the current scene. The training data of the preset model is a dataset generated by the interaction between the robot and the elevator control system. An optimization module is used to dynamically optimize the joint weight parameters of the preset model using the distance value to obtain the optimized preset model, wherein the joint weight parameters are used to characterize the importance of different robotic arm joints of the robot in motion prediction; The control module is used to control the robot's robotic arm using the optimized preset model, so that when each joint of the robot's robotic arm performs a first pressing action according to the first action parameters output by the optimized preset model, it triggers the first elevator button. The training module is used to train the preset model in the following manner before obtaining the preset model: obtaining the dataset generated by the interaction between the robot and the elevator control system; adding the joint weight parameters to the initial loss function of the initial model to obtain the target loss function; and iteratively training the initial model using the target loss function and the dataset to obtain the preset model. The acquisition module is further configured to acquire a region image of the first elevator button, and acquire the current joint information of the robotic arm and the posture information of the end effector, wherein the current joint information is used to characterize the current internal configuration state of the robotic arm; to extract features from the region image, the current joint information and the posture information respectively to obtain a first feature, a second feature and a third feature; to fuse the first feature, the second feature and the third feature to obtain a fused feature vector; to compress the fused feature vector to map the fused feature vector into a scalar; and to determine the scalar as the distance value between the end effector of the robot and the first elevator button.

8. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program executable on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method of any one of claims 1 to 6.

Images