A charging mechanical arm control method and device and electronic equipment

Abstract

This invention discloses a charging robotic arm control method, device, and electronic device. The method determines the pose information of the target object based on a first image of the target object; based on the pose information, it determines the target pose of the robotic arm's end effector, including a first target pose for the end effector to acquire the charging gun and a second target pose for the end effector to place the charging gun; based on the target pose, the current pose of the end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm, it obtains a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint; it sends each joint motion trajectory in the set of joint motion trajectories to the corresponding joint on the robotic arm, so that each joint moves according to the corresponding joint motion trajectory to achieve the insertion and removal of the charging gun, improving the coordination and accuracy of the robotic arm's movement, thereby ensuring the reliability and precision of the automatic charging operation.

Description

Technical Field

[0001] This invention relates to the field of charging control technology, and in particular to a charging robotic arm control method, device, and electronic device. Background Technology

[0002] In the field of automatic vehicle charging, vision-guided robotic arm charging gun plug-in / plug-out control has become the mainstream development direction.

[0003] Existing automatic charging equipment mostly employs a six-axis robotic arm, with control achieved through independent planning of the gantry translation axis and the six-axis robotic arm itself. However, this approach results in limited mobility for the robotic arm, and due to the lack of redundant degrees of freedom, it sometimes struggles to reach all charging port locations in complex environments, such as those with spatial obstacles, exhibiting limited obstacle avoidance capabilities. Furthermore, existing solutions typically use random sampling algorithms for motion path planning; however, the resulting paths are highly random and cannot meet the high-precision requirements of point-to-point, repeatable, and accurate joint movement in charging tasks. In practical use, this often leads to issues such as deviations from expected intermediate paths and end-point joint drift, resulting in defects like improper charging gun insertion / removal and poor contact, thus failing to meet the high precision and reliability requirements of automatic charging. Summary of the Invention

[0004] This invention provides a control method, device, and electronic device for a charging robotic arm to improve the operational flexibility, accuracy, and reliability of the robotic arm during charging and plugging operations.

[0005] In a first aspect, embodiments of the present invention provide a method for controlling a charging robotic arm, the method comprising: Based on the first image of the target object, the pose information of the target object is determined, and the target object is associated with the current operating stage of the charging gun; Based on the pose information, the target pose of the robotic arm end effector is determined. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. Based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the motion trajectories of each joint is obtained. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint. Each joint motion trajectory in the set of joint motion trajectories is sent to the corresponding joint on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

[0006] In a second aspect, embodiments of the present invention provide a charging robotic arm control device, the device comprising: The pose information determination module is used to determine the pose information of the target object based on the first image of the target object, wherein the target object is associated with the current operating stage of the charging gun. The target pose determination module is used to determine the target pose of the robotic arm end effector based on the pose information. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. The trajectory determination module is used to obtain a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, combined with the unified calculation of all joints of the robotic arm. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint. The trajectory sending module is used to send each of the joint motion trajectories in the set of joint motion trajectories to the corresponding joint on the robotic arm, so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

[0007] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the charging robotic arm control method according to any embodiment of the present invention.

[0008] The technical solution of this invention involves determining the pose information of the target object based on a first image of the target object, with the target object associated with the current operating stage of the charging gun; determining the target pose of the robotic arm end effector based on the pose information, the target pose including a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun; obtaining a set of joint motion trajectories containing the corresponding joint motion trajectories based on the target pose, the current pose of the robotic arm end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm, the robotic arm includes 6 robotic arm body rotation joints and 1 truss translational joint; and sending each joint motion trajectory in the set of joint motion trajectories to the corresponding joint on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to achieve the insertion and removal of the charging gun. This method dynamically adjusts the target object's range by associating it with the current operating stage of the charging gun. It also improves the targeting accuracy of pose calculation by combining the target object's first image with the target object's pose information. Furthermore, it is adaptable to scenarios where multiple charging guns are configured in a charging station and a truss-suspended robotic arm is used for automatic plugging and unplugging. Based on the pose information, the target pose of the robotic arm's end effector is determined, providing a clear and accurate target reference for subsequent trajectory planning of each joint of the robotic arm. Finally, based on the initial pose, final pose, and joint information, the robotic arm's rotational joints and the truss translational joints are treated as a whole. Synchronous calculation improves the matching degree of motion trajectories of each joint, avoiding problems such as trajectory jitter or endpoint deviation caused by asynchronous movement between rotary and translational joints. This enhances the coordination, smoothness, and flexibility of the robotic arm's movement. Furthermore, by generating a unified set of joint motion trajectories through calculation, it ensures that the trajectories are repeatable and drift-free, meeting the high-precision requirements of point-to-point, repeatability, and accurate joint movement for charging and plugging. By distributing the trajectories of each joint in the set of joint motion trajectories to the corresponding joints, the coordinated movement of each joint is driven to achieve the plugging and unplugging of the charging gun, ensuring the smooth completion of automatic charging operations and improving operational reliability and accuracy.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of a charging robotic arm control method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a charging robotic arm control device provided in an embodiment of the present invention; Figure 3 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] It is understood that before using the technical methods disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0015] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as the electronic device, application, server, or storage medium performing the operations of this disclosed technology.

[0016] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0017] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0018] Specifically, embodiments of the present invention provide a control method for a charging robotic arm. Figure 1 This is a flowchart of a charging robotic arm control method provided by an embodiment of the present invention. The embodiment of the present invention is applicable to scenarios where the robotic arm is controlled to automatically charge, especially to scenarios where the robotic arm is controlled to automatically plug and unplug the charging gun to charge the vehicle to be charged. The method can be executed by a charging robotic arm control device, which can be implemented in the form of software and / or hardware, or optionally by an electronic device, preferably a mobile terminal, desktop computer, laptop computer, or server.

[0019] like Figure 1 As shown, the charging robotic arm control method provided in this embodiment of the invention may specifically include: S101. Based on the first image of the target object, determine the pose information of the target object, and associate the target object with the current operation stage of the charging gun.

[0020] The target object can be understood as the object that needs to be identified and located during the robotic arm's operation, providing a basis for determining the pose the robotic arm needs to reach to complete the automatic charging task. The first image can be considered as an image containing the appearance features and position information of the target object, used to provide data support for determining the pose information of the target object. For example, the first image can be acquired in real time by a camera; optionally, the camera can be deployed at the end of the robotic arm and is a depth camera. The pose information can include the position information and attitude information of the target object in a preset coordinate system. The position information can be used to characterize the three-dimensional coordinates of the target object in space, and the attitude information can be used to characterize the placement angle of the target object in space (such as the rotation angle around the horizontal axis, vertical axis, and depth axis). Combining the two can accurately determine the specific state of the target object in space. The current operation stage of the charging gun can be considered as the specific operation step performed by the robotic arm on the charging gun. For example, the current operation phase can be subdivided into a pickup operation phase, an insertion operation phase, an extraction operation phase, and a return operation phase; the current operation phase can also include a gun retrieval operation phase and a return operation phase, with the gun retrieval operation phase including the pickup operation phase and the insertion operation phase, and the return operation phase including the extraction operation phase and the return operation phase.

[0021] Understandably, to enable applications that charge multiple vehicles, this method targets a gantry-suspended robotic arm for completing vehicle charging tasks. The robotic arm's end effector does not carry charging guns, but there can be multiple charging guns, each placed in a pre-defined location. The robotic arm can move along the gantry to reach a specific charging gun's location to retrieve or return it to its original position.

[0022] Therefore, in this embodiment, the target object can be determined based on the current operation stage of the charging gun. When in different operation stages, the range of the target object will be adjusted accordingly based on the task to be performed by the end of the robotic arm corresponding to the current operation stage.

[0023] For example, if the current operation phase includes a gun retrieval phase, when in the gun retrieval phase, the tasks that the robotic arm end effector needs to perform include going to the preset placement position of the charging gun to pick up the charging gun and plugging the charging gun into the charging port of the vehicle to be charged. Therefore, the poses that the robotic arm needs to reach include the poses that can accurately pick up the charging gun and the poses that can plug the charging gun into the charging port of the vehicle to be charged. In order to accurately determine these two poses, the target objects may include the vehicle charging port and the charging gun.

[0024] For example, if the current operation phase includes a plugging operation phase, when in the plugging operation phase, the task that the end effector of the robotic arm needs to perform is to plug the picked-up charging gun into the charging port of the vehicle to be charged. At this time, the pose that the robotic arm needs to reach is a pose that can plug the charging gun into the charging port of the vehicle to be charged. Therefore, the target object can be only the charging port.

[0025] In this embodiment, after determining the target object in response to a trigger event, a first image of the target object can be obtained. The pose information of the target object can be determined based on the first image by: performing target detection and keypoint extraction on the first image of the target object to obtain the feature points corresponding to the target object; and then performing pose estimation based on the feature points to determine the pose information of the target object.

[0026] S102. Based on the pose information, determine the target pose of the robotic arm end effector. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun.

[0027] The end effector of the robotic arm can be understood as the component at the very end of the robotic arm used to grasp and manipulate the charging gun. It is the part that directly contacts the charging gun, and its motion directly determines the operational accuracy of the charging gun. It can realize actions such as picking up, inserting, removing, and placing the charging gun. The first target pose can be considered as the target pose used by the end effector of the robotic arm to acquire (pick up or remove) the charging gun, and it has different meanings depending on the stage of the operation. The second target pose can be understood as the target pose used by the end effector of the robotic arm to place (insert or return) the charging gun, and it also has different meanings depending on the stage of the operation.

[0028] For example, during the gun retrieval phase, the first target pose is the pose where the robotic arm end effector reaches the charging gun placement position and can accurately pick up the charging gun; the second target pose is the pose where the robotic arm end effector carries the charging gun and can accurately plug the charging gun into the vehicle's charging port. During the return phase, the first target pose is the pose where the robotic arm end effector reaches the vehicle's charging port and can accurately unplug the charging gun; the second target pose is the pose where the robotic arm end effector carries the charging gun and can put the charging gun back into the preset placement position.

[0029] In this embodiment, the method for determining the target pose of the robotic arm end effector based on pose information can be as follows: the target pose of the robotic arm end effector is determined by combining the pose information with a coordinate transformation chain and then using coordinate transformation to determine the target pose of the robotic arm end effector. Alternatively, the method can be as follows: the first target pose of the robotic arm end effector is determined by combining the pose information with a coordinate transformation chain and then using a preset charging gun return pose as the second target pose of the robotic arm end effector.

[0030] S103. Based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint.

[0031] The current pose of the robotic arm's end effector can be considered as its initial spatial position and orientation relative to the target pose, serving as the starting reference for trajectory planning. For example, when the target pose is a first target pose, the actual spatial pose of the robotic arm's end effector can be used as the current pose relative to the target pose, which can be obtained in real-time through the robotic arm's own sensors. When the target pose is a second target pose, the first target pose can be directly used as the current pose relative to that target pose. The joint information of the robotic arm includes joint types (such as rotary joints of the robotic arm body and translational joints of the gantry) and the number of joints of each type. The joint information may also include attribute information such as the length and range of rotation of each joint.

[0032] In this embodiment, six robotic arm body rotary joints and one truss translational joint work together to form a 7-axis redundant robotic arm, which can expand the working space of the robotic arm, realize long-stroke (such as 11 meters) and multi-vehicle charging operations, and ensure that the robotic arm can reach charging guns and vehicle-end charging ports at different locations.

[0033] In this embodiment, the method for obtaining a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint, based on the target pose, the current pose of the robotic arm's end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm, can be as follows: Based on the target pose and the joint information of the robotic arm, and combined with a unified calculation of all joints of the robotic arm, obtain the set of joint angles corresponding to the target pose; obtain the set of joint angles of the robotic arm's end effector in the current pose; based on the above two sets of joint angles, generate a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint through appropriate interpolation. Optionally, the set of joint angles corresponding to the pose can be represented in vector form.

[0034] In this embodiment, the method of obtaining a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint based on the target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm, combined with the unified calculation of all joints of the robotic arm, can also be as follows: Based on the target pose and the current pose, a pose sequence is generated by reasonable interpolation; for each pose in the pose sequence, based on the pose and the joint information of the robotic arm, combined with the unified calculation of all joints of the robotic arm, a set of joint angles corresponding to the pose is obtained, and a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is generated based on the set of joint angles.

[0035] S104. Send the joint motion trajectory of each joint in the set of joint motion trajectories to the corresponding joint on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

[0036] This step can be considered as sending the planned joint motion trajectory to each joint for execution, converting the trajectory planning result into the actual movement of the robotic arm, and finally completing the plugging and unplugging of the charging gun to achieve the core function of automatic charging.

[0037] In this embodiment, the joint motion trajectory corresponding to each joint in the generated deterministic set of joint motion trajectories is transmitted to the drive mechanism (such as a motor) of that joint. After receiving the corresponding trajectory command, the drive mechanism of each joint drives the joint to complete the corresponding movement according to the joint angle in the trajectory. Through the coordinated movement of all joints, the end effector of the robotic arm moves accurately and stably from the current pose to the target pose, ensuring the reliability and accuracy of the end effector of the robotic arm in performing the corresponding operation. This enables precise and smooth insertion and removal of the charging gun, ensuring the smooth completion of the automatic charging operation.

[0038] The charging robotic arm control method provided in this embodiment of the invention determines the pose information of the target object based on a first image of the target object, and the target object is associated with the current operation stage of the charging gun; based on the pose information, the target pose of the robotic arm end effector is determined, the target pose including a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun; based on the target pose, the current pose of the robotic arm end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained, the robotic arm including 6 robotic arm body rotation joints and 1 truss translational joint; each joint motion trajectory in the set of joint motion trajectories is sent to the corresponding joint on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to realize the insertion and removal of the charging gun. This method dynamically adjusts the target object's range by associating it with the current operating stage of the charging gun. It also improves the targeting accuracy of pose calculation by combining the target object's first image with the target object's pose information. Furthermore, it is adaptable to scenarios where multiple charging guns are configured in a charging station and a truss-suspended robotic arm is used for automatic plugging and unplugging. Based on the pose information, the target pose of the robotic arm's end effector is determined, providing a clear and accurate target reference for subsequent trajectory planning of each joint of the robotic arm. Finally, based on the initial pose, final pose, and joint information, the robotic arm's rotational joints and the truss translational joints are treated as a whole. Synchronous calculation improves the matching degree of motion trajectories of each joint, avoiding problems such as trajectory jitter or endpoint deviation caused by asynchronous movement between rotary and translational joints. This enhances the coordination, smoothness, and flexibility of the robotic arm's movement. Furthermore, by generating a unified set of joint motion trajectories through calculation, it ensures that the trajectories are repeatable and drift-free, meeting the high-precision requirements of point-to-point, repeatability, and accurate joint movement for charging and plugging. By distributing the trajectories of each joint in the set of joint motion trajectories to the corresponding joints, the coordinated movement of each joint is driven to achieve the plugging and unplugging of the charging gun, ensuring the smooth completion of automatic charging operations and improving operational reliability and accuracy.

[0039] As a first optional embodiment of the present invention, based on the above embodiments, before determining the pose information of the target object according to the first image of the target object, the following optimization can be made: Detect the current operating stage of the charging gun; When the current operation stage is the gun retrieval operation stage, the target object is determined to include the vehicle-end charging port and the charging gun of the vehicle to be charged, and the first image of the vehicle-end charging port and the first image of the charging gun are obtained. The gun retrieval operation stage represents the operation stage in which the end effector of the robotic arm goes to the preset placement position of the charging gun to pick up the charging gun and plugs the charging gun into the corresponding vehicle-end charging port. When the current operation stage is the return operation stage, the target object is determined to be the charging gun, and the first image of the charging gun is acquired. The return operation stage represents the operation stage in which the end effector of the robotic arm pulls the charging gun out of the charging port at the vehicle end and puts the charging gun back into the preset placement position. The first image was acquired by a camera mounted on the end of a robotic arm.

[0040] This optional embodiment can be considered as a detection and target object matching process before the operation phase. In this embodiment, the current operation phase of the charging gun corresponding to the preset trigger operation can be detected in response to a preset trigger operation for initiating the detection of the current operation phase. For example, the preset trigger operation can be a corresponding instruction issued by the vehicle or charging system (such as an instruction to start charging or an instruction to end charging), or the preset trigger operation can be a trigger signal generated when the vehicle to be charged is parked in place. Optionally, the current operation phase of the charging gun can be determined by acquiring and parsing the upper-level charging control task instruction, detecting the electrical connection status between the charging gun and the vehicle-end charging port, and combining at least one of the following: the clamping status of the charging gun by the robotic arm end effector and the placement status of the charging gun.

[0041] For example, when a charging start command is received from the vehicle or charging system, or when it is detected that the charging gun has not established an electrical connection with the vehicle-end charging port, and the end effector of the robotic arm is not holding the charging gun and the charging gun is in a preset placement position, the current operation stage is determined to be the gun retrieval operation stage; when a charging end power-off command is received from the vehicle or charging system, or when it is detected that the vehicle charging process has been completed, the charging gun has disconnected from the vehicle-end charging port, and the charging gun is located at the vehicle-end charging port position (or the charging gun is not in a preset, fixed placement position), the current operation stage is determined to be the return operation stage.

[0042] In this embodiment, when the current operation stage is determined to be the charging gun retrieval stage, this stage indicates that the robotic arm end effector needs to go to the corresponding preset placement position (fixed workstation) of the charging gun to pick up the charging gun and transfer it to the vehicle-side charging port of the vehicle to be charged. Correspondingly, this stage needs to simultaneously identify the vehicle-side charging port and the charging gun. Therefore, the target objects are determined to include the vehicle-side charging port and the charging gun of the vehicle to be charged, and the first image of the vehicle-side charging port and the first image of the charging gun are acquired respectively.

[0043] In this embodiment, when the current operation stage is determined to be the homing operation stage, since this homing operation stage represents the robotic arm end effector removing the charging gun from the vehicle-end charging port and moving the charging gun back to the preset placement position, and since the preset placement position corresponding to each charging gun is fixed, the appropriate and universal posture of the robotic arm end effector when placing the charging gun back at the corresponding preset placement position can be directly determined in advance. Therefore, this stage only needs to identify and locate the charging gun that has established a connection with the vehicle-end charging port, so the target object is determined to be the charging gun, and the first image of the charging gun is obtained.

[0044] It is understandable that the first image corresponding to the vehicle-mounted charging port and the first image corresponding to the charging gun can be acquired in real time by cameras deployed on the end effector of the robotic arm. The cameras move synchronously with the end effector of the robotic arm, enabling the acquisition of clear appearance features of the target object and its position relative to the end effector of the robotic arm from a perspective close to the operation. This provides reliable image data for subsequent accurate calculation of the target pose and planning of effective joint motion trajectories. At least two cameras are deployed on the end effector of the robotic arm to meet the acquisition requirements.

[0045] The technical solution described in this embodiment determines the current operating stage of the charging gun and, based on the different task requirements of that stage, differentiates the target object to be identified and located, and acquires the corresponding first image. This reduces invalid identification and redundant calculations, and accurately locks the target to be identified in accordance with the actual task of the robotic arm, making visual acquisition and pose calculation more consistent with the insertion and removal operation process logic. Simultaneously, using an end-effector camera to uniformly acquire images ensures that the imaging viewpoint is consistent with the operating viewpoint, effectively improving the accuracy and effectiveness of subsequent target pose calculation. This makes the entire process of the robotic arm's end effector picking up, inserting, removing, and returning the charging gun smoother, thus improving the overall intelligence and operational stability of the automatic charging operation.

[0046] As a second optional embodiment of the present invention, based on the above embodiments, determining the pose information of the target object according to the first image of the target object can be specified as the following steps: Based on the first image, determine the feature points corresponding to the target object; For the target object of the vehicle-side charging port, the feature points are filtered to obtain effective points, wherein the feature points are used to characterize the boundaries of each socket in the vehicle-side charging port. Based on the effective points and the geometric features of each of the sockets, the pose information of the target object is determined by a pose estimation algorithm; For a target object that is a charging gun, the pose information of the target object is determined by an identifier pose detection algorithm based on the size and angle information of the feature points and the preset identifier code on the charging gun. The feature points are used to characterize the boundary of the preset identifier code on the charging gun.

[0047] In this context, feature points can be understood as pixels detected and extracted from the first image that characterize key positions such as the outline, corners, and marker edges of the target object. Effective points can be considered as usable feature points that accurately reflect the outline and boundary of the charging port, retained after filtering out interference points, noise points, and invalid edge points from all feature points corresponding to the vehicle-side charging port. The socket can be considered as the standard hole structure on the vehicle-side charging port used for mating and plugging with the charging gun plug. It has fixed external dimensions and spatial geometry, and is the core structural basis for identifying the position and orientation of the charging port. For example, the number of sockets in a vehicle-side charging port is usually three, in a circular shape, with different radii and no overlap. The geometric features of the socket can be understood as the inherent geometric parameters such as the inherent external dimensions and relative spacing of the socket.

[0048] A preset identification code can be understood as a special identification mark with a fixed shape and pattern that is pre-printed and attached to the surface of the charging gun. It has unique identifiability and serves as a dedicated reference object for visually locating the charging gun. The size and angle information of the preset identification code can be understood as the inherent physical length and width dimensions, border ratio, and pre-known parameters such as the preset identification code's deflection angle.

[0049] In this embodiment, as one implementation method, determining the feature points corresponding to the target object based on the first image can be achieved by: segmenting the target object's mask and / or centroid using a trained target detection and segmentation model, such as the YOLO model; generating multiple corresponding points based on the segmented mask and / or centroid to depict the geometric contour of the target object, and determining these corresponding points as the feature points corresponding to the target object. The method for filtering the feature points to obtain valid points can be achieved by: using the Random Sampling Consensus (RANSAC) algorithm based on a threshold to remove outliers from the feature points, retaining only inliers, and determining the retained inliers as valid points. Based on the valid points and the geometric features of each of the sockets, the pose information of the target object can be determined by a pose estimation algorithm as follows: using a pose estimation algorithm (such as the PnP algorithm), based on the two-dimensional pixels of the valid points in the first image and the geometric features of each socket, the spatial transformation relationship between the camera coordinate system and the target object coordinate system is solved; then, based on the spatial transformation relationship and the two-dimensional pixels of the valid points in the first image, the position information (three-dimensional coordinates) and attitude information (rotation angles around the horizontal axis, vertical axis, and depth axis) of the target object (vehicle charging port) in the camera coordinate system are deduced.

[0050] In this embodiment, since the preset identification code is fixed to the surface of the charging gun, its pose is consistent with the overall pose of the charging gun. Therefore, only the pose information of the preset identification code needs to be determined to obtain the pose information of the target object (charging gun), without the need for additional feature extraction of the entire charging gun, thus improving the pose calculation efficiency. The pose information of the charging gun can be the pose information of the parts / areas on the charging gun that can be grasped by the end effector of the robotic arm.

[0051] In an optional embodiment, the preset identification code can be represented as a QR code, and the identification code pose detection algorithm can be a QR code pose detection algorithm, that is, a dedicated visual pose detection algorithm that uses QR codes as exclusive identification markers. Specifically, the feature points can be the boundary feature points of the QR code (such as the pixels corresponding to the four corner points of the QR code), and combined with the preset actual size and angle information of the QR code, the pose information of the QR code in the camera coordinate system is solved through perspective projection transformation and spatial coordinate conversion, thereby indirectly obtaining the pose information of the target object.

[0052] The above-described technical solution in this embodiment uses differentiated algorithms to determine the pose information of the target object based on the structural characteristics of the two types of target objects, thereby improving the accuracy and reliability of pose information estimation.

[0053] As a third optional embodiment of the present invention, based on the above embodiments, the step of determining the target pose of the robotic arm end effector according to the pose information can be optimized as follows: When the target object includes a vehicle-end charging port and a charging gun, the first target pose of the robotic arm end effector is determined by coordinate transformation based on the pose information of the charging gun; the second target pose is determined by coordinate transformation based on the pose information of the vehicle-end charging port. When the target object is a charging gun, the first target pose is determined by coordinate transformation based on the pose information of the charging gun; a preset charging gun return pose is obtained, and the charging gun return pose is used as the second target pose.

[0054] In this embodiment, based on different target objects at different work stages, the first target pose and the second target pose of the robotic arm end effector are determined respectively to ensure that the target pose accurately matches the current work stage and task, providing a clear and reliable endpoint reference for subsequent joint motion trajectory planning. Specifically, this includes a gun retrieval operation stage where the target object includes both the vehicle-side charging port and the charging gun, and a return operation stage where the target object is only the charging gun. In the gun retrieval operation stage, the robotic arm end effector needs to complete the complete task of picking up the charging gun and inserting it into the vehicle-side charging port. Therefore, it is necessary to determine the intermediate pose of the robotic arm end effector for picking up the charging gun, i.e., the first target pose, based on the pose information of the charging gun, and the final pose of the robotic arm end effector for inserting the charging gun into the vehicle-side charging port, i.e., the second target pose, based on the pose information of the vehicle-side charging port, to ensure accurate docking between the charging gun and the vehicle-side charging port. Optionally, the first target pose can be preset and is the same as the preset charging gun return pose. Correspondingly, the first target pose can be directly obtained from the storage location.

[0055] During the repositioning phase, the robotic arm end effector needs to remove the charging gun from the vehicle-end charging port and place it back into the preset position. Since the preset position is fixed, the repositioning pose of the robotic arm end effector can be determined in advance for each charging gun based on its preset position. This pose can be directly obtained from the storage location and used as the second target pose.

[0056] Optionally, the preset charging gun return posture can be determined by: acquiring a first target posture of the charging gun determined during the gun retrieval phase, and determining the first target posture as the preset charging gun return posture. Optionally, the first target posture can also be determined during the return phase by: acquiring a second target posture of the charging gun determined during the gun retrieval phase, and determining the second target posture as the first target posture of the charging gun during the return phase.

[0057] Understandably, since the pose information of the charging gun is a visual pose relative to the current temporary position of the camera or the end effector of the robotic arm, the reference changes when the robotic arm moves, rendering the pose information unusable. Therefore, in this embodiment, the visual pose can be normalized to a fixed global reference, i.e., the global coordinate system corresponding to the fixed truss base, or the world coordinate system, through coordinate transformation.

[0058] As one implementation method, the coordinate transformation is based on a coordinate transformation chain; The coordinate transformation chain includes: the pose transformation matrix of the end flange of the robotic arm relative to the fixed truss base, the pose transformation matrix of the camera deployed at the end of the robotic arm relative to the end flange of the robotic arm, the pose information of the target object, and the pose transformation matrix of the end actuator of the robotic arm relative to the end flange of the robotic arm.

[0059] The coordinate transformation chain can be understood as a link composed of pose transformation matrices between multiple different coordinate systems connected in a fixed order. It is used to realize the step-by-step conversion of the target object's pose between different coordinate systems, and finally transform the target pose obtained by vision calculation (in the camera coordinate system) into a target pose (in the global coordinate system) that can be recognized and executed by the end effector of the robotic arm.

[0060] For example, the coordinate transformation chain can be specifically represented as: ;in, This is the pose transformation matrix of the robotic arm end effector relative to the fixed truss base; This is the pose transformation matrix of the end flange of the robotic arm relative to the fixed truss base; The pose transformation matrix of the camera positioned at the end of the robotic arm relative to the end flange of the robotic arm; The pose information of the target object is the pose transformation matrix of the target object relative to the camera deployed at the end of the robotic arm. This is the pose transformation matrix of the robotic arm end effector relative to the robotic arm end flange, representing the safe distance that the charging gun head or the robotic arm end effector needs to travel. Optionally, the pose transformation matrix of the robotic arm end flange relative to the fixed truss base... It can be determined based on the current rotation angle of each axis on the robotic arm combined with a forward kinematics algorithm; The pose transformation matrix of the robotic arm end effector relative to the robotic arm end flange can be obtained based on camera hand-eye calibration. It can be obtained based on TCP calibration.

[0061] The above-described technical solution in this embodiment provides a method based on coordinate transformation chain to transform the temporary pose calculated by vision into the target pose under a global fixed reference, so that the obtained target pose is not affected by the movement of the robotic arm during the process, laying the foundation for the accurate determination of the subsequent joint motion trajectory.

[0062] As a fourth optional embodiment of the present invention, based on the above embodiments, the method of obtaining a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint by combining the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm with a unified calculation of all joints of the robotic arm can be specifically optimized as follows: Based on the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a first target joint angle set for achieving the first target pose is determined. The first target joint angle set includes the first target joint angle of each joint on the robotic arm. Based on the first target pose, the second target pose, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of second target joint angles for achieving the second target pose is determined. The set of second target joint angles includes the second target joint angles of each joint on the robotic arm. Based on the current joint angle set corresponding to the current pose, the first target joint angle set, and the second target joint angle set, and combined with joint space interpolation, a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint is obtained.

[0063] The first target joint angle set can be understood as the set of first target joint angles required for the end effector of the robotic arm to reach the first target pose in the coordinated operation of all joints (6 rotary joints + 1 gantry translational joint), ensuring that the end effector accurately reaches the pose for picking up or removing the charging gun. The first target joint angle can be considered as the target angle corresponding to a single joint on the robotic arm, that is, the specific angle value that a joint needs to rotate or move to in order to cooperate with the end effector to reach the first target pose. The second target joint angle set can be understood as the set of second target angles required for the end effector to reach the second target pose in the coordinated operation of all joints on the robotic arm, ensuring that the end effector accurately reaches the pose for inserting or returning (resetting) the charging gun. The second target joint angle set includes the second target joint angle corresponding to each joint on the robotic arm, and together with the first target joint angle set, constitutes the joint angle reference for the complete operation process. The second target joint angle can be considered as the target angle corresponding to a single joint in the set of second target joint angles. That is, it is the specific angle value that a certain joint needs to rotate or move to in order to cooperate with the end effector to reach the second target pose. It is the angle reference for a single joint to continue moving after completing the first stage of action. The current joint angle set can be understood as the set of angles of each joint when the end effector of the robotic arm is in the current pose. It includes the real-time angle value of each joint and is the starting angle reference for joint motion trajectory planning. It can be obtained in real time by the robotic arm's own sensors.

[0064] In this embodiment, redundant inverse kinematics, using the Jacobi pseudo-inverse method and null space optimization, can be used to achieve a unified solution for all joints of the robotic arm, determining the corresponding target joint angle set. Specifically, this can be expressed as: determining a 7-dimensional joint angle vector based on the joint information of the robotic arm. , ;in, For truss translational joints; to This refers to the rotary joint of the robotic arm body.

[0065] Alternatively, the Jacobi pseudo-inverse method combined with the null space optimization method can be used to solve for the 7-dimensional joint angle vector. The method can be expressed as: ;in, and For the first The 7-dimensional joint angle vector of the next iteration; Step size factor; for The abbreviation for Jacobian matrix. ; For the first The pose error matrix of the robotic arm's end effector in the next iteration. ; It is the identity matrix; This is a damped least squares pseudo-inverse. , This is a damping factor used to avoid singularities; Optimize the direction vector for null space; The zero-space projection matrix; This is a null-space optimization term used to optimize redundancy metrics (secondary objectives). Preferences can be customized without affecting the primary objective. For example, the null-space optimization direction vector can be represented as: ;in, For a joint to be far from its limit, it can be represented as: , Angles that are preferred for the center of all axes (joints); Alternatively, the first axis can be centered, which can be represented as: , The preferred position of the center of the truss translation axis (truss translation joint); Alternatively, to avoid singularities, it can be represented as: .

[0066] In this embodiment, the method for obtaining a set of joint motion trajectories containing the motion trajectories of each joint based on the current joint angle set corresponding to the current pose, the first target joint angle set, and the second target joint angle set, combined with joint space interpolation, can be as follows: A first motion trajectory set is generated by constructing several intermediate landmarks between the current joint angle set and the first target joint angle set through joint space interpolation; a second motion trajectory set is generated by constructing several intermediate landmarks between the first target joint angle set and the second target joint angle set through joint space interpolation; and the two sets are then integrated to obtain the final set of joint motion trajectories, which contains the continuous and smooth joint motion trajectories corresponding to each joint. This set of joint motion trajectories can guide each joint of the robotic arm to move smoothly from the current joint angle to the first target joint angle, and then continue to the second target joint angle, achieving continuous movement throughout the entire operation.

[0067] For example, the method of constructing several intermediate landmarks between the current set of joint angles and the first target set of joint angles through joint space interpolation can be represented as follows: ; in, For the first A middle road sign; ; , This represents the total number of intermediate road signs; A joint angle vector representing the current set of joint angles; The joint angle vector represents the set of joint angles of the first target.

[0068] Understandably, when generating a set of joint motion trajectories, the electronic device will assign a unique annotation (such as adding an identifier or priority marker) to the first target joint angle set. This annotation information is then sent to the robotic arm synchronously along with the joint motion trajectory. After receiving the corresponding joint motion trajectory, each joint of the robotic arm will identify the annotation information and determine which joint angle set corresponds to the first target pose. When each joint of the robotic arm moves to the first target joint angle corresponding to the annotation, it will precisely control the end effector of the robotic arm to perform the corresponding operation (picking up the charging gun during the gun retrieval phase and unplugging the charging gun during the return phase). After this operation is completed, each joint of the robotic arm will continue to run along the planned joint motion trajectory until it reaches the joint angle corresponding to the second target joint angle set, thereby completing the corresponding operation at that angle (plugging the charging gun into the vehicle's charging port during the gun retrieval phase and placing the charging gun back into the preset position during the return phase). By using a collaborative annotation and action triggering mechanism between the electronic device and the robotic arm, the action node corresponding to the first target joint angle is clearly defined, ensuring that the robotic arm can perform pick-up / pick-up operations at precise angle positions before continuing to complete subsequent actions. This further improves the orderliness, accuracy, and reliability of the robotic arm's operations, adapting to the high-precision operation requirements of charging gun plugging and unplugging.

[0069] The above-described technical solution in this embodiment improves the accuracy and controllability of joint motion trajectories by determining the target joint angle set step by step and generating a motion trajectory set by combining joint angle interpolation, thus ensuring that the robotic arm can complete actions such as picking up / removing, inserting / returning guns in sequence.

[0070] As a fifth optional embodiment of the present invention, based on the above embodiments, the step of obtaining a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint by combining the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm with a unified calculation of all joints of the robotic arm can be further optimized as follows: The first pose sequence is generated by combining the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm with position linear interpolation and attitude spherical interpolation. A second pose sequence is generated based on the first target pose, the second target pose, and the joint information of the robotic arm, combined with position linear interpolation and attitude spherical interpolation. Based on the first pose sequence and the second pose sequence, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained.

[0071] In this embodiment, a first pose sequence is obtained by integrating the current pose, each intermediate pose, and the first target pose, which are generated by an interpolation algorithm from the current pose of the robotic arm end effector to the first target pose. Similarly, a second pose sequence is obtained by integrating the first target pose, each intermediate pose, and the second target pose, which are generated by an interpolation algorithm from the first target pose of the robotic arm end effector to the second target pose.

[0072] In this embodiment, the pose can be represented by seven numbers: three numbers representing the position in three-dimensional space and a quaternion representing the orientation (rotation) in three-dimensional space. For example, the position information and orientation information in the pose can be obtained separately, linear interpolation of the position information can be performed based on the position information, and spherical interpolation of the orientation information (quaternion spherical interpolation) can be performed based on the orientation information.

[0073] For example, a method for performing linear position interpolation based on location information can be represented as: ;in, These are interpolation parameters. ; The starting position; The target location.

[0074] For example, the method of attitude spherical interpolation based on quaternions can be expressed as: ;in, The initial attitude (quaternion); For the target pose (quaternion); The angle between two quaternions. .

[0075] In this embodiment, by integrating the first pose sequence and the second pose sequence, and using the integrated complete pose sequence as a basis, and then through methods such as redundant inverse kinematics or Cartesian path tracking, all joints of the robotic arm are uniformly calculated to determine the joint motion angles of each joint on the robotic arm corresponding to each pose. This generates continuous motion trajectories for each joint, and finally integrates them to form a set of joint motion trajectories containing the motion trajectories of all joints. This set of trajectories can guide the coordinated movement of the robotic arm joints, enabling the end effector to move continuously and smoothly from the current pose to the first target pose and then to the second target pose. This ensures a smooth and continuous workflow and improves the accuracy of the endpoint joints.

[0076] The above-described technical solution in this embodiment achieves path constraint by interpolating between poses, and simultaneously constrains the accuracy of the path and the joint speed, making the joint motion path and speed smoother and more reliable.

[0077] As a sixth optional embodiment of the present invention, based on the above embodiments, further optimizations can be made including: Obtain the scaling factor corresponding to the translational joint of the truss and the preset first speed limit information, wherein the scaling factor includes a linear velocity scaling factor and / or an acceleration scaling factor; Based on the first speed limit information and the scaling factor, the second speed limit information corresponding to the truss translational joint is determined. The first speed limit information includes the maximum value of the baseline linear velocity and / or the maximum value of the baseline acceleration. The second speed limit information is sent to the truss translational joint so that the truss translational joint moves on the joint motion trajectory based on the second speed limit information.

[0078] The scaling factor can be understood as a coefficient used to dynamically adjust the upper limit of the motion parameters (velocity and / or acceleration) set relative to the truss translational joint; the linear velocity scaling factor can be considered as a parameter used to adjust the upper limit of the linear velocity set relative to the truss translational joint; and the acceleration scaling factor can be considered as a parameter used to adjust the upper limit of the acceleration set relative to the truss translational joint. The first speed limit information can be understood as a pre-set basic or general speed limit standard for the truss translational joint, serving as the benchmark for subsequent calculations of the actual speed limit information. Specifically, it includes the maximum baseline linear velocity and / or the maximum baseline acceleration. The maximum baseline linear velocity can be understood as the maximum allowable linear velocity for the truss translational joint set for a general scenario, to avoid excessive speed leading to loss of control, and is the basic benchmark for ensuring stable movement of the truss translational joint; the maximum baseline acceleration can be considered as the maximum allowable acceleration for the truss translational joint set for a general scenario, used to avoid mechanical shock, joint wear, or trajectory deviation caused by excessive acceleration. The second speed limit information can be understood as the actual speed limit standard executed by the truss translational joint. It is a dynamic adjustment of the first speed limit information, which not only meets the motion requirements of the current working scenario, but also ensures motion safety. It may include the maximum actual linear velocity and / or the maximum actual acceleration.

[0079] In this embodiment, the scaling factor and preset first speed limit information corresponding to the truss translational joint can be obtained as follows: if an editing operation on the editing area of ​​the current interface is detected, the scaling factor and first speed limit information corresponding to the truss translational joint are determined based on the editing operation. The editing operation can be the scaling factor and first speed limit information input by the user, or it can be the scaling factor and first speed limit information selected by the user in a drop-down box. Alternatively, the scaling factor and preset first speed limit information corresponding to the truss translational joint can be obtained as follows: the planning request information of the truss translational joint in the current working scenario is obtained, and the scaling factor and first speed limit information are obtained by parsing the planning request information. Another possible method is to obtain the scaling factor and preset first speed limit information corresponding to the truss translational joint: the scaling factor and first speed limit information are fixed in the robotic arm control system. This embodiment of the invention does not impose any limitations on these methods.

[0080] In this embodiment, taking the first speed limit information including the maximum baseline linear velocity and the scaling factor including the linear velocity scaling factor as an example, the method for determining the second speed limit information corresponding to the truss translational joint based on the first speed limit information and the scaling factor can be as follows: multiply the maximum baseline linear velocity by the linear velocity scaling factor to obtain the maximum actual linear velocity value that can be operated by the truss translational joint. By determining the actual operable second speed limit information corresponding to the truss translational joint in the current working scenario, dynamic and flexible adaptation of the speed limit information is achieved, making the motion state of the truss translational joint more adaptable to actual needs, and avoiding the problems of low efficiency or motion overload caused by using a universal and fixed baseline speed limit extreme value.

[0081] Optionally, speed limit information can be determined for all joints and sent to the corresponding joints.

[0082] It is understood that the timing of sending the second speed limit information to the truss translational joint should be before or simultaneously with the sending of each joint motion trajectory in the set of joint motion trajectories to the corresponding joint on the robotic arm as described in step S104, so as to ensure that the linear velocity and / or acceleration of the truss translational joint when it moves on the joint motion trajectory can be effectively constrained.

[0083] Figure 2 This is a schematic diagram of the structure of a charging robotic arm control device provided in an embodiment of the present invention. Figure 2 As shown, the device includes: a pose information determination module 21, a target pose determination module 22, a trajectory determination module 23, and a trajectory transmission module 24, wherein... The pose information determination module 21 is used to determine the pose information of the target object based on the first image of the target object, wherein the target object is associated with the current operating stage of the charging gun. The target pose determination module 22 is used to determine the target pose of the robotic arm end effector based on the pose information. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. The trajectory determination module 23 is used to obtain a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, combined with the unified calculation of all joints of the robotic arm. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint. The trajectory sending module 24 is used to send each of the joint motion trajectories in the set of joint motion trajectories to the corresponding joint on the robotic arm, so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

[0084] The charging robotic arm control device provided in this embodiment of the invention determines the pose information of the target object based on a first image of the target object, and the target object is associated with the current operation stage of the charging gun. Based on the pose information, the device determines the target pose of the robotic arm end effector, which includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. Based on the target pose, the current pose of the robotic arm end effector, and the joint information of the robotic arm, and combined with a unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translational joint. The set of joint motion trajectories is sent to the corresponding joints on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to realize the insertion and removal of the charging gun. This device dynamically adjusts the target object's range by associating it with the current operating stage of the charging gun. It also calculates the target object's pose information using the first image of the target object, improving the targeting accuracy of the pose calculation. Furthermore, it is adaptable to scenarios where multiple charging guns are configured in a charging station and a truss-suspended robotic arm is used for automatic plugging and unplugging. Based on the pose information, the target pose of the robotic arm's end effector is determined, providing a clear and accurate target reference for subsequent trajectory planning of each joint of the robotic arm. Based on the initial pose, final pose, and joint information, the robotic arm's rotational joints and the truss translational joints are treated as a whole. Synchronous calculation improves the matching degree of motion trajectories of each joint, avoiding problems such as trajectory jitter or endpoint deviation caused by asynchronous movement between rotary and translational joints. This enhances the coordination, smoothness, and flexibility of the robotic arm's movement. Furthermore, by generating a unified set of joint motion trajectories through calculation, it ensures that the trajectories are repeatable and drift-free, meeting the high-precision requirements of point-to-point, repeatability, and accurate joint movement for charging and plugging. By distributing the trajectories of each joint in the set of joint motion trajectories to the corresponding joints, the coordinated movement of each joint is driven to achieve the plugging and unplugging of the charging gun, ensuring the smooth completion of automatic charging operations and improving operational reliability and accuracy.

[0085] Furthermore, the device also includes a target object determination module, which can be specifically used for: Before determining the pose information of the target object based on the first image of the target object, the current operating stage of the charging gun is detected; When the current operation stage is the gun retrieval operation stage, the target object is determined to include the vehicle-end charging port and the charging gun of the vehicle to be charged, and the first image of the vehicle-end charging port and the first image of the charging gun are obtained. The gun retrieval operation stage represents the operation stage in which the end effector of the robotic arm goes to the preset placement position of the charging gun to pick up the charging gun and plugs the charging gun into the corresponding vehicle-end charging port. When the current operation stage is the return operation stage, the target object is determined to be the charging gun, and the first image of the charging gun is acquired. The return operation stage represents the operation stage in which the end effector of the robotic arm pulls the charging gun out of the charging port at the vehicle end and puts the charging gun back into the preset placement position. The first image was acquired by a camera mounted on the end of a robotic arm.

[0086] Furthermore, the pose information determination module 21 can be specifically used for: Based on the first image, determine the feature points corresponding to the target object; For the target object of the vehicle-side charging port, the feature points are filtered to obtain effective points, wherein the feature points are used to characterize the boundaries of each socket in the vehicle-side charging port. Based on the effective points and the geometric features of each of the sockets, the pose information of the target object is determined by a pose estimation algorithm; For a target object that is a charging gun, the pose information of the target object is determined by an identifier pose detection algorithm based on the size and angle information of the feature points and the preset identifier code on the charging gun. The feature points are used to characterize the boundary of the preset identifier code on the charging gun.

[0087] Furthermore, the target pose determination module 22 can specifically be used for: When the target object includes a vehicle-end charging port and a charging gun, the first target pose of the robotic arm end effector is determined by coordinate transformation based on the pose information of the charging gun. The second target pose is determined by coordinate transformation based on the pose information of the vehicle-side charging port. When the target object is a charging gun, the first target pose is determined by coordinate transformation based on the pose information of the charging gun. Obtain the preset charging gun return pose and use the charging gun return pose as the second target pose.

[0088] Furthermore, the coordinate transformation is implemented based on a coordinate transformation chain; The coordinate transformation chain includes: the pose transformation matrix of the end flange of the robotic arm relative to the fixed truss base, the pose transformation matrix of the camera deployed at the end of the robotic arm relative to the end flange of the robotic arm, the pose information of the target object, and the pose transformation matrix of the end actuator of the robotic arm relative to the end flange of the robotic arm.

[0089] Furthermore, the trajectory determination module 23 can specifically be used for: Based on the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a first target joint angle set for achieving the first target pose is determined. The first target joint angle set includes the first target joint angle of each joint on the robotic arm. Based on the first target pose, the second target pose, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of second target joint angles for achieving the second target pose is determined. The set of second target joint angles includes the second target joint angles of each joint on the robotic arm. Based on the current joint angle set corresponding to the current pose, the first target joint angle set, and the second target joint angle set, and combined with joint space interpolation, a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint is obtained.

[0090] Furthermore, the trajectory determination module 23 can also be specifically used for: The first pose sequence is generated by combining the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm with position linear interpolation and attitude spherical interpolation. A second pose sequence is generated based on the first target pose, the second target pose, and the joint information of the robotic arm, combined with position linear interpolation and attitude spherical interpolation. Based on the first pose sequence and the second pose sequence, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained.

[0091] Furthermore, the device also includes a limit information determination module, which can specifically be used for: Obtain the scaling factor corresponding to the translational joint of the truss and the preset first speed limit information, wherein the scaling factor includes a linear velocity scaling factor and / or an acceleration scaling factor; Based on the first speed limit information and the scaling factor, the second speed limit information corresponding to the truss translational joint is determined. The first speed limit information includes the maximum value of the baseline linear velocity and / or the maximum value of the baseline acceleration. The second speed limit information is sent to the truss translational joint so that the truss translational joint moves on the joint motion trajectory based on the second speed limit information.

[0092] The charging robotic arm control device provided in this embodiment of the invention can execute the charging robotic arm control method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0093] Figure 3A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0094] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0095] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as the charging robotic arm control method.

[0097] In some embodiments, the charging robotic arm control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded into and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the charging robotic arm control method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the charging robotic arm control method by any other suitable means (e.g., by means of firmware).

[0098] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0102] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0103] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0104] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0105] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A charging robot control method, characterized by, include: Based on the first image of the target object, the pose information of the target object is determined, and the target object is associated with the current operating stage of the charging gun; Based on the pose information, the target pose of the robotic arm end effector is determined. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. Based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the motion trajectories of each joint is obtained. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint. Each joint motion trajectory in the set of joint motion trajectories is sent to the corresponding joint on the robotic arm so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

2. The method of claim 1, wherein, Before determining the pose information of the target object based on the first image of the target object, the method further includes: Detect the current operating stage of the charging gun; When the current operation stage is the gun retrieval operation stage, the target object is determined to include the vehicle-end charging port and the charging gun of the vehicle to be charged, and the first image of the vehicle-end charging port and the first image of the charging gun are obtained. The gun retrieval operation stage represents the operation stage in which the end effector of the robotic arm goes to the preset placement position of the charging gun to pick up the charging gun and plugs the charging gun into the corresponding vehicle-end charging port. When the current operation stage is the return operation stage, the target object is determined to be the charging gun, and the first image of the charging gun is acquired. The return operation stage represents the operation stage in which the end effector of the robotic arm pulls the charging gun out of the charging port at the vehicle end and puts the charging gun back into the preset placement position. The first image was acquired by a camera mounted on the end of a robotic arm.

3. The method according to claim 1 or 2, characterized in that, Based on the first image of the target object, determine the pose information of the target object, including: Based on the first image, determine the feature points corresponding to the target object; For the target object of the vehicle-side charging port, the feature points are filtered to obtain effective points, wherein the feature points are used to characterize the boundaries of each socket in the vehicle-side charging port. Based on the effective points and the geometric features of each of the sockets, the pose information of the target object is determined by a pose estimation algorithm; For a target object that is a charging gun, the pose information of the target object is determined by an identifier pose detection algorithm based on the size and angle information of the feature points and the preset identifier code on the charging gun. The feature points are used to characterize the boundary of the preset identifier code on the charging gun.

4. The method of claim 1, wherein, Determining the target pose of the robotic arm end effector based on the pose information includes: When the target object includes a vehicle-end charging port and a charging gun, the first target pose of the robotic arm end effector is determined by coordinate transformation based on the pose information of the charging gun. The second target pose is determined by coordinate transformation based on the pose information of the vehicle-side charging port. When the target object is a charging gun, the first target pose is determined by coordinate transformation based on the pose information of the charging gun. Obtain the preset charging gun return pose and use the charging gun return pose as the second target pose.

5. The method of claim 4, wherein, The coordinate transformation is implemented based on a coordinate transformation chain; The coordinate transformation chain includes: the pose transformation matrix of the end flange of the robotic arm relative to the fixed truss base, the pose transformation matrix of the camera deployed at the end of the robotic arm relative to the end flange of the robotic arm, the pose information of the target object, and the pose transformation matrix of the end actuator of the robotic arm relative to the end flange of the robotic arm.

6. The method of claim 1, wherein, The method involves obtaining a set of joint motion trajectories, including the joint motion trajectories corresponding to each joint, based on the target pose, the current pose of the robotic arm end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm. Based on the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a first target joint angle set for achieving the first target pose is determined. The first target joint angle set includes the first target joint angle of each joint on the robotic arm. Based on the first target pose, the second target pose, and the joint information of the robotic arm, and combined with the unified calculation of all joints of the robotic arm, a set of second target joint angles for achieving the second target pose is determined. The set of second target joint angles includes the second target joint angles of each joint on the robotic arm. Based on the current joint angle set corresponding to the current pose, the first target joint angle set, and the second target joint angle set, and combined with joint space interpolation, a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint is obtained.

7. The method of claim 1, wherein, The method involves obtaining a set of joint motion trajectories, including the joint motion trajectories corresponding to each joint, based on the target pose, the current pose of the robotic arm end effector, and the joint information of the robotic arm, combined with a unified calculation of all joints of the robotic arm. The first pose sequence is generated by combining the first target pose, the current pose of the end effector of the robotic arm, and the joint information of the robotic arm with position linear interpolation and attitude spherical interpolation. A second pose sequence is generated based on the first target pose, the second target pose, and the joint information of the robotic arm, combined with position linear interpolation and attitude spherical interpolation. Based on the first pose sequence and the second pose sequence, and combined with the unified calculation of all joints of the robotic arm, a set of joint motion trajectories containing the corresponding joint motion trajectories of each joint is obtained.

8. The method of claim 1, wherein, Also includes: Obtain the scaling factor corresponding to the translational joint of the truss and the preset first speed limit information, wherein the scaling factor includes a linear velocity scaling factor and / or an acceleration scaling factor; Based on the first speed limit information and the scaling factor, the second speed limit information corresponding to the truss translational joint is determined. The first speed limit information includes the maximum value of the baseline linear velocity and / or the maximum value of the baseline acceleration. The second speed limit information is sent to the truss translational joint so that the truss translational joint moves on the joint motion trajectory based on the second speed limit information.

9. A charging robot control device characterized by comprising: include: The pose information determination module is used to determine the pose information of the target object based on the first image of the target object, wherein the target object is associated with the current operating stage of the charging gun. The target pose determination module is used to determine the target pose of the robotic arm end effector based on the pose information. The target pose includes a first target pose for the robotic arm end effector to acquire the charging gun and a second target pose for the robotic arm end effector to place the charging gun. The trajectory determination module is used to obtain a set of joint motion trajectories containing the joint motion trajectories corresponding to each joint based on the target pose, the current pose of the end effector of the robotic arm and the joint information of the robotic arm, combined with the unified calculation of all joints of the robotic arm. The robotic arm includes 6 robotic arm body rotation joints and 1 truss translation joint. The trajectory sending module is used to send each of the joint motion trajectories in the set of joint motion trajectories to the corresponding joint on the robotic arm, so that each joint moves according to the corresponding joint motion trajectory to realize the plugging and unplugging of the charging gun.

10. An electronic device, comprising: The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the charging robotic arm control method according to any one of claims 1-8.

Images