Battery swapping station robotic arm calibration method, controller, storage medium and battery swapping station

Abstract

This application relates to the field of battery swapping technology, specifically to a method for calibrating a robotic arm in a battery swapping station, a controller, a storage medium, and a battery swapping station. It aims to solve the technical problem of how to achieve fast, convenient, high-precision, and automated calibration of the robotic arm in a battery swapping station. To this end, the battery swapping station of this application is equipped with a vision sensor, and at least one reference mark is set at a preset position in the station. The robotic arm to be calibrated is controlled to move to the preset calibration position; visual features of the robotic arm are acquired based on the vision sensor, obtaining a first visual feature acquisition result. Based on the first visual feature acquisition result, the current pose of the robotic arm's preset features in a preset coordinate system is determined. The robotic arm is calibrated based on the current pose and the standard pose of the preset features. This enables a precise and automated calibration process for the robotic arm, which not only reduces labor costs but also significantly improves the robustness of the calibration process.

Description

Technical Field

[0001] This application relates to the field of battery swapping technology, specifically to a method for calibrating a robotic arm in a battery swapping station, a controller, a storage medium, and a battery swapping station. Background Technology

[0002] To free battery swapping from the constraints of vehicle wheelbase and battery specifications, a brand-new flexible battery swapping platform has been introduced at the swapping station. Unlike the fixed logic of its predecessor, this platform is equipped with dual lifting arms and multiple movable robotic arms. By integrating the positioning pins and the swapping nozzles onto the movable robotic arms, the new generation of swapping stations has almost completely freed itself from the constraints of the physical dimensions of the vehicle and battery, achieving a qualitative leap in the flexibility of battery swapping.

[0003] However, this "flexibility" also brings considerable challenges. Traditional battery swapping logic relies on "hard coding"—that is, the robotic arm strictly follows preset fixed coordinates. This mode proves inadequate when facing the need for compatibility with multiple vehicle models. If the actual position of the vehicle or battery deviates from the calibrated value by even a millimeter, the hard-coded logic will cause the gun head to fail to connect, or even damage the equipment.

[0004] This extreme demand for precision stems from three unavoidable uncertainties in reality:

[0005] 1. Static tolerances in infrastructure and installation: From ground flatness to steel structure welding, and then to the installation error of core components, the construction in the physical world can never reach the absolute zero point in mathematics.

[0006] 2. Dynamic differences between vehicle models and platforms: The new generation of battery swapping stations needs to be compatible with multiple brands and models of vehicles. Different wheelbases and chassis heights, coupled with the inherent assembly tolerances of mass-produced vehicles, make each vehicle's entry into the station a variable.

[0007] 3. Instantaneous disturbances of environment and load: Minor parking deviations of the vehicle when entering the station, suspension deformation caused by changes in load, and even interference from extreme weather on sensors all continuously challenge the stability of the system.

[0008] Traditional calibration methods typically rely on manual physical calibration using tooling and coordinate measuring machines. The shortcomings and deficiencies of existing technologies are mainly as follows:

[0009] 1. Low level of automation: Traditional manual calibration is inefficient. First, a very heavy tooling needs to be moved (usually requiring 2-4 people), then the gun head needs to be removed, a special calibration rod needs to be installed, and finally the tooling can be used for calibration. It cannot be performed frequently in the daily operation and maintenance of the battery swapping station, and it is difficult to detect the end-point deviation caused by mechanical wear and collision in a timely manner.

[0010] 2. Lack of closed-loop feedback: Existing vision solutions are often only used for "seeing" and lack a closed-loop mechanism to compare the detection results with standard values ​​and automatically trigger alarms or guide rework, which makes it impossible to prevent potential faults (such as unlocking failures) in advance.

[0011] Accordingly, there is a need in this field for a new calibration scheme for the robotic arm of a battery swapping station to solve the above problems. Summary of the Invention

[0012] In order to overcome the above-mentioned deficiencies, this application is made to solve, or at least partially solve, the technical problem of how to achieve fast, convenient, high-precision and automated calibration of the robotic arm of the battery swapping station.

[0013] In a first aspect, a method for calibrating a robotic arm at a battery swapping station is provided, wherein a vision sensor is installed on the battery swapping station and located above the robotic arm; at least one reference mark is set at a preset position on the battery swapping station; the method includes:

[0014] Control the robotic arm to be calibrated to move to the preset calibration position;

[0015] Based on the vision sensor, visual features are acquired from the preset features of the robotic arm to obtain the first visual feature acquisition result;

[0016] Based on the first visual feature acquisition result, the current pose of the preset feature of the robotic arm in the preset coordinate system is determined;

[0017] The robotic arm is calibrated based on the current pose and the preset standard pose of the preset features;

[0018] Wherein, the standard pose is the pose of the preset feature in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined based on the position of at least one of the reference marks in the preset coordinate system.

[0019] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the preset coordinate system is the world coordinate system;

[0020] The step of determining the current pose of the robotic arm's preset features in a preset coordinate system based on the first visual feature acquisition result includes:

[0021] Based on the first visual feature acquisition result and the preset extrinsic parameters of the visual sensor, the pose of the robotic arm in the world coordinate system is obtained according to the preset features.

[0022] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the method includes obtaining the extrinsic parameters of the vision sensor according to the following steps:

[0023] When the robotic arm has no calibration deviation and is located at the calibration position, the vision sensor is controlled to collect visual features of the preset feature and the reference mark to obtain a second visual feature collection result.

[0024] Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the second visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

[0025] In one technical solution of the above-mentioned method for calibrating the robotic arm at a battery swapping station, the method further includes obtaining the standard pose of the robotic arm based on preset features according to the following steps:

[0026] Based on the second visual feature acquisition result, obtain the visual coordinate system pose of the preset feature;

[0027] Based on the visual coordinate system pose and the extrinsic parameters, the visual coordinate system pose is transformed to the world coordinate system to obtain the standard pose.

[0028] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the first visual feature acquisition result includes the acquisition result of the reference mark; the method includes obtaining the extrinsic parameters of the visual sensor based on the first visual feature acquisition result according to the following steps:

[0029] Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the first visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

[0030] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the method further includes:

[0031] The extrinsic parameters of the visual sensor are obtained based on the PnP algorithm.

[0032] In one technical solution of the above-mentioned method for calibrating a robotic arm at a battery swapping station, calibrating the robotic arm based on the current pose and a preset standard pose with preset features includes:

[0033] The current pose is compared with the standard pose to obtain the pose deviation;

[0034] If the pose deviation is greater than a preset deviation threshold, an alarm message is generated;

[0035] The alarm information includes at least one of the following: the robot arm's number, the direction of deviation, and the magnitude of deviation.

[0036] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the method further includes:

[0037] If the pose deviation is greater than the deviation threshold, the pose deviation is compensated in real time based on motion commands.

[0038] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the characteristic is that...

[0039] The vision sensor is at least one of an industrial camera, a lidar, or a structured light depth camera.

[0040] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station,

[0041] The robotic arm is characterized by one of the following: a two-dimensional visual marker, a passive radio frequency identification tag, a magnetic marker, or a reflector.

[0042] In one technical solution of the above-mentioned method for calibrating the robotic arm of a battery swapping station, the reference mark is set on the V-groove center plate of the working platform of the battery swapping station.

[0043] In a second aspect, a controller is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above-described technical solutions for the calibration method of the robotic arm at the battery swapping station.

[0044] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method described in any of the technical solutions of the above-described method for calibrating a robotic arm at a battery swapping station.

[0045] In a fourth aspect, a battery swapping station is provided, which includes a vision sensor, a robotic arm, and the controller described in the above-mentioned controller technical solution;

[0046] The vision sensor is located above the robotic arm; at least one reference mark is set at a preset position of the battery swapping station.

[0047] Solution 1. A method for calibrating a robotic arm at a battery swapping station, characterized in that a vision sensor is installed on the battery swapping station, the vision sensor being located above the robotic arm; at least one reference mark is set at a preset position on the battery swapping station; the method includes:

[0048] Control the robotic arm to be calibrated to move to the preset calibration position;

[0049] Based on the vision sensor, visual features are acquired from the preset features of the robotic arm to obtain the first visual feature acquisition result;

[0050] Based on the first visual feature acquisition result, the current pose of the preset feature of the robotic arm in the preset coordinate system is determined;

[0051] The robotic arm is calibrated based on the current pose and the preset standard pose of the preset features;

[0052] The standard pose is the pose of the preset feature in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined based on the position of at least one reference mark in the preset coordinate system.

[0053] Scheme 2. The method for calibrating the robotic arm of a battery swapping station as described in Scheme 1, characterized in that the preset coordinate system is the world coordinate system;

[0054] The step of determining the current pose of the robotic arm's preset features in a preset coordinate system based on the first visual feature acquisition result includes:

[0055] Based on the first visual feature acquisition result and the preset extrinsic parameters of the visual sensor, the pose of the robotic arm in the world coordinate system is obtained according to the preset features.

[0056] Option 3. The method for calibrating the robotic arm of a battery swapping station according to Option 2, characterized in that the method includes obtaining the extrinsic parameters of the vision sensor according to the following steps:

[0057] When the robotic arm has no calibration deviation and is located at the calibration position, the vision sensor is controlled to collect visual features of the preset feature and the reference mark to obtain a second visual feature collection result.

[0058] Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the second visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

[0059] Solution 4. The method for calibrating the robotic arm of a battery swapping station according to Solution 3, characterized in that the method further includes obtaining the standard pose of the robotic arm based on preset features according to the following steps:

[0060] Based on the second visual feature acquisition result, obtain the visual coordinate system pose of the preset feature;

[0061] Based on the visual coordinate system pose and the extrinsic parameters, the visual coordinate system pose is transformed to the world coordinate system to obtain the standard pose.

[0062] Option 5. The method for calibrating the robotic arm of a battery swapping station according to Option 2, characterized in that the first visual feature acquisition result includes the acquisition result of the reference mark; the method includes obtaining the extrinsic parameters of the visual sensor based on the first visual feature acquisition result according to the following steps:

[0063] Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the first visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

[0064] Option 6. The method for calibrating the robotic arm of a battery swapping station according to any one of Options 2 to 5, characterized in that the method further includes:

[0065] The extrinsic parameters of the visual sensor are obtained based on the PnP algorithm.

[0066] Scheme 7. The method for calibrating the robotic arm of a battery swapping station according to Scheme 1, characterized in that,

[0067] The calibration of the robotic arm based on the current pose and the preset standard pose of the preset features includes:

[0068] The current pose is compared with the standard pose to obtain the pose deviation;

[0069] If the pose deviation is greater than a preset deviation threshold, an alarm message is generated;

[0070] The alarm information includes at least one of the following: the robot arm's number, the direction of deviation, and the magnitude of deviation.

[0071] Option 8. The method for calibrating the robotic arm of a battery swapping station according to Option 7, characterized in that the method further includes:

[0072] If the pose deviation is greater than the deviation threshold, the pose deviation is compensated in real time based on motion commands.

[0073] Scheme 9. The method for calibrating the robotic arm of a battery swapping station according to any one of Schemes 1 to 8, characterized in that,

[0074] The vision sensor is at least one of an industrial camera, a lidar, or a structured light depth camera.

[0075] Scheme 10. The method for calibrating the robotic arm of a battery swapping station according to any one of Schemes 1 to 8, characterized in that,

[0076] The robotic arm is characterized by one of the following: a two-dimensional visual marker, a passive radio frequency identification tag, a magnetic marker, or a reflector.

[0077] Scheme 11. The method for calibrating the robotic arm of a battery swapping station according to any one of Schemes 1 to 8, characterized in that,

[0078] The reference mark is set on the V-groove center plate of the working platform of the battery swapping station.

[0079] Option 12. A controller, characterized in that it comprises:

[0080] At least one processor;

[0081] And, a memory communicatively connected to the at least one processor;

[0082] The memory stores a computer program, which, when executed by the at least one processor, implements the battery swapping station robotic arm calibration method as described in any one of schemes 1 to 11.

[0083] Scheme 13. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the battery swapping station robotic arm calibration method as described in any one of Schemes 1 to 11.

[0084] Solution 14. A battery swapping station, characterized in that it includes a vision sensor, a robotic arm, and the controller described in Solution 12;

[0085] The vision sensor is located above the robotic arm; at least one reference mark is set at a preset position of the battery swapping station.

[0086] The above-described technical solutions of this application have at least one or more of the following beneficial effects:

[0087] In implementing the technical solution of the battery swapping station robotic arm calibration method provided in this application, a vision sensor is installed on the battery swapping station, located above the robotic arm; at least one reference mark is set at a preset position of the battery swapping station. The robotic arm to be calibrated is controlled to move to the preset calibration position; visual features of the robotic arm are acquired based on the vision sensor, obtaining the first visual feature acquisition result. Based on the first visual feature acquisition result, the current pose of the robotic arm's preset features in the preset coordinate system is determined. The robotic arm is calibrated based on the current pose and the standard pose of the preset features. The standard pose is the pose of the preset features in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined based on the position of at least one reference mark in the preset coordinate system. Through the above configuration, this application can realize a stable visual reference system based on the vision sensor, reference mark, and preset features of the robotic arm, enabling accurate and automated calibration of the robotic arm, which not only reduces labor costs but also enables rapid and quick calibration of the robotic arm. Meanwhile, since the calibration process is based on visual feature recognition by a visual sensor, it can significantly improve the robustness of the calibration process.

[0088] Furthermore, this application uses two-dimensional visual markers, passive radio frequency identification markers, magnetic markers, etc., set on the robotic arm as preset features of the robotic arm. Combined with the reference marker, the automatic marking process of the robotic arm is realized. It can achieve dual marking collaboration. The reference marker is used to correct the small changes of the visual sensor in real time, and the preset features are used to determine the current pose of the robotic arm. This avoids the uncertainty of using only a single marker for calibration and eliminates the impact of the installation error of the visual sensor on the calibration accuracy.

[0089] Furthermore, the calibration process of this application can generate alarm information when the pose deviation between the current pose and the standard pose of the robotic arm exceeds the deviation threshold. It can realize the complete business logic of standard setting, automatic verification and out-of-tolerance alarm of the robotic arm, and realize the effective quality control process of the robotic arm. Attached Figure Description

[0090] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein:

[0091] Figure 1 This is a schematic flowchart of the main steps of a battery swapping station robotic arm calibration method according to an embodiment of this application;

[0092] Figure 2 This is an example schematic diagram of reference markings according to one embodiment of the present application;

[0093] Figure 3 This is an example schematic diagram of a positioning mark according to one embodiment of the present application;

[0094] Figure 4 This is a schematic flowchart of the main steps of a battery swapping station robotic arm calibration method according to one embodiment of the present application. Detailed Implementation

[0095] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0096] In the description of this application, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, and memory, and may also include software components, such as program code, or a combination of software and hardware. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular forms of the terms "a" and "this" can also include plural forms.

[0097] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0098] The personal information processed in this application will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. This application will treat the user's personal information and its processing with the utmost diligence.

[0099] This application attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

[0100] In this embodiment, a vision sensor is installed on the battery swapping station, located above the robotic arm. At least one reference marker is positioned at a predetermined location on the battery swapping station. (See appendix.) Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a battery swapping station robotic arm calibration method according to an embodiment of this application. Figure 1As shown, the method for calibrating the robotic arm of a battery swapping station in this application mainly includes the following steps S101 to S104.

[0101] Step S101: Control the robotic arm to be calibrated to move to the preset calibration position.

[0102] In this embodiment, the robotic arm to be calibrated can be controlled to move to a determined calibration position for calibration.

[0103] In one implementation, the visual sensor can be an industrial camera, a LiDAR, a structured light depth camera, etc. For an industrial camera, the corresponding visual feature acquisition result can be an image. For a LiDAR, the corresponding visual feature acquisition result can be a point cloud. For a structured light depth camera, the corresponding visual feature acquisition structure can be a depth map.

[0104] In one implementation, the reference marker can be a two-dimensional visual marker (e.g., AprilTag).

[0105] In one embodiment, the preset features of the robotic arm can be its physical characteristics, such as the shape of the gun head or bolt holes.

[0106] In one embodiment, the preset features of the robotic arm can be two-dimensional visual markers, passive radio frequency identification tags, magnetic markers, reflectors, etc., set on the robotic arm.

[0107] In one embodiment, there may be one or more vision sensors. The intrinsic parameters and distortion parameters of the vision sensors are known and they are firmly fixed to the top of the battery swapping station to ensure that the field of view of the vision sensors can cover the reference marks and the preset features of the robotic arm.

[0108] In one implementation, there can be multiple reference markers, such as four, for example... Figure 2 As shown. The reference mark can be set on the center plate of the V-groove of the working platform of the battery swapping station. It should be noted that the reference mark can also be set in other locations of the battery swapping station, as long as it is not obstructed and can be installed securely, all of which are within the protection scope of this application.

[0109] In one implementation, the calibration process of the robotic arm can be triggered in several ways, such as time-triggered (e.g., every 4 hours of operation); counter-triggered (e.g., after every 100 battery swapping cycles); or manual command-triggered (e.g., initiated by maintenance personnel).

[0110] Step S102: Based on the vision sensor, perform visual feature acquisition on the preset features of the robotic arm to obtain the first visual feature acquisition result.

[0111] In this embodiment, the vision sensor can be controlled to collect visual features of the robotic arm's preset features, thereby obtaining the first visual feature collection result.

[0112] Step S103: Based on the first visual feature acquisition results, determine the current pose of the robot arm's preset features in the preset coordinate system.

[0113] In this embodiment, the current pose of the robot arm's preset features in the preset coordinate system can be determined based on the first visual feature acquisition results and the external parameters between the visual feature coordinate system and the preset coordinate system.

[0114] In one embodiment, the preset coordinate system can be the world coordinate system. The origin of the world coordinate system can be the circular hole at the zero point of the left push rod of the V-groove in front of the battery swapping station. It should be noted that the preset coordinate system can also be other coordinate systems, as long as the calibration method of this application can be implemented, they are all within the protection scope of this application.

[0115] Step S104: The robotic arm is calibrated according to the current pose and the standard pose of the preset feature; wherein, the standard pose is the pose of the preset feature in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined according to the position of at least one reference mark in the preset coordinate system.

[0116] In this embodiment, the robotic arm can be calibrated by combining its current pose and standard pose. The standard pose is the pose of a preset feature in a preset coordinate system when the robotic arm is in its calibration position without any calibration deviation. In some cases, the standard pose can be considered the pose of the preset feature in the preset coordinate system when the robotic arm is in its calibration position during the initial calibration phase, without any calibration deviation. This initial calibration phase can be performed once after system setup or major overhaul.

[0117] In one implementation, when the robotic arm has no calibration deviation and is in the calibration position, the vision sensor can be controlled to acquire visual features from preset features and reference marks to obtain a second visual feature acquisition result. Based on the visual coordinate system coordinates and world coordinate system coordinates of the reference marks in the second visual feature acquisition result, the extrinsic parameters of the vision sensor are obtained. Obtaining the extrinsic parameters of the vision sensor based on the second visual feature acquisition result allows for the calculation and storage of the transformation relationship between the visual coordinate system and the world coordinate system during the initial calibration stage. The extrinsic parameters can then be directly accessed during routine calibration, resulting in less computation and a more efficient calibration process.

[0118] In one implementation, the visual coordinate system pose of a preset feature can be obtained based on the second visual feature acquisition result; the visual coordinate system pose can be transformed to the world coordinate system based on the visual coordinate system pose and extrinsic parameters to obtain the standard pose.

[0119] In another embodiment, the extrinsic parameters of the visual sensor can be obtained based on the visual coordinate system coordinates of the reference marker and the world coordinate system coordinates of the reference marker in the first visual feature acquisition result. Obtaining the extrinsic parameters of the visual sensor based on a visual feature acquisition result ensures that even if the visual sensor has slight displacement, the extrinsic parameters can still accurately reflect the transformation relationship between the visual coordinate system and the world coordinate system, improving the accuracy and anti-interference capability of the calibration process.

[0120] In one implementation, the extrinsic parameters of the visual sensor can be obtained based on the PnP algorithm.

[0121] In another implementation, a deep learning model (e.g., CNN or Transformer) can be sampled to directly regress and predict the current pose of a robotic arm based on preset features. The deep learning module can be trained on a large amount of labeled data, and the results of the first visual feature acquisition can be input into the trained deep learning model to predict the current pose of the robotic arm's preset features.

[0122] In one implementation, the current pose can be compared with a standard pose to obtain the pose deviation; if the pose deviation is greater than a preset deviation threshold, an alarm message can be generated; wherein, the alarm message may include at least one of the following: the robot arm's number, the deviation direction, and the deviation magnitude. Those skilled in the art can set the value of the deviation threshold according to the needs of the actual application.

[0123] In one implementation, if the pose deviation is greater than the deviation threshold, the pose deviation can be compensated in real time based on motion commands.

[0124] In one implementation, a high-speed vision sensor can be used to capture the motion trajectory of the robotic arm and calculate its real-time pose deviation. An alarm is triggered when the pose deviation exceeds a threshold. Optical flow can be used to obtain the real-time pose deviation.

[0125] In one implementation, there can be two reference markers, and an IMU (Inertial Measurement Unit) is installed inside the vision sensor. The extrinsic parameters of the vision sensor can be obtained by combining the two reference markers and the IMU. That is, the attitude information of the vision sensor is obtained through the IMU, the extrinsic parameters of the vision sensor are calculated by combining the positions of the two reference markers, and the posture deviation of the robotic arm is calculated by fusing the visual and IMU data through Kalman filtering.

[0126] In one implementation, positioning marks at the end caps of multiple robotic arms can be used as mutual references to achieve collaborative calibration of the robotic arms. For example, two robotic arms in a battery swapping station are each equipped with positioning marks, and their respective pose deviations can be calculated by photographing each other's positioning marks.

[0127] In one implementation, offline calibration can be performed during the initial calibration phase, and the standard pose of the robotic arm can be stored. During daily operation, an online rapid detection can be performed every hour (e.g., identifying only a reference marker and a positioning marker). If the pose deviation is less than the deviation threshold, operation continues; otherwise, offline recalibration is triggered.

[0128] In one implementation, the calibration process of the robotic arm can be achieved using a cloud-edge collaborative approach. The edge device (e.g., a battery swapping station) performs real-time pose detection, the cloud collects data from multiple sites, optimizes the calibration algorithm through big data analysis (e.g., adaptively adjusts the threshold), and pushes the updated algorithm to the edge device.

[0129] See the appendix below. Figure 4 Taking a preset feature as a two-dimensional visual marker (i.e., a positioning marker) set on the robotic arm, an industrial camera as the visual sensor, and a world coordinate system as the preset coordinate system as an example, the calibration method of the robotic arm for the battery swapping station in this application embodiment will be further explained based on the initial calibration stage and the daily operation stage.

[0130] Phase 1: Initial Calibration Phase

[0131] S1: Perform hardware setup. This involves installing industrial cameras and setting up reference markers and 2D vision markers on the robotic arm (i.e., Figure 3 (The positioning marks shown). Use a coordinate measuring machine to measure and obtain the precise 3D coordinates (X_w, Y_w, Z_w) of all reference marks in the world coordinate system (the origin of the world coordinate system can be the circular hole at the zero point of the left push rod of the V-slot in front of the battery swapping station), and enter them into the system.

[0132] S2: Determine the transformation relationship between the image coordinate system and the world coordinate system during the initial calibration phase. Manually or automatically control the robotic arm to move to the calibration position. Trigger the industrial camera to capture an image (i.e., the second visual feature acquisition result). Based on image processing algorithms, identify all reference markers in the image and obtain the image coordinate system coordinates (u_i, v_i) of the reference markers. Since the world coordinate system coordinates (X_w, Y_w, Z_w) of the reference markers are known, the PnP algorithm can be used to solve for the transformation relationship matrix (i.e., the extrinsic parameter of the industrial camera) T_cam^world of the image coordinate system relative to the world coordinate system. This matrix serves as the bridge for the transformation from the image coordinate system to the world coordinate system.

[0133] S3: In the same image (i.e., the result of the second visual feature acquisition), identify the positioning marker (i.e., the preset feature) at the end of the robotic arm and obtain its coordinates in the image coordinate system. Using the extrinsic parameter T_cam^world obtained in S2, transform the coordinates of the positioning marker in the image coordinate system to the world coordinate system to obtain the pose (i.e., the standard pose) P_marker_standard of the positioning marker in the world coordinate system during the initial calibration stage. P_marker_standard, as the standard pose of the robotic arm at this calibration position, can be persistently stored in the system for use in daily calibration processes.

[0134] Phase Two: Routine Operation Phase (Periodic or Event-Triggered Execution)

[0135] C1: Trigger calibration. Calibration can be triggered in various ways: such as time trigger (e.g., every 4 hours of operation); counter trigger (after every 100 battery swap cycles); manual command trigger (e.g., initiated by maintenance personnel), etc.

[0136] C2: Control the robotic arm to move back to the calibration position. The industrial camera captures the current image (i.e., the first visual feature acquisition result). Since the industrial camera may move slightly, a reference marker can be identified first, and the current extrinsic parameter T_cam^world_current of the industrial camera can be recalculated based on the image coordinate system coordinates and world coordinate system coordinates of the reference marker. This step ensures the accuracy of the extrinsic parameter even with slight movement of the industrial camera, improving the anti-interference capability of the calibration process.

[0137] C3: Calculate the current pose: Identify the localization marker (i.e., the preset feature) in the current image (i.e., the first visual feature acquisition result), and use the current extrinsic parameter T_cam^world_current obtained in C2 to obtain the current P_marker_current of the localization marker in the world coordinate system (i.e., the current pose).

[0138] C4: Deviation Calculation and Judgment: Read the standard pose P_marker_standard of the pre-stored positioning marker. Calculate the pose deviation based on the following formula (1):

[0139] Δ = P_marker_current - P_marker_standard (1)

[0140] Here, Δ represents the pose deviation, which includes translational deviations in the X, Y, and Z directions.

[0141] The magnitude of Δ (or the components in each direction) can be compared with a preset deviation threshold to determine the current state of the robotic arm.

[0142] C5: Closed-loop feedback and control:

[0143] Scenario A: If the pose deviation is less than or equal to the deviation threshold, the robotic arm is considered to be in normal condition. The calibration process ends, and the robotic arm continues to be put into the battery swapping process. The data of this calibration process is recorded in the log for traceability.

[0144] Scenario B: If the pose deviation exceeds the deviation threshold, an alarm message is generated and notified to maintenance personnel via sound and light, SMS, or a host computer interface. The alarm message clearly indicates which robotic arm (ID), the direction of deviation, and the magnitude of the deviation (e.g., "Robotic arm #02, X-direction deviation +2.3mm, exceeding limit"). Simultaneously, if the robotic arm control system supports this, the pose deviation can be sent to the controller, which generates motion commands to compensate for the pose deviation in real time, temporarily correcting it and maintaining normal operation of the battery swapping station until manual maintenance is required. After manual maintenance, if mechanical adjustments or repairs are needed, a recalibration operation can be performed, i.e., running steps S2 and S3 of the initial calibration phase to update the pre-stored standard pose.

[0145] Based on the methods described in steps S101 to S104 above, a vision sensor is installed on the battery swapping station in this embodiment of the application, located above the robotic arm; at least one reference marker is positioned at a preset location on the battery swapping station. The robotic arm to be calibrated is controlled to move to a preset calibration position; visual features of the robotic arm are acquired based on the vision sensor, obtaining a first visual feature acquisition result. Based on the first visual feature acquisition result, the current pose of the robotic arm's preset features in a preset coordinate system is determined. The robotic arm is calibrated based on the current pose and the standard pose of the preset features. The standard pose is the pose of the preset features in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined based on the position of at least one reference marker in the preset coordinate system. Through the above configuration, this embodiment of the application can realize a stable visual reference system based on the vision sensor, reference marker, and preset features of the robotic arm, enabling precise and automated calibration of the robotic arm, reducing labor costs and achieving rapid and efficient calibration of the robotic arm. Meanwhile, since the calibration process is based on visual feature recognition by a visual sensor, it can significantly improve the robustness of the calibration process.

[0146] Furthermore, in this embodiment, two-dimensional visual markers, passive radio frequency identification markers, magnetic markers, etc., set on the robotic arm are used as preset features of the robotic arm. Combined with the reference marker, the automatic marking process of the robotic arm is realized. This enables dual marking collaboration. The reference marker is used to correct minute changes in the visual sensor in real time, and the preset features are used to determine the current pose of the robotic arm. This avoids the uncertainty of using only a single marker for calibration and eliminates the impact of the installation error of the visual sensor on the calibration accuracy.

[0147] Furthermore, the calibration process in this application embodiment can generate alarm information when the pose deviation between the current pose and the standard pose of the robotic arm exceeds the deviation threshold. It can realize the complete business logic of standard setting, automatic verification and out-of-tolerance alarm of the robotic arm, and realize the effective quality control process of the robotic arm.

[0148] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application and therefore will also fall within the protection scope of this application.

[0149] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0150] Another aspect of this application provides a computer-readable storage medium.

[0151] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program for executing the battery swapping station robotic arm calibration method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described battery swapping station robotic arm calibration method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device including various electronic devices, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0152] Another aspect of this application provides a controller.

[0153] In one embodiment of a controller according to this application, the controller may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the method described in any of the embodiments of the above-described battery swapping station robotic arm calibration method.

[0154] In some embodiments of this application, the processor may be a central processing unit, a microprocessor, a graphics processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor may be implemented in software, in hardware, or a combination of both.

[0155] The controller described in this application can be an industrial PC, an embedded controller, or other devices. As long as the controller has sufficient computing power to run computer vision algorithms and coordinate transformations, it is acceptable. This application does not limit this.

[0156] In one implementation, the connection between the controller and the vision sensor can be based on MIPI (Mobile Industry Processor Interface) or Ethernet.

[0157] In one implementation, the communication between the controller and the robotic arm can be via TCP / IP-based Ethernet communication.

[0158] Furthermore, this application also provides a battery swapping station.

[0159] In one embodiment of a battery swapping station according to this application, the battery swapping station mainly includes a vision sensor, a robotic arm, and the controller described in the above-described controller embodiment.

[0160] In this embodiment, the vision sensor can be located above the robotic arm; at least one reference marker can be set at the preset location of the battery swapping station.

[0161] The technical solution of this application has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for calibrating a robotic arm at a battery swapping station, characterized in that, The battery swapping station is equipped with a vision sensor located above the robotic arm; at least one reference marker is set at a preset position on the battery swapping station; the method includes: Control the robotic arm to be calibrated to move to the preset calibration position; Based on the vision sensor, visual features are acquired from the preset features of the robotic arm to obtain the first visual feature acquisition result; Based on the first visual feature acquisition result, the current pose of the preset feature of the robotic arm in the preset coordinate system is determined; The robotic arm is calibrated based on the current pose and the preset standard pose of the preset features; The standard pose is the pose of the preset feature in the preset coordinate system when the robotic arm is in the calibration position without calibration deviation; the standard pose is determined based on the position of at least one reference mark in the preset coordinate system.

2. The method for calibrating the robotic arm of a battery swapping station according to claim 1, characterized in that, The preset coordinate system is the world coordinate system; The step of determining the current pose of the robotic arm's preset features in a preset coordinate system based on the first visual feature acquisition result includes: Based on the first visual feature acquisition result and the preset extrinsic parameters of the visual sensor, the pose of the robotic arm in the world coordinate system is obtained according to the preset features.

3. The method for calibrating the robotic arm of a battery swapping station according to claim 2, characterized in that, The method includes obtaining the extrinsic parameters of the visual sensor according to the following steps: When the robotic arm has no calibration deviation and is located at the calibration position, the vision sensor is controlled to collect visual features of the preset feature and the reference mark to obtain a second visual feature collection result. Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the second visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

4. The method for calibrating the robotic arm of a battery swapping station according to claim 3, characterized in that, The method further includes obtaining the standard pose of the preset features of the robotic arm according to the following steps: Based on the second visual feature acquisition result, obtain the visual coordinate system pose of the preset feature; Based on the visual coordinate system pose and the extrinsic parameters, the visual coordinate system pose is transformed to the world coordinate system to obtain the standard pose.

5. The method for calibrating the robotic arm of a battery swapping station according to claim 2, characterized in that, The first visual feature acquisition result includes the acquisition result of the reference marker; the method includes obtaining the extrinsic parameters of the visual sensor based on the first visual feature acquisition result according to the following steps: Based on the visual coordinate system coordinates of the reference mark and the world coordinate system coordinates of the reference mark in the first visual feature acquisition result, the extrinsic parameters of the visual sensor are obtained.

6. The method for calibrating the robotic arm of a battery swapping station according to any one of claims 2 to 5, characterized in that, The method further includes: The extrinsic parameters of the visual sensor are obtained based on the PnP algorithm.

7. The method for calibrating the robotic arm of a battery swapping station according to claim 1, characterized in that, The calibration of the robotic arm based on the current pose and the preset standard pose of the preset features includes: The current pose is compared with the standard pose to obtain the pose deviation; If the pose deviation is greater than a preset deviation threshold, an alarm message is generated; The alarm information includes at least one of the following: the robot arm's number, the direction of deviation, and the magnitude of deviation.

8. The method for calibrating the robotic arm of a battery swapping station according to claim 7, characterized in that, The method further includes: If the pose deviation is greater than the deviation threshold, the pose deviation is compensated in real time based on motion commands.

9. The method for calibrating a robotic arm at a battery swapping station according to any one of claims 1 to 8, characterized in that, The vision sensor is at least one of an industrial camera, a lidar, or a structured light depth camera.

10. The method for calibrating a robotic arm at a battery swapping station according to any one of claims 1 to 8, characterized in that, The robotic arm is characterized by one of the following: a two-dimensional visual marker, a passive radio frequency identification tag, a magnetic marker, or a reflector.

Images