A Fully Automated Battery Replacement System and Method for Unmanned Aerial Vehicles Based on Binocular Vision Positioning

Abstract

This invention relates to a fully automated UAV battery replacement system and method based on binocular vision positioning, belonging to the field of UAV ground support equipment technology. It addresses the shortcomings of existing technologies, such as insufficient fault tolerance and adaptability in the face of landing deviations, environmental changes, or multi-model requirements, making reliable operation difficult in complex real-world scenarios. It also clarifies the lack of consideration for battery lock identification, which prevents the robotic arm from successfully grasping and installing batteries in practical applications. The system involves: acquiring images after UAV landing; segmenting and calculating the pose of the battery lock; unlocking; performing 3D reconstruction of acquired images; identifying UAV information and calculating the poses of the battery to be replaced and the empty charging compartment; planning a trajectory to place the battery to be replaced into the charging compartment; acquiring the pose of the fully charged battery and the empty compartment position; planning an installation trajectory to complete the battery replacement; and repeating the above steps until all batteries are replaced. This invention enables precise, fully automated UAV battery replacement.

Description

Technical Field

[0001] This invention relates to the field of UAV ground support equipment technology, specifically to a fully automated UAV battery replacement system and method based on binocular vision positioning. Background Technology

[0002] In recent years, drones have been widely used in fields such as inspection, surveying, logistics, and security. However, their continuous operation capability is limited by the limited flight time of onboard batteries. Frequent manual battery replacements are not only inefficient and increase operating costs, but also difficult to implement in field, high-altitude, or inaccessible environments. Therefore, achieving fully automated drone battery replacement has become a key infrastructure for improving the continuity and automation level of drone operations.

[0003] In the prior art, Chinese patent document CN121269152A discloses a "method and system for automatic battery swapping of unmanned aerial vehicles based on 3D vision guidance." This method constructs target feature templates for the battery compartment and the feeding battery, performs multi-level pyramid matching between the point cloud data to be matched and the feature templates, calculates the pose deviation value of the feeding battery, and controls a robotic arm to complete battery grasping and installation. In the prior art, Chinese patent document CN119428327A discloses a "method and system for installing and managing unmanned aerial vehicle batteries in an intelligent charging cabinet." This method uses a binocular vision system to acquire images of the unmanned aerial vehicle's battery compartment, identifies feature points of the battery compartment using the SURF feature extraction algorithm, calculates the spatial coordinates and attitude angles of the battery compartment using stereo matching and triangulation, and then controls a robotic arm to complete battery replacement.

[0004] However, all the above-mentioned technical solutions directly skip the accurate identification and unlocking operation of the battery lock, assuming that the battery lock is always open. This is seriously out of touch with actual battery swapping scenarios. In practical applications, due to the lock body obstructing the view or directly aligning the lock without unlocking, the robotic arm may be unable to complete the battery insertion or removal, or even cause jamming or collision accidents. Moreover, 3D point cloud matching has extremely poor segmentation capabilities for small-sized, low-texture precision structures like battery locks, easily blending the lock body with the battery compartment background and failing to extract independent features of the lock body. Single SURF feature extraction is completely ineffective for these weak feature areas of the lock body. Even if identification is attempted, insufficient feature points will lead to errors in pose calculation, making it impossible to guide the precise movement of the locking and unlocking mechanism.

[0005] In summary, existing technologies suffer from insufficient fault tolerance and adaptability when faced with landing deviations, environmental changes, or the needs of multiple aircraft models, making it difficult to work reliably in complex real-world scenarios. Furthermore, the lack of consideration for battery lock identification and processing leads to technical problems where the robotic arm cannot complete battery grabbing and installation in practical applications. Summary of the Invention

[0006] This invention solves the technical problems of existing technologies, such as insufficient fault tolerance and adaptability when facing landing deviations, environmental changes, or multi-model requirements, making it difficult to work reliably in complex real-world scenarios, and the failure to consider battery lock identification and processing, which leads to the inability of the robotic arm to complete battery grabbing and installation in practical applications.

[0007] The fully automatic battery replacement system for drones based on binocular vision positioning described in this invention includes:

[0008] A binocular vision system is used to simultaneously acquire image pairs of the drone's underside area and the surrounding charging compartment area;

[0009] The vision processing unit receives the battery status and the image pairs acquired by the binocular vision system, and performs the following:

[0010] The battery lock in the acquired image pair is segmented at the pixel level, the pose of the battery lock is calculated, the pose of the battery lock is transmitted to the lock opening mechanism to perform the unlocking action, after the unlocking is completed, the binocular vision system is triggered to acquire the image pair again, and the re-acquired image pair is reconstructed in three dimensions to obtain a dense three-dimensional point cloud.

[0011] A deep learning recognition and segmentation model is used to identify the corresponding UAV information. Based on dense 3D point cloud and UAV information, the pose of the battery to be replaced and the pose of the idle charging compartment are calculated.

[0012] The path planning module is used to receive dense 3D point clouds and the pose of the target object, obtain the current joint angle of the robotic arm, and plan the operation trajectory.

[0013] Robotic arm control is used to control the actuator to perform corresponding actions based on the work trajectory output by the path planning module;

[0014] An actuator, wherein the end effector of the actuator integrates an adsorption gripper and a force sensor, for performing corresponding actions;

[0015] A switch-lock mechanism is used to perform unlocking or locking actions based on battery lock position data.

[0016] When the actuator performs an action, the central control unit and force sensor provide force feedback to confirm that the contact force during the action meets the preset standard.

[0017] The fully automated battery replacement method for drones based on binocular vision positioning described in this invention, the method being built upon the aforementioned system, includes the following steps:

[0018] Step 1: After the drone lands on the helipad area, simultaneously acquire images of the drone's underside and the surrounding charging compartment area.

[0019] Step 2: Perform pixel-level segmentation on the battery lock in the image pair acquired in Step 1, calculate the pose of the battery lock, transmit the pose of the battery lock to the lock switch mechanism to perform the unlocking action, after the unlocking is completed, acquire the image pair again, perform 3D reconstruction on the acquired image pair to obtain a dense 3D point cloud, and use a deep learning recognition and segmentation model to identify the corresponding UAV information.

[0020] Step 3: Calculate the pose of the battery to be replaced and the pose of the idle charging compartment based on dense 3D point cloud and UAV information.

[0021] Step 4: Obtain the current joint angle of the robotic arm, combine the pose of the battery to be replaced and the pose of the empty charging compartment, plan the operation trajectory to place the battery to be replaced into the empty charging compartment, and execute the operation trajectory.

[0022] Step 5: Obtain the battery status in the charging compartment, calculate the pose of the fully charged battery based on the battery status, take a picture of the empty space in the drone's battery compartment, and calculate the pose of the current empty space in the drone's battery compartment.

[0023] Step 6: Based on the pose of the fully charged battery and the pose of the empty space in the current drone battery compartment, plan the operation trajectory to install the fully charged battery into the empty space in the drone battery compartment, and execute the operation trajectory to complete the replacement of a single battery to be replaced.

[0024] Step 7: Repeat steps 3 through 6 until all batteries to be replaced in the drone have been replaced.

[0025] Furthermore, in one embodiment of the present invention, step 2 involves pixel-level segmentation of the battery lock in the image pair acquired in step 1, calculation of the battery lock's pose, and control of the lock-on / lock-off mechanism to perform an unlocking action, including the following steps:

[0026] Step 21: Perform pixel-level segmentation on the image pairs to obtain the lock body region image;

[0027] Step 22: Extract rotation-invariant feature points from the lock body region image, calculate the three-dimensional coordinates of the rotation-invariant feature points, and achieve coarse positioning of the battery lock;

[0028] Step 23: Based on the coarse positioning results of the battery lock, perform edge detection on the lock body area image, extract the contour curves of the lock cylinder and the latch, fit the circular contour of the lock cylinder and the straight contour of the latch, and combine the PnP algorithm to optimize the pose calculation results to obtain the battery lock pose.

[0029] Step 24: Transmit the battery lock position data to the lock switch mechanism and control the lock switch mechanism to perform the unlocking action.

[0030] Furthermore, in one embodiment of the present invention, the drone information in step 2 includes the drone model, the number of batteries to be replaced, and the current location coordinates of the batteries to be tested.

[0031] Furthermore, in one embodiment of the present invention, step 3, which calculates the pose of the battery to be replaced and the pose of the idle charging compartment based on dense 3D point cloud and UAV information, includes the following steps:

[0032] Step 31: Based on the drone model in the drone information, obtain the standard feature point set of the battery to be tested for the corresponding drone model;

[0033] Step 32: In the dense 3D point cloud and the re-acquired image pair, identify and extract the image coordinates of at least 4 non-coplanar key feature points on the battery under test to obtain the current feature point set;

[0034] Step 33: Solve for the optimal spatial transformation between the standard feature point set and the current feature point set, and perform dynamic error compensation on the real-time calculated battery pose data to obtain the pose of the battery to be replaced.

[0035] Step 34: Identify the charging compartment that is in an idle state, extract the three-dimensional coordinates of the preset positioning feature points of the charging compartment, calculate the pose of the center point of the placement reference surface of the charging compartment in the visual coordinate system, and obtain the pose of the idle charging compartment.

[0036] Furthermore, in one embodiment of the present invention, step 4, which involves planning and executing a work trajectory for placing the battery to be replaced into an empty charging compartment, includes the following steps:

[0037] Step 41: Use the path planning module to generate the joint space trajectory from the current position of the actuator to the pose of the battery to be replaced;

[0038] Step 42: Control the actuator to execute the joint space trajectory, grab the battery to be replaced, and determine through force feedback that the force of grabbing the battery to be replaced meets the preset standard.

[0039] Step 43: Use a binocular vision system to confirm the pose of the idle charging compartment, use the path planning module to generate a placement trajectory from the pose of the battery to be replaced to the pose of the idle charging compartment, control the actuator to execute the placement trajectory, and use force feedback to determine that the force applied to place the battery to be replaced meets the preset standard.

[0040] Furthermore, in one embodiment of the present invention, the force feedback specifically refers to:

[0041] When the force sensor of the actuator detects that the contact force reaches the preset standard, it triggers the suction gripper of the actuator to grasp or release the battery.

[0042] Furthermore, in one embodiment of the present invention, step 32 uses a hybrid feature extraction method to extract at least four non-coplanar key feature points on the battery under test, and combines them with edge contour fitting to obtain the current feature point set.

[0043] Furthermore, in one embodiment of the present invention, the method further includes a data transmission step, specifically:

[0044] The battery pose to be replaced and the pose of the idle charging compartment calculated in step 3 are encoded into data frames via the Profinet Ethernet protocol and transmitted to the PLC. After the PLC verifies and decodes the data frames, they are stored in the PLC's storage area. Based on the stored battery pose to be replaced and idle charging compartment pose, the PLC calls the S-curve acceleration / deceleration algorithm and the spatial linear interpolation algorithm to generate the XYZU four-axis linkage operation trajectory. The operation trajectory is then sent to the actuator driver via PROFINET IRT isochronous synchronization, controlling the actuator to execute the operation trajectory.

[0045] Furthermore, in one embodiment of the present invention, after all batteries have been replaced, the binocular vision system is activated to identify the real-time position of the drone battery lock and transmit the real-time position to the locking mechanism. The locking mechanism is then controlled to perform a locking action. During the locking process, the torque of the motor of the locking mechanism is monitored in real time. If the torque exceeds a preset threshold and the time exceeding the preset threshold reaches a preset time, it is considered a locking failure, the locking task is stopped, the robotic arm is controlled to advance the Y-axis of the drone battery by 1mm, and then the locking mechanism is controlled to perform the locking action again. If the locking action fails more than 3 times, a manual assistance warning is automatically triggered and the subsequent process is suspended.

[0046] This invention addresses the technical problems of existing technologies, such as insufficient fault tolerance and adaptability in the face of landing deviations, environmental changes, or multi-aircraft requirements, making it difficult to work reliably in complex real-world scenarios, and the failure to consider battery lock identification and processing, which prevents the robotic arm from successfully grasping and installing batteries in practical applications. Specific beneficial effects include:

[0047] 1. This invention proposes a fully automated UAV battery replacement method based on binocular vision positioning. By simultaneously acquiring image pairs of the UAV's bottom area and the surrounding charging compartment area, the influence of UAV vibration on the accuracy of three-dimensional measurement is eliminated. Before each action, the next target point is visually repositioned to confirm the latest pose. The path of each action is generated in real time based on the current scene state, enabling the system to cope with various uncertainties such as landing deviation and fuselage shaking after battery removal. It has extremely high environmental anti-interference capability, does not rely on preset trajectories or precision mechanical positioning, and greatly improves the success rate and robustness of operations in complex environments. At the same time, it achieves true flexible automation by combining full-process dynamic planning, realizing a leap from fixed programming to adaptive generation.

[0048] 2. This invention proposes a fully automatic battery replacement method for drones based on binocular vision positioning. By extracting independent regions of the lock body through pixel-level segmentation, and combining ORB (Oriented Fast and Rotated BRIEF) feature points with contour fitting, the lock body pose is accurately calculated to guide the locking and unlocking mechanism to complete the action. After unlocking, secondary image acquisition and battery pose calculation are triggered, which effectively eliminates the small displacement caused by the unlocking action, improves the pose calculation accuracy, fills the technical gap in battery lock visual processing, and enables the system to operate stably under harsh conditions such as lock body occlusion and weak texture.

[0049] 3. This invention proposes a fully automatic battery replacement method for drones based on binocular vision positioning. By adopting a cyclical strategy of "one positioning, one replacement, one charging", the complex multi-battery replacement task is decomposed into multiple traceable and verifiable independent units, which greatly reduces the complexity of the task. At the same time, the repositioning mechanism ensures the accuracy of each step of the operation, and the discrete cyclic operation strategy ensures the absolute reliability of multi-battery replacement.

[0050] 4. This invention proposes a fully automated battery replacement method for drones based on binocular vision positioning. The central control unit performs global optimization scheduling of battery status, charging resources, and robotic arm tasks; the hybrid control mode of vision and force sensing balances operation speed and safety while ensuring accuracy. This enables the system to efficiently complete battery replacement while intelligent scheduling and hybrid control improve the overall system efficiency, achieving intelligent management of drones and battery assets throughout their entire lifecycle. Attached Figure Description

[0051] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0052] Figure 1 This is the overall architecture diagram of the UAV intelligent battery swapping system described in Implementation Method 1;

[0053] Figure 2 This is a flowchart of the intelligent battery swapping system for unmanned aerial vehicles described in Implementation Method 2. Detailed Implementation

[0054] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0055] Implementation Method 1: Existing technologies are mostly limited to verification of a single model of UAV or in a controlled experimental environment, and have inherent defects in terms of fault tolerance, environmental adaptability, multi-model compatibility and operational safety.

[0056] To address the aforementioned technical problems, this embodiment proposes a fully automated UAV battery replacement system based on binocular vision positioning, such as... Figure 1 The above includes:

[0057] A binocular vision system is used to simultaneously acquire image pairs of the drone's underside area and the surrounding charging compartment area;

[0058] The vision processing unit receives the battery status and the image pairs acquired by the binocular vision system, and performs the following:

[0059] The battery lock in the acquired image pair is segmented at the pixel level, the pose of the battery lock is calculated, the pose of the battery lock is transmitted to the lock opening mechanism to perform the unlocking action, after the unlocking is completed, the binocular vision system is triggered to acquire the image pair again, and the re-acquired image pair is reconstructed in three dimensions to obtain a dense three-dimensional point cloud.

[0060] A deep learning recognition and segmentation model is used to identify the corresponding UAV information. Based on dense 3D point cloud and UAV information, the pose of the battery to be replaced and the pose of the idle charging compartment are calculated.

[0061] The path planning module is used to receive dense 3D point clouds and the pose of the target object, obtain the current joint angle of the robotic arm, and plan the operation trajectory.

[0062] Robotic arm control is used to control the actuator to perform corresponding actions based on the work trajectory output by the path planning module;

[0063] An actuator, wherein the end effector of the actuator integrates an adsorption gripper and a force sensor, for performing corresponding actions;

[0064] A switch-lock mechanism is used to perform unlocking or locking actions based on battery lock position data.

[0065] When the actuator performs an action, the central control unit and force sensor provide force feedback to confirm that the contact force during the action meets the preset standard.

[0066] This implementation, within a binocular vision framework, first addresses the challenges of precise segmentation and unlocking of the battery lock, then performs battery attitude recognition. It employs an adaptive method combining real-time visual measurement, dynamic path planning, and servo control. The binocular vision system actively identifies and measures the actual pose of the drone's battery compartment, generating a matching robotic arm motion path in real time. The core lies in perceptive real-time adaptation and decision-making. Furthermore, by integrating model-specific visual recognition and a parameterized database, it aims to achieve compatibility with various drone models and enhance practicality and robustness in complex outdoor environments. During application, operation is divided into two phases: large-scale robotic arm movements are guided by a visual closed-loop system to the vicinity of the target, while precise operations such as contact, grasping, placement, and installation are switched to force-sensing servo control. This hybrid mechanism combines the accuracy of large-scale positioning with the smoothness and reliability of contact operations.

[0067] Implementation Method 2: A fully automated UAV battery replacement method based on binocular vision positioning, the method being built upon the system described in Implementation Method 1, and including the following steps:

[0068] Step 1: The drone autonomously flies to the battery swapping platform and lands in the designated parking area, which is marked with visual guidance signs. The binocular vision system installed on the platform is triggered, and the binocular camera, which has been pre-calibrated with high precision stereo, simultaneously acquires image pairs of the drone's underside and the surrounding charging compartment area.

[0069] Step 2: The images acquired in Step 1 are processed by grayscale conversion and Gaussian filtering for noise reduction to eliminate interference from ambient lighting and image noise in subsequent segmentation, while preserving the effective visual features of the lock body and battery compartment. The preprocessed binocular images are input into the lightweight SAM segmentation model (Segment Everything Model). Three key prompt points for the battery lock are selected in the image: the center of the lock cylinder, the upper edge of the latch, and the lower edge of the latch. These three points are fitted into a straight line, which is compared with the horizontal baseline. Based on the prompt points, the model performs global feature learning and local region segmentation, outputting an independent binary segmentation mask for the battery lock. This achieves pixel-level separation of the lock body from the battery compartment and background, extracting a pure lock body region image without background interference.

[0070] This implementation method overcomes the segmentation bottleneck of small-sized, low-texture lock bodies, abandons traditional visual matching or feature extraction methods, and uses a lightweight SAM segmentation model to perform pixel-level precise segmentation of the drone battery lock area based on images acquired by binocular vision. The lock body can be completely separated from the battery compartment and background with only a few visual cue points.

[0071] For the pure lock body region segmented by the lightweight SAM segmentation model, the ORB algorithm is used to extract rotation-invariant feature points, establishing the correspondence between lock body feature points from the left and right perspectives. Using the binocular vision triangulation principle, the three-dimensional spatial coordinates of the lock body feature points are calculated to achieve coarse lock body localization. Canny edge detection is then performed on the lock body region to extract the contour curves of the lock cylinder and latch. The circular contour of the lock cylinder and the linear contour of the latch are fitted using the least squares method. The pose calculation results are then optimized using the PnP algorithm (perspective n-point localization algorithm), ultimately obtaining the three-dimensional position coordinates and attitude parameters of the battery lock. The pose detection accuracy can reach the millimeter level. The calculated battery lock pose data is transmitted to the locking and unlocking mechanism to guide the mechanism in completing a precise unlocking action. After unlocking, a secondary acquisition command is triggered by the vision system.

[0072] This implementation uses vision to accurately identify the battery lock's position and unlock it without relying on additional equipment. Based on the independent lock body region segmented from the SAM large model, it integrates ORB feature point extraction and contour fitting algorithms to accurately calculate the three-dimensional position of the battery lock and the posture parameters required for opening and closing actions, such as the lock cylinder rotation angle and the latch push-pull direction, thus guiding the lock opening and closing mechanism to complete accurate unlocking.

[0073] The binocular cameras resynchronized and acquired images of the unobstructed bottom area of ​​the drone and the surrounding charging compartment area for I. left and I right To eliminate the impact of minute battery displacement / attitude shifts caused by unlocking actions, the image acquisition synchronization time error must be less than or equal to 1ms. This parameter ensures that the left and right cameras capture the drone's attitude at the same moment, avoiding 3D measurement failures due to minute drone movements. The image resolution should be no less than 1920×1080 pixels. The platform provides auxiliary lighting to ensure ambient illuminance is within the range of 100 Lux to 10000 Lux.

[0074] The preprocessing process is repeated for the re-acquired image pairs to ensure that the images are free from motion blur and feature shift caused by the unlocking action. Then, the vision processing unit is used to process the re-acquired image pairs. left and I righ Stereo matching is performed, and the disparity d of each matched pixel is calculated in pixels. Based on the binocular vision geometric model, the 3D coordinates (x, y, d) of the feature points in the camera coordinate system are calculated. c , y c , z c ):

[0075] z c = ;

[0076] x c = ;

[0077] y c = ;

[0078] Where f is the camera focal length in pixels, B is the baseline distance of the binocular camera in millimeters, (u,v) are the pixel coordinates of the feature points in the left image, and the output is the dense 3D point cloud P of the entire working scene. Cloud3D The disparity calculation accuracy of the stereo matching algorithm must be less than or equal to 0.5 pixels.

[0079] Based on the reconstructed 3D point cloud P Cloud3D The model generates a two-dimensional image of the drone and runs a deep learning recognition and segmentation model. The model outputs the drone's Model ID, the number of batteries N to be replaced, and the current coordinates of the batteries.

[0080] This implementation ensures spatiotemporal consistency of the left and right images through hardware synchronous triggering (error ≤ 1ms), and integrates deep learning to directly output the real-time, absolute 3D pose of the target (battery, charging compartment), providing a unique and true spatial reference for dynamic planning. Simultaneously, a full-process visual connection mechanism between the lock body and the battery is designed to achieve real-time data linkage. After the unlocking process is completed, the vision system automatically triggers secondary binocular image acquisition and feature refresh of the battery compartment area, preventing the slight displacement / attitude shift of the battery caused by the unlocking action from being ignored. Then, an optimized hybrid feature algorithm is used to calculate the battery attitude, ensuring absolutely accurate pose data for subsequent battery alignment.

[0081] Step 3: Calculate the pose of the battery to be replaced and the pose of the idle charging compartment based on dense 3D point cloud and UAV information.

[0082] Based on the Model ID, retrieve the standard 3D model of the first battery of this model to be replaced from the database, along with its standard feature point set P in the UAV body coordinate system. uav i In P Cloud3D In the re-acquired image pairs, the ORB+SIFT (Scale Invariant Feature Transform) dual feature point extraction algorithm is used to identify and extract the image coordinates of at least four non-coplanar key feature points on the battery surface. A left-right viewpoint feature point matching relationship is established, and triangulation is combined to achieve coarse battery localization, addressing the problem of insufficient feature points in sparse texture scenes. Canny edge detection and Hough line detection are performed on the battery swapping interface area to extract the straight / rounded contours of the interface. Precise contour parameters are fitted, and the 3D coordinates of the feature points are fused with the contour parameters to obtain the current feature point set P. cam i .

[0083] Using the PnP algorithm, solve for P cam i To Puav i The optimal spatial transformation is used to obtain the battery pose P1 = [R1, T1] relative to the visual coordinate system, where R1 is the rotation matrix and T1 is the translation vector. The pose calculation accuracy must be less than or equal to 1 mm and the offset angle must be less than or equal to 0.5°. The Kalman filter algorithm is used to track and correct the real-time calculated battery pose data to offset the small shaking error of the UAV parking. At the same time, the dynamic self-calibration mechanism of the binocular camera is used to correct the camera intrinsic parameter drift and extrinsic parameter deviation to ensure the stability of the pose data.

[0084] Dense 3D point cloud P in the same working scene Cloud3D In the process, the charging compartment in an idle state is identified. The three-dimensional coordinates of the preset positioning feature points of the charging compartment are extracted, and the pose of the center point of the placement reference surface of the charging compartment in the visual coordinate system is calculated as P2 = [R2, T2]. The pose measurement accuracy of the charging compartment must be less than or equal to 2mm.

[0085] Step 4: Obtain the current joint angle of the robotic arm, combine the pose of the battery to be replaced and the pose of the empty charging compartment, use the path planning module to plan the operation trajectory to place the battery to be replaced into the empty charging compartment, and use the actuator to execute the operation trajectory.

[0086] Step 5: Obtain the battery status in the charging compartment, calculate the pose of the fully charged battery based on the battery status, use a binocular vision system to take pictures of the empty space in the drone's battery compartment, and calculate the pose of the current empty space in the drone's battery compartment.

[0087] The process involves identifying and calculating the pose of a fully charged battery (P3) and re-localizing the empty battery compartment (P4). The central management system schedules a fully charged battery of the same model with a charge level ≥95% and instructs the vision system to locate the center point of the grasping reference surface of the charging slot containing that battery, calculating its pose P3 = [R3, T3]. Since the drone's center of gravity and attitude may change slightly after the first battery is removed, a second rapid visual localization is triggered. The binocular system again simultaneously captures, reconstructs, and identifies features in the empty area of ​​the drone's battery compartment, calculating the precise pose of the current empty compartment P4 = [R4, T4]. To ensure battery swapping efficiency, the response time for the second visual localization is less than or equal to 200 ms.

[0088] Step 6: Based on the pose of the fully charged battery and the pose of the empty battery compartment of the drone, the path planning module is used to plan the operation trajectory for installing the fully charged battery into the empty battery compartment of the drone. The actuator is used to execute the operation trajectory to complete the replacement of a single battery to be replaced.

[0089] The process involves grabbing and installing a fully charged battery, with the target point at P3. Using the path planning module, a trajectory called "Trackfullpick" (full battery grabbing trajectory) is planned and executed from the current position of the robotic arm to P3 to grab the battery. Next, using the same path planning module, a trajectory is planned to install the fully charged battery into the empty space in the drone's battery compartment, with the target point at P4. This trajectory is then executed from P3 to P4, called "Trackfullplace" (full battery installation trajectory), to accurately install the battery back into the drone's battery compartment. After force feedback confirms proper installation, the drone's battery compartment latch is triggered to lock, completing the single battery replacement cycle.

[0090] Update system status: Mark the charging slot where the original fully charged battery was located as "idle", and mark the battery compartment of the drone as "fully charged".

[0091] Step 7: Repeat steps 3 through 6 until all batteries to be replaced in the drone have been replaced.

[0092] For multi-battery drones, a discrete, cyclical operation is employed until all batteries are replaced. If the number of drone batteries, N, equals two, steps 3 through 6 are repeated to replace the second battery. Each battery replacement involves an independent, complete perception-planning-execution process. Each grab and installation step within the cycle is based on the latest visual positioning results, with independent path planning to ensure adaptability to any changes in the drone's pose throughout the process. The entire process is intelligently scheduled by the central control unit, forming a fully adaptive, highly reliable closed-loop operation. This effectively isolates and eliminates accumulated drone attitude errors caused by the removal of the previous battery, ensuring the overall accuracy of multi-battery replacement.

[0093] This implementation proposes an adaptive battery swapping method based on real-time perception and intelligent decision-making. It dynamically identifies the drone model and precise battery compartment pose using a binocular vision system, intelligently identifies the drone model and battery quantity, and performs synchronous binocular 3D reconstruction of the landing drone and charging compartment environment to obtain a full-field point cloud with millimeter-level accuracy. Based on this, it plans the robotic arm's motion path in real time, entering a pick-up-place-load cycle. The core of this implementation lies in the closed-loop control paradigm of synchronous perception-real-time decision-making-compliant execution, enabling the system to proactively adapt to landing deviations and be compatible with multiple drone models. Furthermore, within each cycle:

[0094] 1) The vision processing unit accurately calculates the pose P1 of the battery to be replaced and the pose P2 of the idle charging compartment;

[0095] 2) The path planning module dynamically generates a grasping path based on the real-time measured pose P1 of the battery to be replaced and the current environment, and the robotic arm grasps smoothly under the force servo.

[0096] 3) The path planning module generates a placement path based on the pose of the idle charging compartment P2 to complete the battery transfer and charging storage;

[0097] 4) The system schedules a fully charged battery and quickly relocates the drone's battery compartment to the empty P4 slot;

[0098] 5) The robotic arm grabs the fully charged battery P3 and installs it into the empty battery compartment P4 of the drone.

[0099] like Figure 2 As shown, within a single replacement cycle, the system needs to generate three different optimal trajectories (grabbing, placing, and installing) online based on four different real-time poses (P1, P2, P3, P4). Each planning step uses the latest visual measurements as input, ensuring the system's real-time adaptability to all state changes during the operation.

[0100] This implementation addresses the high real-time requirements of battery swapping processes in industrial settings. It solves the problems of large parameter count, long inference time, and poor adaptability of redundant feature layers when the original SAM segmentation model is directly applied to UAV battery lock segmentation scenarios. Based on the PyTorch framework, the model undergoes scenario-specific adaptation modifications including convolutional network layer pruning, parameter freezing, and INT8 lightweight quantization. First, redundant semantic feature extraction layers and multi-scale fusion redundant layers for general scenarios are removed from the model, retaining only the core bottom-level feature extraction layer and target mask generation layer required for battery lock segmentation. Simultaneously, the pre-trained parameters of the core feature extraction layer are frozen to avoid damage to basic feature extraction capabilities. Only a small number of parameters in the mask generation layer are fine-tuned to reduce computational load and the risk of overfitting. To address the accuracy loss introduced by quantization, a triple mechanism of dynamic range calibration, KL divergence optimization, and pseudo-quantization training is employed for precise control. This involves determining the quantization range by statistically analyzing the activation value distribution of battery lock scenario samples, selecting the quantization threshold with the smallest distribution difference, and simulating quantization errors during the fine-tuning phase to adapt the model. Ultimately, this achieves optimization results: a 75% reduction in model size, a 2-4 times increase in inference speed, and a more than 50% reduction in power consumption. Furthermore, the segmentation accuracy decreases by only 0.2% after quantization, and the segmentation inference time for the battery lock region is controlled within 50ms, perfectly adapting to the deployment requirements of embedded industrial control computers in battery swapping systems and meeting both real-time and accuracy requirements. Simultaneously, a dedicated pose data linkage transmission mechanism is established to uniformly transmit the pose calculation data of the lock body and battery to the same main control system. Built-in hard-link logic of "unlocking action completed → battery pose real-time refresh" ensures zero-delay and uninterrupted data transmission through transmission priority configuration and real-time verification, guaranteeing smooth workflow between lock opening / closing and battery pose detection.

[0101] Implementation Method 3: The difference between this implementation method and Implementation Method 2 is that step 4 involves planning the operational trajectory for placing the battery to be replaced into an empty charging compartment and executing this operational trajectory, including the following steps:

[0102] Step 41: Use the path planning module to generate the joint space trajectory from the current position of the actuator to the pose of the battery to be replaced;

[0103] Step 42: Control the actuator to execute the joint space trajectory, grab the battery to be replaced, and determine through force feedback that the force of grabbing the battery to be replaced meets the preset standard.

[0104] Step 43: Use a binocular vision system to confirm the pose of the idle charging compartment, use the path planning module to generate a placement trajectory from the pose of the battery to be replaced to the pose of the idle charging compartment, control the actuator to execute the placement trajectory, and use force feedback to determine that the force applied to place the battery to be replaced meets the preset standard.

[0105] Plan the adaptive grasping path and execution from the current position of the robotic arm to P1:

[0106] The path planning module receives the pose P1, the current joint angles of the robotic arm, and the overall 3D point cloud PCloud3D. Using P1 as the target point for grasping, and considering the kinematic constraints, dynamic constraints, and collision avoidance with the drone body, it generates a smooth, collision-free joint space trajectory, Trackpick (the trajectory for grasping the battery to be replaced), from the current position to P1. The robotic arm is controlled to move along the Trackpick trajectory to P1. In the final stage, it switches to force servo control. When the end effector's six-dimensional force sensor detects that the contact force reaches a preset threshold range, it triggers the adsorption gripper to stably grasp the battery.

[0107] Among them, the maximum speed V at the end of the planned trajectory max Less than or equal to 0.5 m / s. Maximum contact force F during the contact phase. contact The value should be less than or equal to 10 N to prevent damage to the drone structure. Path planning, from receiving P1 to generating the Trackpick, should take less than or equal to 300 ms to ensure real-time performance.

[0108] Plan the placement path and execution from P1 to P2:

[0109] After the robotic arm picks up the battery, the path planner uses P2 as the placement target point and generates a placement trajectory Trackplace (the placement trajectory of the battery to be replaced) from the current pose P2.

[0110] The robotic arm moves along the trackplace, precisely transporting the depleted battery to the available charging compartment P2. After confirming the battery is in place via force feedback, the gripper releases the battery, triggering the locking mechanism of the charging compartment to secure it and initiating the charging process. The central management system updates the charging compartment's status to "Charging".

[0111] Among them, the maximum speed V at the end of the planned trajectory placeMinimum speed of 0.3 m / s. Ensure stable placement. Alignment error between battery and charging slot contacts should be less than or equal to 0.5 mm.

[0112] In this embodiment, when the force sensor of the actuator detects that the contact force reaches a preset standard, it triggers the suction gripper of the actuator to grasp or release the battery.

[0113] Implementation Method Four: The difference between this implementation method and Implementation Method Two is that the method further includes a data transmission step, specifically:

[0114] The battery pose to be replaced and the pose of the idle charging compartment calculated in step 3 are encoded into data frames using the Profinet Ethernet (an automation bus standard based on industrial Ethernet) protocol and transmitted to the PLC (Programmable Logic Controller). The PLC verifies and decodes the data frames and stores them in the PLC's storage area. Step 4 is executed after the data is ready.

[0115] Current technology lacks a standardized transmission protocol, and visual data is merely theoretical, unable to be recognized by PLCs or actuators. This could lead to transmission interruptions during actual battery swapping due to data format incompatibility. Secondly, current technology lacks a data verification mechanism; erroneous data directly drives the actuator, causing unlocking or alignment actions to completely deviate from the target.

[0116] To address the aforementioned technical problems, this embodiment proposes a data transmission method, specifically including the following steps:

[0117] Step 1: Visual data preprocessing and encoding:

[0118] 1. After the binocular vision computing end completes the pose calculation of the battery lock, battery, and battery box, it performs "range normalization" processing on the raw data - converting the coordinate values ​​(mm) and angle values ​​(°) into 16-bit unsigned integers supported by the Profinet protocol;

[0119] 2. Pack the data according to the Profinet PDO format (Profinet protocol-based process data object format): Divide the three types of pose data (a total of 18 parameters) into 3 data segments, each with 6 parameters, and add a frame header (0x5A5A), a frame tail (0xA5A5), and a CRC32 checksum to form a complete data frame (total length 64 bytes).

[0120] 3. The vision-end Profinet communication module (such as Siemens CP243-1 IT) encapsulates the encoded data frame into Ethernet packets and sets the communication cycle to 10ms to ensure real-time performance.

[0121] Step 2, Profinet Ethernet data transmission:

[0122] 1. The vision device establishes a physical connection with the Siemens S7-200 SMART PLC through an industrial Ethernet switch, using the TCP / IP v4 protocol. The vision device's IP is set to 192.168.0.10, and the PLC's IP is set to 192.168.0.1.

[0123] 2. The PLC acts as a Profinet IO controller, sending data request commands to the vision device (IO device). The vision device actively uploads the encoded pose data frames at 10ms intervals.

[0124] 3. During transmission, enable the "Real-time Channel (RT)" mode of the Profinet protocol to prioritize the transmission of pose data and avoid delays caused by other data occupying bandwidth in the industrial field.

[0125] Step 3: PLC-side data decoding and verification:

[0126] 1. After receiving a data frame, the PLC first checks whether the frame header / frame tail is the preset value (0x5A5A / 0xA5A5). If it is abnormal, the frame is discarded and a retransmission command is triggered.

[0127] 2. Perform CRC32 check on the data frame: The PLC calculates the check code of the received data according to the same algorithm and compares it with the check code in the frame. If they match, the data is considered valid. If they do not match, the vision end is triggered to retransmit. If the number of retransmissions is greater than 3 and it still fails, an alarm is triggered.

[0128] 3. Decode valid data: Restore the 16-bit integer to the original pose data (e.g., 12532 → 125.32mm), store it in the V storage area of ​​the PLC (battery lock data is stored in V0.0-V5.0, battery data is stored in V6.0-V11.0, and battery box data is stored in V12.0-V17.0), and mark the "data ready" status.

[0129] Step 4: Linkage Trigger Trajectory Planning:

[0130] 1. After the PLC detects that all three types of pose data in the V storage area are "ready", it sends a "pose data ready" signal (M0.0 set to 1) to the trajectory generation module (integrated in the PLC or host computer).

[0131] 2. After receiving the signal, the trajectory generation module reads the original pose data from the PLC V storage area as the core input parameter for trajectory generation;

[0132] 3. If data transmission is interrupted / error occurs (PLC marks "data abnormal"), M0.0 is set to 0, the trajectory generation module is paused, and the PLC controls the actuator to be in standby mode to avoid malfunction.

[0133] This implementation uses Siemens STEP 7-Micro / WIN SMART software. The PLC is configured as a Profinet IO controller, and a vision device is added as an IO device. The input / output areas corresponding to the pose data are mapped, and the data update cycle is set to 10ms to match the transmission cycle of the vision device. Both the encoding algorithm of the vision device and the decoding algorithm of the PLC are programmatically fixed (the vision device is written in Python, and the PLC is written in STL ladder logic), requiring no manual intervention and ensuring stability in the industrial environment. An exception handling mechanism is employed, specifically:

[0134] 1. Transmission timeout (PLC does not receive data within 20ms): Triggers the vision terminal to restart the communication module, and the PLC issues an audible and visual alarm at the same time;

[0135] 2. If the data fails to verify three times consecutively: pause the battery swapping process, and re-acquire and calculate the pose data at the vision end to avoid the accumulation of erroneous data;

[0136] 3. IP Conflict Prevention: A preset backup IP network segment (192.168.1.0 / 24) is automatically switched to ensure uninterrupted transmission links.

[0137] Step 4, which is executed after the data is ready, specifically involves:

[0138] Based on the stored battery pose to be replaced and the pose of the idle charging compartment, the PLC calls the S-curve acceleration / deceleration algorithm and the spatial linear interpolation algorithm to generate the XYZU four-axis linkage operation trajectory. The operation trajectory is then sent to the actuator driver via PROFINET IRT isochronous synchronization, controlling the actuator to execute the operation trajectory.

[0139] Existing technologies do not employ standardized industrial-grade acceleration and deceleration algorithms, relying only on simple trapezoidal acceleration and deceleration or lacking acceleration and deceleration planning. This results in discontinuous acceleration and deceleration, leading to sudden changes in speed or acceleration during axis movement and significant mechanical shock, which completely fails to meet the high-precision battery swapping requirements of the XYZU four-axis system.

[0140] To address the aforementioned technical issues, this implementation method, based on the motion control firmware and process objects of a Siemens S7-200 SMART PLC, designs an XYZU four-axis trajectory planning system with single-axis S-curve acceleration / deceleration and multi-axis spatial linear interpolation. This system connects with the pose data transmitted via Profinet, achieving smooth, shock-free, synchronous, high-precision, and industrial-grade real-time motion of the battery swapping mechanism. Specifically, the method includes the following steps:

[0141] Step 1: The PLC reads the pose data and determines the motion mode.

[0142] Step 11: The PLC reads the battery lock / battery / battery box pose data transmitted by Profinet from the V storage area;

[0143] Step 12: Determine the operating mode based on the battery swapping conditions:

[0144] Independent point mode: Only a single axis needs to be in position, and the single-axis S-shaped acceleration and deceleration algorithm is called;

[0145] Linked mode: Requires four axes to be synchronized in place, and calls the S-shaped acceleration and deceleration + spatial linear interpolation combination algorithm.

[0146] Step 2, Single-axis independent point motion (S-curve acceleration / deceleration algorithm only):

[0147] Step 21: Create "axis process objects" for each axis of XYZU, enable the default 7-segment S-type acceleration and deceleration, and configure parameters such as jerk, acceleration, and maximum speed;

[0148] Step 22: The PLC calls the `MC_MoveAbsolute` instruction to send "starting coordinates + target coordinates + motion parameters" to the target axis. The PLC firmware automatically generates a 7-segment S-shaped velocity / acceleration curve for that axis.

[0149] Step 23: During the motion, the PLC collects the actual position of the axis at millisecond intervals and performs closed-loop correction with the target position, achieving millimeter-level positioning accuracy for a single axis.

[0150] Step 3: Multi-axis linkage linear motion (S-shaped acceleration / deceleration + spatial linear interpolation):

[0151] Step 31, Configure linkage parameters: Create a "four-axis linkage group" in "motion control process object", enable the spatial linear interpolation function, and unify the jerk and interpolation speed of the synthetic trajectory;

[0152] Step 32: The PLC calls the `MC_MoveLinearAbsolute` instruction to send the start coordinates and target coordinates of the XYZU four axes;

[0153] Step 33: The PLC firmware automatically executes the following logic:

[0154] 1. Calculate the composite trajectory length: Calculate the total length L of the composite trajectory from the starting point to the target point on the four axes according to the four-dimensional spatial distance formula;

[0155] 2. Generate the composite trajectory S-shaped curve: Based on L, generate a 7-segment S-shaped acceleration and deceleration composite velocity curve according to the configuration parameters;

[0156] 3. Axis speed distribution: The composite speed curve is decomposed into the component speed curves of each axis XYZU according to the proportion of each axis displacement to L, to ensure that the acceleration / constant speed / deceleration phases of each axis are completely synchronized.

[0157] 4. Real-time closed-loop control: The PLC sends the position setpoint of each axis to the servo drive via PROFINET IRT at millisecond intervals. The drive performs position closed-loop control to achieve synchronous start and stop and simultaneous arrival of the four axes.

[0158] Step 4: Motion completion verification and linkage coordination:

[0159] Step 41: After the motion is completed, the PLC collects the actual position of each axis and compares it with the target pose data. If the error is ≤0.05mm, it is determined that the motion is in place.

[0160] Step 42: Trigger the next step according to the battery swapping process to achieve seamless connection between "trajectory planning and action execution".

[0161] In this embodiment, parameters such as jerk and acceleration can be adjusted online according to mechanical characteristics without restarting the PLC, adapting to the mechanical characteristics of different battery swapping devices. If an axis position deviation exceeding 0.1mm is detected during the movement, the PLC immediately triggers an "S-shaped deceleration stop" to avoid hard stop impact. After rereading the pose data, the trajectory is planned to ensure battery swapping safety. If continuous multi-segment linear linkage is required (such as multi-battery swapping), it can be upgraded to a Siemens S7-1500 PLC, enabling a look-ahead control algorithm to pre-plan the connection speed of multiple trajectory segments, achieving uninterrupted continuous motion.

[0162] Implementation Method 5: The difference between this implementation method and Implementation Method 4 is that this implementation method is equipped with an integrated switch and lock mechanism. The existing unlocking method uses fixed coordinate values, which can only complete the unlocking task of fixed model drones and does not have flexibility.

[0163] This implementation combines binocular vision positioning technology with the vertical alignment characteristics of the drone's mounting platform. The platform secures the drone, ensuring the locking / unlocking mechanism is perpendicular to the plane containing the drone's battery and battery lock, thus completing the entire battery replacement process. The specific steps are as follows:

[0164] After the drone docks and is secured on the fixed platform, the binocular vision system is activated to acquire images and perform pixel-level segmentation of the drone's battery lock area, extracting independent regions of the lock body. Combining ORB (Oriented Fast and Rotated BRIEF) feature point extraction and contour fitting algorithms, the shape and spatial position parameters of the lock body are accurately calculated. Based on the calculation results, the locking and unlocking mechanism is controlled to move in a direction perpendicular to the battery lock plane to complete the unlocking operation of the drone's battery lock.

[0165] Currently, the locking operation relies on preset fixed coordinate values, which is only suitable for scenarios where the drone is fully positioned under ideal conditions. If the actual position of the battery lock deviates from the preset coordinates due to landing deviations or drone movement during the battery swapping process, the locking mechanism will be unable to accurately align with the lock body, easily leading to locking failure or damage to the lock. Furthermore, the current locking process lacks a real-time feedback mechanism; when the lock body is stuck or not fully in place, the abnormality cannot be detected, and only a simple error report and shutdown can be issued, resulting in poor fault tolerance and difficulty in ensuring the continuity and reliability of the battery swapping process.

[0166] Therefore, in this embodiment, after all batteries have been replaced, the binocular vision system is activated to identify the real-time position of the drone battery lock and transmit the real-time position to the locking mechanism. The locking mechanism is then controlled to perform the locking action. During the locking process, the torque of the motor of the locking mechanism is monitored in real time. If the torque exceeds a preset threshold and the time exceeding the preset threshold reaches a preset time, it is considered a locking failure, the locking task is stopped, the robotic arm is controlled to advance the Y-axis of the drone battery by 1mm, and then the locking mechanism is controlled to perform the locking action again. If the locking action fails more than 3 times, a manual assistance warning is automatically triggered and the subsequent process is suspended.

[0167] Implementation Method Six: This implementation method is a specific embodiment proposed for the system and method described in Implementation Methods One to Five.

[0168] Assuming an industrial drone (model: Model_X) with dual batteries performs a battery swapping task, the battery swapping platform is based on closed-loop control of "binocular vision positioning → Profinet data transmission → Siemens PLC trajectory planning → robotic arm movement". The complete battery swapping process is as follows:

[0169] Step 1: Battery swapping platform standby and drone landing trigger

[0170] The battery swapping platform is initially in standby mode, the helipad clamping device is not activated, and the binocular vision system, Siemens S7-200 SMART PLC, and robotic arm are all in low-power ready mode. When the drone lands in the preset area of ​​the helipad, the helipad sensor triggers the "drone in position" signal and uploads it to the PLC. The PLC immediately controls the clamping device to move and accurately fix the drone, ensuring that the robotic arm, the switch lock mechanism, and the plane where the drone battery and battery lock are located remain perpendicular.

[0171] Step 2: Battery lock identification and unlocking control

[0172] 1. Visual Acquisition and Pose Calculation: The PLC triggers the start of the binocular vision system, the auxiliary light source illumination is stabilized at 500 Lux, and the camera acquires images of the UAV battery lock area at a resolution of 2448×2048 pixels; after pixel-level segmentation using a lightweight SAM model, the three-dimensional pose parameters of the battery lock are calculated by combining ORB feature point extraction and SGM stereo matching algorithm, with a segmentation and inference time of ≤50ms.

[0173] 2. Profinet data transmission: The vision device encodes the battery lock pose data in Profinet format and transmits it to the PLC at 10ms intervals with a transmission error of ≤0.01mm. After the PLC verifies the data as valid, it marks "Lock body pose ready".

[0174] 3. PLC-controlled unlocking: The PLC calls a single-axis 7-segment S-shaped acceleration and deceleration algorithm, and plans the unlocking path of the robot arm through the `MC_MoveAbsolute` instruction. It controls the robot arm to complete the unlocking action in a direction perpendicular to the plane of the battery lock. After unlocking, the robot arm sends a "unlocked in place" signal back to the PLC. The vision system simultaneously verifies the lock body status. After the two signals are confirmed, the next step is initiated.

[0175] Step 3: Remove the first battery

[0176] Step 31: Battery to be replaced pose recognition and grasping path planning

[0177] 1. The vision system reacquires images, identifies the three-dimensional pose coordinates of the first battery to be replaced on the drone, and transmits them to the PLC via Profinet;

[0178] 2. The PLC uses the RRT-Connect algorithm to complete the global path planning for the robotic arm grasping, combined with the time-optimal trajectory algorithm for smoothing. The maximum joint speed of the robotic arm under no-load is set to 80% of the rated value. When it approaches the battery by 0.1 meters, the speed drops to 0.05m / s. The time for a single path planning is ≤200ms.

[0179] 3. The PLC issues trajectory instructions to control the robotic arm to move to the battery position according to the planned path. The force control module sets the contact force threshold to 4-6N. After stable contact for 0.2s, the adsorption and gripping are triggered to ensure undamaged gripping.

[0180] Step 32: Rechargeable battery box pose recognition and battery placement path planning

[0181] 1. The vision system identifies the three-dimensional coordinates of the empty compartments in the charging battery box and transmits them to the PLC;

[0182] 2. The PLC uses a spatial linear interpolation algorithm to plan the linkage path of the robotic arm from the drone to the battery box;

[0183] 3. The PLC-controlled robotic arm places the first battery to be charged into the designated compartment of the battery box according to the planned path. After the battery box sensor confirms that the battery is in place, it sends a message to the PLC that "battery insertion is complete".

[0184] Step 33: Fully charged battery capture path planning

[0185] 1. The rechargeable battery management system sends the coordinate data of the location of the fully charged battery compartment to the PLC;

[0186] 2. The PLC replans the robotic arm's grasping path, controls the robotic arm to grasp the fully charged battery, and waits for the next instruction after the grasp is completed.

[0187] Step 4: Install the first fully charged battery

[0188] 1. The vision system identifies the empty battery compartment pose coordinates of the drone and transmits them to the PLC;

[0189] 2. The PLC uses a spatial linear interpolation algorithm to plan the installation path and controls the robotic arm to deliver a fully charged battery into the empty drone bay;

[0190] 3. The vision system verifies that the battery is installed in place, and the robotic arm sends back a "installation complete" signal. After double confirmation, the first battery replacement is completed.

[0191] Step 5: Removing and installing the second battery (same as steps 3-4)

[0192] 1. The vision system identifies the pose coordinates of the second battery to be replaced on the drone and transmits them to the PLC via Profinet;

[0193] 2. The PLC repeats the path planning logic of step 3 three times to complete the actions of removing the second battery to be replaced, placing it into the charging battery box, and grabbing the fully charged battery.

[0194] 3. The vision system identifies the coordinates of the empty compartment of the second battery, the PLC plans the installation path, and controls the robotic arm to install the second fully charged battery into the empty compartment of the drone. The vision and robotic arm verify that the installation is in place.

[0195] Step 6: Battery lock off

[0196] 1. The vision system acquires the current posture coordinates of the battery lock and transmits them to the PLC;

[0197] 2. The PLC plans the locking path for the robotic arm, controls the robotic arm to perform the locking action, and monitors the torque of the locking motor in real time. If the torque exceeds the normal threshold and lasts for 2 seconds, the PLC immediately pauses the locking process, controls the robotic arm to push the battery to the correct position, replans the locking path, and executes it. If the locking failures accumulate to 3 times, the PLC triggers a manual assistance warning and suspends the process.

[0198] 3. After successful locking, the vision system verifies the lock body's closed state, the PLC confirms "locking in place" and marks "battery replacement complete".

[0199] Step 7: Drone takeoff and platform reset

[0200] 1. The PLC sends a "battery swap complete" signal to the drone, controlling the landing pad clamping device to release;

[0201] After the drone completes its self-check and takes off, the battery swapping platform triggers a reset process: the robotic arm returns to its initial position, the vision system is turned off, the PLC clears the data from this battery swap, and the platform returns to the "waiting for drone landing" standby state, ready for the next battery swapping task.

[0202] The above provides a detailed description of the fully automatic UAV battery replacement system and method based on binocular vision positioning proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A fully automated battery replacement method for drones based on binocular vision positioning, characterized in that, Includes the following steps: Step 1: After the drone lands on the helipad area, simultaneously acquire images of the drone's underside and the surrounding charging compartment area. Step 2: Perform pixel-level segmentation on the battery lock in the image pair acquired in Step 1, calculate the pose of the battery lock, transmit the pose of the battery lock to the lock switch mechanism to perform the unlocking action, after the unlocking is completed, acquire the image pair again, perform 3D reconstruction on the acquired image pair to obtain a dense 3D point cloud, and use a deep learning recognition and segmentation model to identify the corresponding UAV information. Step 3: Calculate the pose of the battery to be replaced and the pose of the idle charging compartment based on dense 3D point cloud and UAV information. Step 4: Obtain the current joint angle of the robotic arm, combine the pose of the battery to be replaced and the pose of the empty charging compartment, plan the operation trajectory to place the battery to be replaced into the empty charging compartment, and execute the operation trajectory. Step 5: Obtain the battery status in the charging compartment, calculate the pose of the fully charged battery based on the battery status, take a picture of the empty space in the drone's battery compartment, and calculate the pose of the current empty space in the drone's battery compartment. Step 6: Based on the pose of the fully charged battery and the pose of the empty space in the current drone battery compartment, plan the operation trajectory to install the fully charged battery into the empty space in the drone battery compartment, and execute the operation trajectory to complete the replacement of a single battery to be replaced. Step 7: Repeat steps 3 to 6 until all batteries to be replaced in the drone have been replaced. The method is based on a fully automated drone battery replacement system using binocular vision positioning. This fully automated drone battery replacement system includes the following modules: A binocular vision system is used to simultaneously acquire image pairs of the drone's underside area and the surrounding charging compartment area; The vision processing unit receives the battery status and the image pairs acquired by the binocular vision system, and performs the following: The battery lock in the acquired image pair is segmented at the pixel level, the pose of the battery lock is calculated, the pose of the battery lock is transmitted to the lock opening mechanism to perform the unlocking action, after the unlocking is completed, the binocular vision system is triggered to acquire the image pair again, and the re-acquired image pair is reconstructed in three dimensions to obtain a dense three-dimensional point cloud. A deep learning recognition and segmentation model is used to identify the corresponding UAV information. Based on dense 3D point cloud and UAV information, the pose of the battery to be replaced and the pose of the idle charging compartment are calculated. The path planning module is used to receive dense 3D point clouds and the pose of the target object, obtain the current joint angle of the robotic arm, and plan the operation trajectory. Robotic arm control is used to control the actuator to perform corresponding actions based on the work trajectory output by the path planning module; An actuator, wherein the end effector of the actuator integrates an adsorption gripper and a force sensor, for performing corresponding actions; A switch-lock mechanism is used to perform unlocking or locking actions based on battery lock position data. When the actuator performs an action, the central control unit and force sensor provide force feedback to confirm that the contact force during the action meets the preset standard.

2. The fully automatic UAV battery replacement method based on binocular vision positioning according to claim 1, characterized in that, Step 2 involves pixel-level segmentation of the battery lock in the image pair acquired in Step 1, calculation of the battery lock's pose, and control of the lock mechanism to perform the unlocking action. This includes the following steps: Step 21: Perform pixel-level segmentation on the image pairs to obtain the lock body region image; Step 22: Extract rotation-invariant feature points from the lock body region image, calculate the three-dimensional coordinates of the rotation-invariant feature points, and achieve coarse positioning of the battery lock; Step 23: Based on the coarse positioning results of the battery lock, perform edge detection on the lock body area image, extract the contour curves of the lock cylinder and the latch, fit the circular contour of the lock cylinder and the straight contour of the latch, and combine the PnP algorithm to optimize the pose calculation results to obtain the battery lock pose. Step 24: Transmit the battery lock position data to the lock switch mechanism and control the lock switch mechanism to perform the unlocking action.

3. The fully automatic UAV battery replacement method based on binocular vision positioning according to claim 1, characterized in that, The drone information in step 2 includes the drone model, the number of batteries to be replaced, and the current location coordinates of the batteries to be tested.

4. The fully automatic UAV battery replacement method based on binocular vision positioning according to claim 1, characterized in that, Step 3, based on dense 3D point cloud and UAV information, calculates the pose of the battery to be replaced and the pose of the idle charging compartment, including the following steps: Step 31: Based on the drone model in the drone information, obtain the standard feature point set of the battery to be tested for the corresponding drone model; Step 32: In the dense 3D point cloud and the re-acquired image pair, identify and extract the image coordinates of at least 4 non-coplanar key feature points on the battery under test to obtain the current feature point set; Step 33: Solve for the optimal spatial transformation between the standard feature point set and the current feature point set, and perform dynamic error compensation on the real-time calculated battery pose data to obtain the pose of the battery to be replaced. Step 34: Identify the charging compartment that is in an idle state, extract the three-dimensional coordinates of the preset positioning feature points of the charging compartment, calculate the pose of the center point of the placement reference surface of the charging compartment in the visual coordinate system, and obtain the pose of the idle charging compartment.

5. The fully automatic battery replacement method for unmanned aerial vehicles based on binocular vision positioning according to claim 1, characterized in that, Step 4 involves planning and executing a work trajectory for placing the battery to be replaced into an empty charging compartment, including the following steps: Step 41: Use the path planning module to generate the joint space trajectory from the current position of the actuator to the pose of the battery to be replaced; Step 42: Control the actuator to execute the joint space trajectory, grab the battery to be replaced, and determine through force feedback that the force of grabbing the battery to be replaced meets the preset standard. Step 43: Use a binocular vision system to confirm the pose of the idle charging compartment, use the path planning module to generate a placement trajectory from the pose of the battery to be replaced to the pose of the idle charging compartment, control the actuator to execute the placement trajectory, and use force feedback to determine that the force applied to place the battery to be replaced meets the preset standard.

6. The fully automatic UAV battery replacement method based on binocular vision positioning according to claim 5, characterized in that, The force feedback is specifically as follows: When the force sensor of the actuator detects that the contact force reaches the preset standard, it triggers the suction gripper of the actuator to grasp or release the battery.

7. The fully automatic UAV battery replacement method based on binocular vision positioning according to claim 4, characterized in that, Step 32 uses a hybrid feature extraction method to extract at least four non-coplanar key feature points on the battery under test, and combines them with edge contour fitting to obtain the current feature point set.

8. The fully automatic battery replacement method for unmanned aerial vehicles based on binocular vision positioning according to claim 1, characterized in that, The method further includes a data transmission step, specifically: The battery pose to be replaced and the pose of the idle charging compartment calculated in step 3 are encoded into data frames via the Profinet Ethernet protocol and transmitted to the PLC. After the PLC verifies and decodes the data frames, they are stored in the PLC's storage area. Based on the stored battery pose to be replaced and idle charging compartment pose, the PLC calls the S-curve acceleration / deceleration algorithm and the spatial linear interpolation algorithm to generate the XYZU four-axis linkage operation trajectory. The operation trajectory is then sent to the actuator driver via PROFINET IRT isochronous synchronization, controlling the actuator to execute the operation trajectory.

9. The fully automatic battery replacement method for unmanned aerial vehicles based on binocular vision positioning according to claim 1, characterized in that, After all batteries have been replaced, the binocular vision system is activated to identify the real-time position of the drone battery lock and transmit the real-time position to the locking mechanism. The locking mechanism is then controlled to perform the locking action. During the locking process, the torque of the motor of the locking mechanism is monitored in real time. If the torque exceeds a preset threshold and the time exceeding the preset threshold reaches a preset time, it is considered a locking failure, and the locking task is stopped. The robotic arm is then controlled to advance the drone battery's Y-axis by 1mm, and the locking mechanism is controlled to perform the locking action again. If the locking action fails more than 3 times, a manual assistance warning is automatically triggered and the subsequent process is suspended.

Images