General battery replacement system and method based on visual identification

Abstract

The invention relates to the technical field of automatic battery replacement, and discloses a general battery replacement system and method based on visual identification. The method comprises the following steps: synchronously capturing images of a vehicle entrance, a battery compartment carrier and a vehicle chassis through multiple groups of visual equipment, and synchronously analyzing and extracting three-dimensional space coordinates of a vehicle outline, a carrier marking point and a chassis battery joint point; the coordinates are fused to generate system-level global positioning and attitude compensation parameters, and the parameters guide the macroscopic movement of the battery transfer device and the real-time pose fine adjustment of the tail end clamping tool at the same time, so that the grabbing, transfer and accurate installation of the battery pack are completed. According to the method, the unified space reference is constructed and applied to the whole operation process, the adaptability of the system to the pose deviation of the vehicle and the carrier is enhanced, and automatic battery replacement operation with higher precision and reliability is achieved.

Description

Technical Field

[0001] This invention relates to the field of automated battery replacement technology, specifically to a universal battery replacement system and method based on vision recognition. Background Technology

[0002] Currently, most automated battery swapping systems employ a guidance method based on preset locations. A common approach requires the vehicle to be precisely parked at a fixed landmark, with the equipment operating according to a pre-programmed trajectory. Another approach uses a single visual marker placed in the battery compartment or on the vehicle chassis to guide the actuators for local alignment. All these technologies are based on the assumption that the relative positions of the vehicle, carrier, and actuators are fixed and known.

[0003] The drawback of existing technologies lies in their discrete and isolated sensing and positioning. The system struggles to simultaneously acquire and process the real-time, dynamic spatial relationships between the overall vehicle outline, the battery compartment platform, and the chassis battery connection points. When the vehicle is parked with positional or angular deviations, or when the battery compartment platform is not parked in a standard posture, the discrete positioning information cannot form a unified environmental model, resulting in insufficient fault tolerance and adaptability of the system.

[0004] A technology is needed to collaboratively process visual information from multiple perspectives and calculate the precise three-dimensional spatial coordinates of multiple key entities in the work environment in real time. Furthermore, based on this fused spatial information, a unified pose reference needs to be generated that can guide the entire process from macroscopic transport to fine-grained end-point operations, in order to address the inevitable position and attitude deviations in practical applications and achieve reliable and accurate battery replacement operations. Summary of the Invention

[0005] The purpose of this invention is to provide a universal battery replacement system and method based on visual recognition to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a universal battery replacement method based on visual recognition, the method comprising: Initialize multiple sets of vision acquisition devices in the general battery replacement system to capture images of the entrance scene of the vehicle to be replaced, the layout image of the battery compartment vehicle that is already in place, and the battery pack installation image of the vehicle chassis area. The entrance scene image, the layout image, and the battery pack installation image are analyzed simultaneously to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack joint points. Based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack connection points, system-level global positioning and attitude compensation parameters are generated. Based on the system-level global positioning and attitude compensation parameters, the battery transfer device is driven to move to the preset battery grabbing preparation position. The battery gripping tool located at the end of the battery transfer device is controlled to perform fine-tuning of its position and posture based on the system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data, thereby completing the gripping, transfer and installation operations of the battery pack.

[0007] Preferably, the step of simultaneously analyzing the entrance scene image, the layout image, and the battery pack installation image to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack connection points includes: Edge contour detection is performed on the entrance scene image to identify the vehicle's outer boundary. The identified vehicle's outer boundary is then transformed into the world coordinate system to generate the spatial coordinates of the vehicle's outline. The layout image is subjected to preset pattern recognition to locate multiple optical marks pasted on the surface of the battery compartment carrier, and the precise position of each optical mark in three-dimensional space is calculated as the spatial coordinates of the carrier mark point; Feature point matching is performed on the battery pack installation image to identify the key features of the preset battery pack joining structure on the chassis, and the three-dimensional position of these key features is calculated based on the principle of binocular vision as the spatial coordinates of the chassis battery pack joining point.

[0008] Preferably, the step of generating system-level global positioning and attitude compensation parameters based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack junction points includes: The spatial coordinates of the vehicle outline are compared with the pre-stored vehicle model database to determine the pose deviation of the vehicle relative to the standard entry position of the universal battery swapping system, and the vehicle pose compensation amount is generated. Based on the spatial coordinates of the vehicle's marker points, the current flatness and horizontal tilt angle of the battery compartment vehicle are calculated, and combined with the vehicle's design position, the vehicle's attitude compensation amount is generated. By integrating the spatial coordinates of the chassis battery pack junction point, the vehicle pose compensation amount, and the vehicle attitude compensation amount, system-level global positioning and attitude compensation parameters are calculated through coordinate transformation to guide the precise movement of the battery transfer device and battery clamping tool.

[0009] Preferably, the step of driving the battery transfer device to a preset battery grasping preparation position based on the system-level global positioning and attitude compensation parameters includes: The system-level global positioning and attitude compensation parameters are input into the motion controller of the battery transfer device; The motion controller calculates the motion trajectory of each joint of the battery transfer device based on the target point coordinates and desired attitude contained in the system-level global positioning and attitude compensation parameters. The servo motors of each joint of the battery transfer device run according to the calculated motion trajectory, so that the end flange of the battery transfer device accurately reaches the preset battery gripping preparation position.

[0010] Preferably, during the movement of the battery transfer device, a real-time pose verification step is also included: Images of the calibration plate on the end flange of the battery transfer device are continuously acquired by vision sensors installed on the base and joints of the battery transfer device. The actual pose of the end flange of the battery transfer device is calculated in real time from continuously acquired images. The actual pose of the end flange of the battery transfer device is compared with the theoretical pose planned according to the system-level global positioning and attitude compensation parameters. If the deviation exceeds the set threshold, the motion is paused and the system-level global positioning and attitude compensation parameters are updated according to the deviation value. The motion trajectory is then replanned based on the updated system-level global positioning and attitude compensation parameters.

[0011] Preferably, the control is set at the battery clamping tool at the end of the battery transfer device, so that it performs fine-tuning of its posture based on the system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data, to complete the steps of gripping, transferring and installing the battery pack, including: After the battery transfer device reaches the preset battery gripping preparation position, the close-range vision sensor integrated on the battery clamping tool is activated to scan the surface features of the battery pack to be replaced located in the battery compartment carrier. Based on the scanning results of the near-field vision sensor, the position of the battery pack gripping point in the system-level global positioning and attitude compensation parameters is corrected with millimeter-level precision to generate gripping point fine-tuning parameters. Based on the fine-tuning parameters of the gripping point, the battery clamping tool is controlled to move to the corrected precise gripping point; During the gripping process, data from the multi-dimensional force sensor and displacement sensor on the battery gripper's jaws are read in real time, and the closing force and position of the jaws are dynamically adjusted to ensure stable gripping without damaging the battery pack.

[0012] Preferably, the process of gripping, transferring, and installing the battery pack specifically includes three sub-steps: gripping, transferring, and installing. In the gripping sub-step, the battery gripping tool successfully grips the battery pack to be replaced based on the gripping point fine-tuning parameters and dynamic adjustment results, and then uploads the gripping success signal and the final gripper pose data during gripping. In the transfer sub-step, the battery transfer device plans an obstacle avoidance path from the battery compartment carrier to the vehicle chassis based on the received successful grab signal, and controls the battery clamping tool to carry the battery pack along the obstacle avoidance path, while continuously monitoring the attitude stability of the battery pack. During the installation sub-step, when the battery pack is transported to the vicinity of the target installation position on the vehicle chassis, the close-range vision sensor on the battery clamping tool is activated again to make a final alignment confirmation of the battery pack engagement point on the vehicle chassis, and based on the confirmation result, a final straight insertion action is performed to complete the installation of the battery pack.

[0013] Preferably, the step of calculating the precise position of each optical mark in three-dimensional space as the spatial coordinates of the vehicle mark point specifically includes: Acquire the layout image containing the optical markers, which is simultaneously captured from different perspectives by multiple sets of visual acquisition devices; Each of the layout images is preprocessed to enhance the pattern features of the optical markers and the image plane coordinates of each optical marker are identified; Based on the intrinsic and extrinsic parameter matrices of multiple vision acquisition devices obtained through pre-calibration, stereo matching is performed on the image plane coordinates of the same identified optical mark in different layout images. Based on the results of stereo matching, the forward intersection algorithm is used to calculate the three-dimensional coordinates of each optical mark in the world coordinate system, which are used as the precise position of the optical mark in three-dimensional space, i.e., the spatial coordinates of the vehicle mark point.

[0014] Preferably, the step of calculating the current flatness and horizontal tilt angle of the battery compartment vehicle based on the spatial coordinates of the vehicle marker points, and generating the vehicle attitude compensation amount in combination with the vehicle's design position, specifically includes: Select the spatial coordinates of at least three non-collinear vehicle marker points, and use the least squares method to fit a reference plane equation representing the actual plane of the battery compartment vehicle; Calculate the angle between the reference plane equation and the horizontal plane, and use it as the current horizontal tilt angle of the battery compartment vehicle; Calculate the distances from the spatial coordinates of all the vehicle marker points used for fitting to the reference plane, and use the maximum difference as the current flatness of the battery compartment vehicle; The current horizontal tilt angle and the current flatness are compared with the theoretical horizontal tilt angle and theoretical flatness corresponding to the pre-stored design position of the battery compartment vehicle. The translation and rotation amounts used to correct the deviation between the actual position and the design position of the battery compartment vehicle are calculated, and the vehicle attitude compensation amount is generated.

[0015] Preferably, the present invention also includes a universal battery replacement system based on visual recognition, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the universal battery replacement method based on visual recognition as described above.

[0016] Compared with the prior art, the beneficial effects of the present invention are: Multiple sets of visual acquisition devices simultaneously capture and analyze images of the entrance scene, battery compartment vehicle layout, and vehicle chassis area, extracting the spatial coordinates of each target. This enables real-time, parallel perception and modeling of multiple key entities in a unified three-dimensional space within the working environment. Compared to conventional techniques that rely on a single beacon or fixed procedures, it constructs a complete environmental spatial model including vehicle outlines, vehicle positions, and chassis engagement points. This allows the system to accurately understand and quantify the real-time relative positions and attitudes between the vehicle, the vehicle, and the engagement points, providing a direct data foundation for addressing vehicle parking deviations and vehicle position changes, and enhancing the system's general adaptability to different working conditions.

[0017] Based on the extracted multiple sets of spatial coordinates, system-level global positioning and attitude compensation parameters are generated and used to guide the entire process from the macroscopic movement of the battery transfer device to the fine-tuning of the end-effector. This establishes a unified spatial reference system and compensation framework throughout the entire process. In conventional technologies, each movement stage often relies on independent and potentially inconsistent positioning references, leading to error propagation. This approach ensures that the reference for large-scale movement and the reference for fine-tuning originate from the same set of fused, high-precision spatial alignment parameters. This makes the macroscopic positioning of the mechanical system and the end-effector fine-tuning actions spatially logically continuous and consistent, reducing the cumulative error in multiple operation stages and improving the overall accuracy and collaborative reliability of the entire battery gripping, transfer, and installation process. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the working principle of the universal battery replacement method based on visual recognition described in this invention. Figure 2 The flowchart for image synchronization analysis and coordinate extraction; Figure 3 A flowchart for generating system-level global positioning and attitude compensation parameters. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 This invention provides a universal battery replacement method based on visual recognition. The method includes: acquiring scene information through multiple sets of visual acquisition devices, generating precise motion guidance parameters through synchronous analysis and calculation, and finally driving an actuator to complete the battery replacement. The method first initializes multiple sets of visual acquisition devices in the universal battery replacement system. These devices capture images of the entrance scene of the vehicle to be replaced, the layout of the positioned battery compartment carrier, and the battery pack installation image in the vehicle chassis area. Then, the entrance scene image, layout image, and battery pack installation image are synchronously analyzed to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the carrier marker points, and the spatial coordinates of the chassis battery pack connection points. Based on the extracted spatial coordinates of the vehicle outline, the carrier marker points, and the chassis battery pack connection points, system-level global positioning and attitude compensation parameters are generated. Based on these system-level global positioning and attitude compensation parameters, the battery transfer device is driven to move to a preset battery gripping preparation position. Finally, the battery clamping tool at the end of the battery transfer device is controlled to perform fine-tuning of its posture based on the system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data, completing the gripping, transfer, and installation of the battery pack.

[0021] In one embodiment of the present invention, see [reference] Figure 2The system simultaneously analyzes entrance scene images, layout images, and battery pack installation images to extract the spatial coordinates of the vehicle outline, vehicle marker points, and chassis-battery pack junction points. Edge contour detection is performed on the entrance scene image to identify the vehicle's external boundaries. These boundaries are then transformed to the world coordinate system to generate the vehicle's spatial coordinates. Preset pattern recognition is performed on the layout image to locate multiple optical markers affixed to the surface of the battery compartment vehicle. The precise position of each optical marker in three-dimensional space is calculated as the spatial coordinates of the vehicle marker points. Calculating the precise position of each optical marker in three-dimensional space involves acquiring layout images containing the optical markers, simultaneously captured from different perspectives by multiple sets of vision acquisition devices. Each layout image is preprocessed to enhance the pattern features of the optical markers and identify their image plane coordinates. Based on the pre-calibrated intrinsic and extrinsic parameter matrices of the multiple sets of vision acquisition devices, stereo matching is performed on the image plane coordinates of the same optical marker in different layout images. Based on the stereo matching results, a forward intersection algorithm is used to calculate the three-dimensional coordinates of each optical marker in the world coordinate system, which serves as the precise position of the optical marker in three-dimensional space, i.e., the spatial coordinates of the vehicle marker point. Feature point matching is performed on the battery pack installation images to identify the key features of the pre-set battery pack joining structure on the chassis. The three-dimensional positions of these key features are calculated based on the principle of binocular vision, serving as the spatial coordinates of the chassis-battery pack joining point.

[0022] In practice, the entrance scene image, layout image, and battery pack installation image are analyzed simultaneously to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack joint points. Edge contour detection is performed on the entrance scene image to identify the vehicle's external boundaries. These boundaries are then transformed into the world coordinate system to generate the spatial coordinates of the vehicle outline. Feature point matching is performed on the battery pack installation image to identify key features of the pre-defined battery pack joint structure on the chassis. The three-dimensional positions of these key features are calculated based on binocular vision principles to serve as the spatial coordinates of the chassis battery pack joint points.

[0023] In some embodiments, multiple optical markers are positioned and pasted onto the surface of the battery compartment carrier by pre-defined pattern recognition of the layout image. The precise position of each optical marker in three-dimensional space is calculated as the spatial coordinates of the carrier marker point. The calculation process involves acquiring layout images containing optical markers simultaneously captured from different perspectives by multiple sets of visual acquisition devices. Each layout image is pre-processed to enhance the pattern features of the optical markers and identify the image plane coordinates of each optical marker. Based on the intrinsic and extrinsic parameter matrices of the multiple sets of visual acquisition devices obtained through pre-calibration, stereo matching is performed on the image plane coordinates of the same optical marker in different layout images. In a specific implementation, based on the stereo matching results, a forward intersection algorithm is used to calculate the three-dimensional coordinates of each optical marker in the world coordinate system, which serves as the precise position of the optical marker in three-dimensional space, i.e., the spatial coordinates of the carrier marker point. For a stereo vision system consisting of two perspectives, the formula for solving the three-dimensional coordinates of the optical marker point using forward intersection is expressed as: in: Indicates the first An optical marker in the world coordinate system The three-dimensional position coordinate vector below, This represents the forward intersection function that calculates 3D coordinates based on image coordinates and projection matrices from two viewpoints. and Indicates the first The horizontal and vertical coordinates of the pixels of an optical marker in the layout image captured by the first visual acquisition device. and Indicates the first The horizontal and vertical coordinates of the pixels of the optical marker in the layout image captured by the second visual acquisition device. This represents the projection matrix from the world coordinate system to the image coordinate system of the first visual acquisition device. This represents the projection matrix from the world coordinate system to the image coordinate system of the second visual acquisition device.

[0024] Optionally, the entrance scene image is captured by a vision acquisition device installed above the battery swapping channel entrance, the layout image is captured synchronously by two sets of vision acquisition devices deployed above the battery compartment vehicle area, and the battery pack installation image is captured by a set of binocular vision acquisition devices mounted on a movable gimbal. It is understood that the acquisition and analysis of the entrance scene image, layout image, and battery pack installation image are synchronized in time or closely linked in a preset order, thereby ensuring that the extracted spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack joint points have a consistent time reference. In some embodiments, the transformation of the vehicle's external boundary to the world coordinate system relies on high-precision hand-eye calibration parameters pre-completed by the vision acquisition device. These hand-eye calibration parameters establish a definite transformation relationship between the vision acquisition device's image coordinate system and the world coordinate system. Optionally, when performing feature point matching on the battery pack installation image, the key features used include, but are not limited to, the center of the bolt holes, the edge corners of the locating pins, or the corners of preset QR code markers.

[0025] In one embodiment of the present invention, see [reference] Figure 3 The system generates system-level global positioning and attitude compensation parameters based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack junction points. The spatial coordinates of the vehicle outline are compared with a pre-stored vehicle model database to determine the vehicle's pose deviation relative to the standard entry position of the universal battery swapping system, generating vehicle pose compensation values. Based on the spatial coordinates of the vehicle marker points, the current flatness and tilt angle of the battery compartment vehicle are calculated, and combined with the vehicle's designed position, vehicle attitude compensation values ​​are generated. The process of calculating the current flatness and horizontal tilt angle of the battery compartment carrier and generating the carrier attitude compensation amount involves selecting the spatial coordinates of at least three non-collinear carrier marker points, fitting a reference plane equation representing the actual plane of the battery compartment carrier using the least squares method, calculating the angle between the reference plane equation and the horizontal plane as the current horizontal tilt angle of the battery compartment carrier, calculating the maximum difference between the spatial coordinates of all carrier marker points used for fitting and the distance to the reference plane as the current flatness of the battery compartment carrier, comparing the current horizontal tilt angle and current flatness with the theoretical horizontal tilt angle and theoretical flatness corresponding to the pre-stored design position of the battery compartment carrier, and calculating the translation and rotation amounts used to correct the deviation between the actual position and the design position of the battery compartment carrier to generate the carrier attitude compensation amount. By integrating the spatial coordinates of the chassis battery pack junction point, the vehicle pose compensation amount, and the carrier attitude compensation amount, system-level global positioning and attitude compensation parameters are calculated through coordinate transformation to guide the precise movement of the battery transfer device and battery clamping tool.

[0026] In practice, system-level global positioning and attitude compensation parameters are generated based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack junction points. The spatial coordinates of the vehicle outline are compared with a pre-stored vehicle model database to determine the vehicle's pose deviation relative to the standard entry position of the universal battery swapping system, generating vehicle pose compensation values. Based on the spatial coordinates of the vehicle marker points, the current flatness and tilt angle of the battery compartment vehicle are calculated, and combined with the vehicle's designed position, vehicle attitude compensation values ​​are generated. Select the spatial coordinates of at least three non-collinear vehicle marker points. Use the least squares method to fit a reference plane equation representing the actual plane of the battery compartment vehicle. Calculate the angle between the reference plane equation and the horizontal plane as the current horizontal tilt angle of the battery compartment vehicle. Calculate the maximum difference between the distances of all the spatial coordinates of the vehicle marker points used for fitting to the reference plane as the current flatness of the battery compartment vehicle. Compare the current horizontal tilt angle and current flatness with the pre-stored theoretical horizontal tilt angle and theoretical flatness corresponding to the design position of the battery compartment vehicle to calculate the translation and rotation amounts used to correct the deviation between the actual and design positions of the battery compartment vehicle, generating the vehicle attitude compensation amount. The mathematical expression of the vehicle attitude compensation amount involves the compensation translation vector and the compensation rotation matrix, and their calculation relationship can be expressed as: in: This represents the three-dimensional translation vector in the vehicle attitude compensation. This represents the three-dimensional rotation matrix in the vehicle attitude compensation. This represents the process of calculating translation and rotation amounts using the coordinates of the marked points and design parameters. This represents the spatial coordinate vector of the selected n vehicle marker points in the world coordinate system. This indicates the theoretical horizontal tilt angle corresponding to the designed location of the battery compartment vehicle. This indicates the theoretical flatness corresponding to the designed location of the battery compartment carrier.

[0027] In some embodiments, the vehicle model database contains standard 3D contour models and standard entry poses for various vehicle models. It can be understood that the vehicle pose compensation includes the lateral and longitudinal offsets of the vehicle in the horizontal plane, as well as the yaw angle deviation of the vehicle around the vertical axis. In specific implementations, the fusion of the spatial coordinates of the chassis battery pack junction point, the vehicle pose compensation, and the vehicle attitude compensation is achieved through a series of coordinate transformations. The coordinate transformation sequentially transforms the spatial coordinates of the chassis battery pack junction point from the vehicle coordinate system to an intermediate coordinate system based on the standard entry position, and then further transforms it to the battery compartment vehicle coordinate system, which incorporates the vehicle attitude compensation, ultimately obtaining the precise target pose used to guide motion in the system's global coordinate system. Optionally, the system-level global positioning and attitude compensation parameters are ultimately expressed as the target pose matrix of the end flange of the battery transfer device and the pose adjustment of the battery clamping tool relative to the target gripping point.

[0028] In some embodiments, the calculation of vehicle attitude compensation also considers torsional deformation of the battery compartment vehicle due to long-term use or uneven ground. Torsional deformation is identified by analyzing the patterns of spatial coordinate deviations of multiple non-coplanar vehicle marker points from the fitted plane. It can be understood that the introduction of vehicle attitude compensation can offset the difference between the actual placement position and the ideal design position of the battery compartment vehicle, ensuring the accuracy of the subsequent grasping operation reference. Optionally, the generation process of vehicle pose compensation may call a point cloud registration algorithm to match the vehicle contour point cloud extracted from the entrance scene image with the standard model point cloud in the vehicle model database, thereby calculating the translation and rotation deviations. In the process of generating vehicle pose compensation, the specific implementation of the point cloud registration algorithm is as follows: First, the vehicle contour point cloud is obtained by edge contour detection of the entrance scene image. This point cloud consists of a set of spatial coordinate points transformed from the vehicle's outer boundary to the world coordinate system. Second, the standard 3D contour model point cloud corresponding to the vehicle to be replaced is retrieved from the pre-stored vehicle model database. This standard model point cloud defines the ideal contour coordinates of the vehicle at the standard entrance position. The point cloud registration algorithm then inputs the above two point clouds and iteratively optimizes and calculates an optimal spatial transformation matrix. This matrix contains translation and rotation components, so that the extracted vehicle contour point cloud and the standard model point cloud achieve optimal alignment in the least squares sense. After registration, the algorithm directly calculates the translation and rotation deviations of the vehicle's actual position relative to the standard entrance position from this spatial transformation matrix. These deviation values ​​are the core components of the vehicle pose compensation.

[0029] In one embodiment of the present invention, the battery transport device is driven to move to a preset battery gripping preparation position based on system-level global positioning and attitude compensation parameters. The system-level global positioning and attitude compensation parameters are input into the motion controller of the battery transport device. The motion controller calculates the motion trajectory of each joint of the battery transport device based on the target point coordinates and desired posture contained in the system-level global positioning and attitude compensation parameters, and controls the servo motors of each joint of the battery transport device to run according to the calculated motion trajectory, so that the end flange of the battery transport device accurately reaches the preset battery gripping preparation position. The movement of the battery transport device also includes a real-time pose verification step. Images of the calibration plate on the end flange of the battery transport device are continuously acquired by vision sensors installed on the base and joints of the battery transport device. The actual pose of the end flange of the battery transport device is calculated in real time from the continuously acquired images. The actual pose of the end flange of the battery transport device is compared with the theoretical pose planned according to the system-level global positioning and attitude compensation parameters. If the deviation exceeds a set threshold, the movement is paused, and the system-level global positioning and attitude compensation parameters are updated according to the deviation value. The motion trajectory is then replanned based on the updated system-level global positioning and attitude compensation parameters.

[0030] In practice, the battery transfer device is driven to move to the preset battery gripping preparation position based on system-level global positioning and attitude compensation parameters. The system-level global positioning and attitude compensation parameters are input into the motion controller of the battery transfer device. The motion controller calculates the motion trajectory of each joint of the battery transfer device based on the target point coordinates and desired attitude contained in the system-level global positioning and attitude compensation parameters, and controls the servo motors of each joint of the battery transfer device to run according to the calculated motion trajectory, so that the end flange of the battery transfer device accurately reaches the preset battery gripping preparation position.

[0031] In practical implementation, a real-time pose verification step is also included during the movement of the battery transport device. Vision sensors installed on the base and joints of the battery transport device continuously acquire images of the calibration plate on the end flange of the battery transport device. The actual pose of the end flange of the battery transport device is calculated in real time from the continuously acquired images, and then compared with the theoretical pose planned based on system-level global positioning and attitude compensation parameters. The calculation of pose deviation involves quantifying the differences between the actual pose and the theoretical pose in translation and rotation components, and its mathematical expression is as follows: in: This represents the translational deviation vector between the actual and theoretical poses of the end flange of the battery transfer device. This represents the rotational deviation vector between the actual and theoretical positions of the end flange of the battery transfer device. This represents a function that calculates the translational and rotational differences between the actual pose and the theoretical pose. This represents the actual pose matrix of the end flange of the battery transfer device, obtained through real-time image processing. This represents the theoretical pose matrix of the end flange of the battery transfer device, generated based on system-level global positioning and attitude compensation parameters.

[0032] In some embodiments, vision sensors mounted on the battery transport device base and joints constitute an independent vision measurement network. This network calculates the pose by observing the high-contrast calibration plate pattern on the end flange. It can be understood that if the translational deviation vector... Magnitude or rotational deviation vector If the deviation angle exceeds a set threshold, the movement of the battery transfer device is paused, and the system-level global positioning and attitude compensation parameters are updated based on the deviation value. The motion trajectory is then replanned based on the updated system-level global positioning and attitude compensation parameters. In specific implementations, the motion controller uses a polynomial interpolation algorithm to calculate the motion trajectory of each joint. The polynomial interpolation algorithm generates smooth velocity and acceleration curves in the joint space. Optionally, the preset battery gripping preparation position is a fixed spatial pose relative to a specific optical mark on the battery compartment carrier. The system-level global positioning and attitude compensation parameters will transform this fixed pose to the world coordinate system of the current actual scene. In some embodiments, the running frequency of the real-time pose verification step is higher than the position loop control frequency of the motion controller. The running frequency ensures that pose deviations caused by joint gaps, transmission errors, or external interference can be detected and corrected in a timely manner. It can be understood that when replanning the motion trajectory, the motion controller will use the actual pose of the current pause point as the new path starting point and the preset battery gripping preparation position as the ending point, and perform trajectory interpolation calculation again.

[0033] In one embodiment of the present invention, a battery gripping tool located at the end of a battery transfer device is controlled to perform pose fine-tuning based on system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data to complete the gripping, transfer, and installation operations of the battery pack. After the battery transfer device reaches the preset battery gripping preparation position, the close-range vision sensor integrated on the battery gripping tool is activated to scan the surface features of the battery pack to be replaced located in the battery compartment carrier. Based on the scanning results of the close-range vision sensor, the position of the battery pack gripping point in the system-level global positioning and attitude compensation parameters is corrected with millimeter-level precision to generate gripping point fine-tuning parameters. Based on the gripping point fine-tuning parameters, the battery gripping tool is controlled to move to the corrected precise gripping point. During the gripping process, data from the multi-dimensional force sensor and displacement sensor on the gripper of the battery gripping tool are read in real time to dynamically adjust the closing force and position of the gripper to ensure stable gripping without damaging the battery pack.

[0034] In practical implementation, the battery gripping tool at the end of the battery transfer device is controlled to perform pose fine-tuning based on system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data to complete the gripping, transfer, and installation of the battery pack. After the battery transfer device reaches the preset battery gripping preparation position, the close-range vision sensor integrated on the battery gripping tool is activated to scan the surface features of the battery pack to be replaced located in the battery compartment carrier. The close-range vision sensor scans and acquires three-dimensional point cloud data of the surface feature points of the battery pack. Based on the scanning results of the close-range vision sensor, the position of the battery pack gripping point in the system-level global positioning and attitude compensation parameters is corrected with millimeter-level precision to generate gripping point fine-tuning parameters. The calculation process of the gripping point fine-tuning parameters involves registering the scanned point cloud with the ideal model of the battery pack gripping point, and its mathematical expression is as follows: in: This represents the position correction vector in the fine-tuning parameters of the capture point. This represents the attitude correction vector in the fine-tuning parameters of the grab point. This represents a registration algorithm function that calculates pose corrections based on scanned point clouds, an ideal model, and the initial capture point coordinates. This represents a set of three-dimensional point cloud data of feature points on the surface of the battery pack obtained by near-field visual sensor scanning. This represents the pre-stored ideal 3D model data of the gripping surface of the battery pack to be replaced. This represents the initial gripping point coordinates determined based on system-level global positioning and attitude compensation parameters.

[0035] Based on the fine-tuning parameters of the gripping point, the battery clamping tool is controlled to move to the corrected precise gripping point. During the gripping process, data from the multi-dimensional force sensors and displacement sensors on the gripper's jaws are read in real time to dynamically adjust the closing force and position of the jaws. The logic for reading and adjusting the jaw sensor data corresponds to the relationship defined in the table below, see Table 1: Table 1: Correspondence between Gripper Sensor Data and Adjustment In some embodiments, the near-field vision sensor employs a structured light 3D scanner, which can acquire high-precision point clouds of the battery pack latch or positioning slot area within hundreds of milliseconds. It is understood that the fine-tuning parameters of the grasping points are crucial. and The generation of this technology improves the accuracy of system-level global positioning and attitude compensation parameters from the centimeter level to the millimeter level. In specific implementations, dynamically adjusting the closing force and position of the grippers is a closed-loop control process. This closed-loop control process uses feedback from multi-dimensional force sensors and displacement sensors as control inputs, and the torque and position commands of the gripper servo motors as control outputs. Optionally, the precise gripping point is defined on the symmetrically arranged grippers on both sides of the battery pack. The battery clamping tool controls the left and right grippers to move independently and synchronously to their respective precise gripping points. In some embodiments, the displacement sensors integrated on the grippers are magnetostrictive linear displacement sensors, which measure the absolute displacement of the gripper linear module. It can be understood that real-time reading of the data from the multi-dimensional force sensors and displacement sensors on the battery clamping tool grippers can detect abnormal states such as deformation, slippage, or failure to be clamped to the predetermined position of the battery pack.

[0036] In one embodiment of the present invention, the battery pack gripping, transfer, and installation operation specifically includes three sub-steps: gripping, transfer, and installation. In the gripping sub-step, the battery clamping tool successfully grips the battery pack to be replaced based on the gripping point fine-tuning parameters and dynamic adjustment results, and then uploads a gripping success signal and the final gripper pose data during gripping. In the transfer sub-step, the battery transfer device plans an obstacle avoidance path from the battery compartment carrier to the vehicle chassis based on the received gripping success signal and controls the battery clamping tool to carry the battery pack along the obstacle avoidance path while continuously monitoring the battery pack's attitude stability. In the installation sub-step, when the battery pack is transported to the vicinity of the target installation position on the vehicle chassis, the near-field vision sensor on the battery clamping tool is activated again to perform a final alignment confirmation of the battery pack engagement point on the vehicle chassis, and based on the confirmation result, a final straight-line insertion action is performed to complete the battery pack installation.

[0037] In practice, the battery pack gripping, transfer, and installation operations consist of three sub-steps: gripping, transfer, and installation. In the gripping sub-step, the battery clamping tool successfully grips the battery pack to be replaced based on fine-tuned parameters and dynamic adjustments at the gripping point. It then uploads a successful gripping signal and the final gripper pose data at the moment of gripping. The successful gripping signal is sent as a digital signal to the motion controller of the battery transfer device. The final gripper pose data includes the position and attitude matrix of the battery clamping tool's end face relative to the world coordinate system at the instant the gripping is completed. In the transfer sub-step, the battery transfer device plans an obstacle avoidance path from the battery compartment carrier to the vehicle chassis based on the received successful gripping signal and controls the battery clamping tool to move along the obstacle avoidance path while continuously monitoring the battery pack's attitude stability. The obstacle avoidance path consists of a series of path points. The generation of these path points considers the workspace constraints of the battery transfer device, the known locations of fixed obstacles, and the safety margin of the battery pack during transfer. The smoothness of the path is optimized using a parametric curve, mathematically expressed as: in: Represented by normalized parameters The described smooth spatial path curve, This represents the normalization parameter of the path curve, with values ​​ranging from 0 to 1 corresponding to the path's origin and destination. Denotes the binomial coefficient. This represents the three-dimensional coordinate vector of the k-th control point in the world coordinate system, determined by the path planning algorithm. This indicates the total number of control points used to define the curve.

[0038] In the installation sub-step, when the battery pack is transported to the vicinity of the target installation position on the vehicle chassis, the near-field vision sensor on the battery clamping tool is activated again to perform final alignment confirmation of the battery pack engagement point on the vehicle chassis. Based on the confirmation result, a final straight-line insertion action is performed to complete the installation of the battery pack. In some embodiments, the successful gripping signal and the final gripper pose data during gripping are uploaded to the system main controller in real time via an industrial Ethernet bus. In specific implementations, the path point sequence used when planning the obstacle avoidance path is pre-simulated for collision detection. The collision detection simulation ensures that the battery transport device, battery clamping tool, and carried battery pack do not interfere with the battery compartment carrier, vehicle chassis, and surrounding environment during movement. Optionally, the attitude stability of the battery pack is monitored by comparing the real-time attitude angle output by the inertial measurement unit with the preset desired attitude angle during path planning. If the attitude angle deviation exceeds a safety threshold, deceleration or pausing is immediately triggered. In some embodiments, the final alignment confirmation process for the battery pack engagement point on the vehicle chassis calculates the remaining position and angle deviation between the positioning features on the battery pack and the chassis engagement point features. The remaining deviation is used to generate a small straight-line insertion path correction amount. It is understandable that the final linear insertion action is performed independently by a linear motion axis of the battery transfer device, which moves in a direction perpendicular to the vehicle chassis mounting plane.

[0039] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0040] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A universal battery replacement method based on visual recognition, characterized in that, include: Initialize multiple sets of vision acquisition devices in the general battery replacement system to capture images of the entrance scene of the vehicle to be replaced, the layout image of the battery compartment vehicle that is already in place, and the battery pack installation image of the vehicle chassis area. The entrance scene image, the layout image, and the battery pack installation image are analyzed simultaneously to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack joint points. Based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack connection points, system-level global positioning and attitude compensation parameters are generated. Based on the system-level global positioning and attitude compensation parameters, the battery transfer device is driven to move to the preset battery grabbing preparation position. The battery gripping tool located at the end of the battery transfer device is controlled to perform fine-tuning of its position and posture based on the system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data, thereby completing the gripping, transfer and installation operations of the battery pack.

2. The universal battery replacement method based on visual recognition according to claim 1, characterized in that, The step of simultaneously analyzing the entrance scene image, the layout image, and the battery pack installation image to extract the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack connection points includes: Edge contour detection is performed on the entrance scene image to identify the vehicle's outer boundary. The identified vehicle's outer boundary is then transformed into the world coordinate system to generate the spatial coordinates of the vehicle's outline. The layout image is subjected to preset pattern recognition to locate multiple optical marks pasted on the surface of the battery compartment carrier, and the precise position of each optical mark in three-dimensional space is calculated as the spatial coordinates of the carrier mark point; Feature point matching is performed on the battery pack installation image to identify the key features of the preset battery pack joining structure on the chassis, and the three-dimensional position of these key features is calculated based on the principle of binocular vision as the spatial coordinates of the chassis battery pack joining point.

3. The universal battery replacement method based on visual recognition according to claim 2, characterized in that, The step of generating system-level global positioning and attitude compensation parameters based on the spatial coordinates of the vehicle outline, the spatial coordinates of the vehicle marker points, and the spatial coordinates of the chassis battery pack connection points includes: The spatial coordinates of the vehicle outline are compared with the pre-stored vehicle model database to determine the pose deviation of the vehicle relative to the standard entry position of the universal battery swapping system, and the vehicle pose compensation amount is generated. Based on the spatial coordinates of the vehicle's marker points, the current flatness and horizontal tilt angle of the battery compartment vehicle are calculated, and combined with the vehicle's design position, the vehicle's attitude compensation amount is generated. By integrating the spatial coordinates of the chassis battery pack junction point, the vehicle pose compensation amount, and the vehicle attitude compensation amount, system-level global positioning and attitude compensation parameters are calculated through coordinate transformation to guide the precise movement of the battery transfer device and battery clamping tool.

4. The universal battery replacement method based on visual recognition according to claim 1, characterized in that, The step of driving the battery transfer device to a preset battery gripping preparation position based on the system-level global positioning and attitude compensation parameters includes: The system-level global positioning and attitude compensation parameters are input into the motion controller of the battery transfer device; The motion controller calculates the motion trajectory of each joint of the battery transfer device based on the target point coordinates and desired attitude contained in the system-level global positioning and attitude compensation parameters. The servo motors of each joint of the battery transfer device run according to the calculated motion trajectory, so that the end flange of the battery transfer device accurately reaches the preset battery gripping preparation position.

5. A universal battery replacement method based on visual recognition according to claim 4, characterized in that, During the movement of the battery transfer device, a real-time pose verification step is also included: Images of the calibration plate on the end flange of the battery transfer device are continuously acquired by vision sensors installed on the base and joints of the battery transfer device. The actual pose of the end flange of the battery transfer device is calculated in real time from continuously acquired images. The actual pose of the end flange of the battery transfer device is compared with the theoretical pose planned according to the system-level global positioning and attitude compensation parameters. If the deviation exceeds the set threshold, the motion is paused and the system-level global positioning and attitude compensation parameters are updated according to the deviation value. The motion trajectory is then replanned based on the updated system-level global positioning and attitude compensation parameters.

6. The universal battery replacement method based on visual recognition according to claim 1, characterized in that, The control is set at the battery clamping tool at the end of the battery transfer device, enabling it to perform fine-tuning of its posture based on the system-level global positioning and attitude compensation parameters and real-time acquired gripper sensor data, to complete the steps of gripping, transferring, and installing the battery pack, including: After the battery transfer device reaches the preset battery gripping preparation position, the close-range vision sensor integrated on the battery clamping tool is activated to scan the surface features of the battery pack to be replaced located in the battery compartment carrier. Based on the scanning results of the near-field vision sensor, the position of the battery pack gripping point in the system-level global positioning and attitude compensation parameters is corrected with millimeter-level precision to generate gripping point fine-tuning parameters. Based on the fine-tuning parameters of the gripping point, the battery clamping tool is controlled to move to the corrected precise gripping point; During the gripping process, data from the multi-dimensional force sensor and displacement sensor on the battery gripper's jaws are read in real time, and the closing force and position of the jaws are dynamically adjusted to ensure stable gripping without damaging the battery pack.

7. A universal battery replacement method based on visual recognition according to claim 6, characterized in that, The process of picking up, transferring, and installing the battery pack specifically includes three sub-steps: picking up, transferring, and installing. In the gripping sub-step, the battery gripping tool successfully grips the battery pack to be replaced based on the gripping point fine-tuning parameters and dynamic adjustment results, and then uploads the gripping success signal and the final gripper pose data during gripping. In the transfer sub-step, the battery transfer device plans an obstacle avoidance path from the battery compartment carrier to the vehicle chassis based on the received successful grab signal, and controls the battery clamping tool to carry the battery pack along the obstacle avoidance path, while continuously monitoring the attitude stability of the battery pack. During the installation sub-step, when the battery pack is transported to the vicinity of the target installation position on the vehicle chassis, the close-range vision sensor on the battery clamping tool is activated again to make a final alignment confirmation of the battery pack engagement point on the vehicle chassis, and based on the confirmation result, a final straight insertion action is performed to complete the installation of the battery pack.

8. A universal battery replacement method based on visual recognition according to claim 2, characterized in that, The step of calculating the precise position of each optical marker in three-dimensional space, as the spatial coordinates of the vehicle marker point, specifically includes: Acquire the layout image containing the optical markers, which is simultaneously captured from different perspectives by multiple sets of visual acquisition devices; Each of the layout images is preprocessed to enhance the pattern features of the optical markers and the image plane coordinates of each optical marker are identified; Based on the intrinsic and extrinsic parameter matrices of multiple vision acquisition devices obtained through pre-calibration, stereo matching is performed on the image plane coordinates of the same identified optical mark in different layout images. Based on the results of stereo matching, the forward intersection algorithm is used to calculate the three-dimensional coordinates of each optical mark in the world coordinate system, which are used as the precise position of the optical mark in three-dimensional space, i.e., the spatial coordinates of the vehicle mark point.

9. A universal battery replacement method based on visual recognition according to claim 3, characterized in that, The step of calculating the current flatness and horizontal tilt angle of the battery compartment vehicle based on the spatial coordinates of the vehicle marker points, and generating the vehicle attitude compensation amount in combination with the vehicle's design position, specifically includes: Select the spatial coordinates of at least three non-collinear vehicle marker points, and use the least squares method to fit a reference plane equation representing the actual plane of the battery compartment vehicle; Calculate the angle between the reference plane equation and the horizontal plane, and use it as the current horizontal tilt angle of the battery compartment vehicle; Calculate the distances from the spatial coordinates of all the vehicle marker points used for fitting to the reference plane, and use the maximum difference as the current flatness of the battery compartment vehicle; The current horizontal tilt angle and the current flatness are compared with the theoretical horizontal tilt angle and theoretical flatness corresponding to the pre-stored design position of the battery compartment vehicle. The translation and rotation amounts used to correct the deviation between the actual position and the design position of the battery compartment vehicle are calculated, and the vehicle attitude compensation amount is generated.

10. A universal battery replacement system based on vision recognition, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a universal battery replacement method based on visual recognition as described in any one of claims 1 to 9.

Images