Electric vehicle charging interface identification and positioning system for charging robot

Abstract

The invention relates to an electric vehicle charging interface identification and positioning system for a charging robot. The system comprises a data acquisition module, a preprocessing module, a charging interface identification module, a positioning module, a posture estimation module, a control module, an environment adaptation module and an automatic learning module. According to the system, images and three-dimensional point cloud data of a vehicle are collected through a camera and a laser radar, the two-dimensional position of a charging interface is automatically recognized through preprocessing and a deep learning model, and the spatial position and posture of the interface are accurately positioned by combining a least square method with three-dimensional coordinates. The control module drives the mechanical arm to be in butt joint with the plug through PID feedback control and A * path planning, and automatic charging is achieved. And the environment adaptation module combines data enhancement and adaptive correction to improve the recognition stability of the system in complex environments such as illumination and background change. The method improves the recognition precision and real-time performance of the charging robot in different environments, and is suitable for an unattended automatic charging scene.

Description

Technical Field

[0001] This invention relates to the field of charging robot control system technology, specifically to an electric vehicle charging interface identification and positioning system for charging robots. Background Technology

[0002] With the increasing popularity of electric vehicles, the demand for charging infrastructure is growing rapidly. Traditional charging methods, which mostly involve manual plugging and unplugging, are not only time-consuming and labor-intensive but also cannot meet the needs of large-scale charging, especially in automated and unattended scenarios, where their efficiency and convenience are greatly limited. Therefore, charging robots are gradually emerging as an alternative, automating the charging process for electric vehicles, reducing human intervention, and improving charging efficiency. However, current charging robot technology still faces many technical challenges in identifying and locating the charging interfaces of electric vehicles:

[0003] The charging ports of electric vehicles vary in location, shape, and size across different models. Charging robots need to switch flexibly between various vehicle types, thus requiring a recognition system capable of accurately locating the charging ports. However, recognition accuracy is low due to factors such as lighting, vehicle parking position and angle, and vehicle exterior color. Traditional recognition methods based on simple image processing or rule matching struggle to cope with diverse real-world scenarios.

[0004] Existing charging port identification systems perform poorly under conditions of varying lighting (such as strong light, shadows, and insufficient light at night), complex backgrounds (such as interference near the vehicle), and weather conditions (such as rain and fog). These factors can lead to charging port identification failures or misidentifications, affecting system stability.

[0005] Charging robots require high real-time performance and precision in practical operation. They need to quickly detect the charging interface location and accurately position it to ensure smooth docking of the robotic arm. Traditional image processing algorithms are computationally intensive and cannot meet real-time requirements, making it impossible to complete positioning in a short time. In addition, simple two-dimensional image information is insufficient to provide accurate depth information, leading to positional deviations during docking.

[0006] Major automakers have not standardized the design of charging interfaces. Different car models have charging interfaces that vary significantly not only in location, size, and shape, but also in subtle differences such as angle and material. This diversity makes it difficult for charging robots to use a single algorithm to adapt to multiple car models, thus limiting the applicability of the recognition system.

[0007] Current recognition systems mostly rely on static algorithms, lacking adaptive and learning capabilities. They cannot continuously adjust their recognition strategies based on actual environments and operational experience. As vehicle models change and environmental conditions vary, the systems struggle to self-optimize and cannot improve recognition accuracy by learning from data in new environments, leading to reduced system efficiency in complex scenarios or with new vehicle models.

[0008] Therefore, we urgently need to design an electric vehicle charging interface identification and positioning system for charging robots to solve the above problems. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing an electric vehicle charging interface identification and positioning system for charging robots, thereby solving the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution: an electric vehicle charging interface identification and positioning system for a charging robot, the system comprising:

[0011] Data acquisition module: includes cameras and LiDAR installed on the charging robot, used to acquire visual images and depth information of the electric vehicle;

[0012] Preprocessing module: used to perform noise reduction, illumination correction and edge detection on the acquired visual images and depth information to enhance the feature areas of the charging interface;

[0013] Charging interface recognition module: Based on a deep learning-based object detection model, it identifies the location of the electric vehicle's charging interface using a convolutional neural network (CNN). The convolutional layers of the CNN calculate the feature map using the following formula:

[0014]

[0015] Where f(x,y) represents the convolution output feature, which is the result of the convolution operation in the feature extraction stage, I(x,y) represents the input image, and w i,j is the kernel weight, and k is the kernel size; the localization and pose estimation module is used to combine visual images and depth information to calculate the 3D spatial coordinates and pose of the charging interface. The calculation method is based on least squares fitting of the 3D coordinates, and the calculation formula is:

[0016] X = (A T A) -1 A T b

[0017] Where X represents the three-dimensional coordinates of the charging interface, A represents the transformation matrix, and b represents the image coordinate data; the control module is used to control the movement of the charging robot's robotic arm based on the recognition and positioning results, so that its charging plug is aligned with and inserted into the charging interface of the electric vehicle.

[0018] As a preferred embodiment of the present invention, the charging interface recognition module uses a convolutional neural network (CNN) model, which is trained using the following object detection algorithm:

[0019]

[0020] Where P(y|x) represents the probability of outputting target y under input image x, f1(x,y) is the output scoring function of the deep learning model, which is the scoring function used for target detection after extracting features from f(x,y), and y′ represents all possible categories in the target detection task.

[0021] As a preferred embodiment of the present invention, the preprocessing module uses an illumination correction algorithm to normalize the value of each pixel in the image, thereby improving the accuracy of image recognition. The illumination correction algorithm is as follows:

[0022]

[0023] Where I represents the original image pixel value, I′ represents the corrected image pixel value, and max(I) and min(I) are the maximum and minimum pixel values ​​in the image, respectively.

[0024] Where X represents the three-dimensional coordinates of the charging interface, A represents the transformation matrix, and b represents the image coordinate data.

[0025] As a preferred embodiment of the present invention, the control module includes a real-time feedback control algorithm for dynamically adjusting the movement trajectory of the robotic arm to ensure that the charging plug is precisely aligned with the charging interface. The feedback control algorithm is as follows:

[0026]

[0027] Where u(t) is the control input, e(t) is the current deviation value, and K p K i and K d These are the proportional, integral, and derivative control parameters, respectively.

[0028] As a preferred technical solution of the present invention, the deep learning model is the YOLO (YouOnIyLookOnce) model. The model is trained using a large amount of charging interface image data, outputs the coordinates of the bounding box of the charging interface in the image, and defines the following loss function to improve the model accuracy:

[0029]

[0030] in, and These are the indication functions for the presence and absence of the charging port, respectively, where S is the number of grid cells, B is the number of candidate bounding boxes, and λ is the value of λ. coord and λ noobj These are the weight parameters.

[0031] As a preferred technical solution of the present invention, the environmental adaptation module uses data augmentation technology to enhance the training dataset, including operations such as rotation, scaling, translation and brightness adjustment, so that the charging interface recognition module can adapt to different lighting, angle and appearance changes.

[0032] As a preferred embodiment of the present invention, the system includes an adaptive deviation correction module, which is used to automatically correct the deviation of image recognition during the charging interface recognition process, and uses a Kalman filter to smooth the output coordinates. The state update formula of the Kalman filter is as follows:

[0033]

[0034] in, K represents the current state estimate. k For Kalman gain, z k Here, H represents the measured value, H is the measurement matrix, and R is the measurement noise covariance.

[0035] As a preferred technical solution of the present invention, the control module of the system realizes the optimal path selection of the robotic arm through a path planning algorithm. The path planning algorithm is based on the A* algorithm, and the heuristic function h(n) is defined as the Euclidean distance between the target position and the current node position:

[0036]

[0037] Among them, (x g oal,y goal ,z goal (x) represents the target location coordinates. n ,y n ,z n ) represents the coordinates of the current node.

[0038] As a preferred technical solution of the present invention, the system includes an automatic learning module, which optimizes the identification and positioning of the charging interface through a reinforcement learning algorithm, using a deep Q-learning algorithm (DQN), the loss function of which is defined as follows:

[0039]

[0040] Where θ is the network parameter, s is the current state, a is the current action, r is the reward value, γ is the discount factor, and Q(s,a;θ) is the Q-value function.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] 3D point cloud-assisted localization and adaptive deviation correction further improve the system's recognition accuracy and stability, enabling it to work reliably in different environments.

[0043] Real-time performance and docking accuracy: The system employs PID real-time feedback control and A* path planning algorithm to achieve efficient and precise docking of the robotic arm, ensuring that the charging plug is accurately inserted into the charging interface. This real-time control design meets the needs of unattended and automated charging, improving charging efficiency and system reliability.

[0044] Adaptive learning capability and scalability: The introduction of a reinforcement learning module enables the system to continuously learn from real-world operational data, optimizing its recognition strategy and positioning accuracy to adapt to different vehicle models' charging interfaces and diverse environments. This adaptive learning function enhances the system's scalability and applicability, supporting the widespread deployment of electric vehicle charging infrastructure. Attached Figure Description

[0045] Figure 1 This is a flowchart of the electric vehicle charging interface identification and positioning system for charging robots proposed in this invention. Detailed Implementation

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The following is in conjunction with the appendix Figure 1 The present invention will be described in detail with reference to several embodiments.

[0048] This invention provides an electric vehicle charging interface identification and positioning system for charging robots. The system, through the collaborative work of multiple modules, automatically identifies and precisely locates the charging interface of an electric vehicle, thereby guiding the charging robot to complete unmanned automated charging tasks. The implementation methods of each module are described in detail below, and several embodiments are provided to illustrate the specific applications of the system.

[0049] I. System Overall Structure

[0050] The system of this invention includes the following modules: a data acquisition module, a preprocessing module, a charging interface identification module, a localization and attitude estimation module, a control module, an environmental adaptation module, and an automatic learning module. Through the coordinated work of these modules, the system can efficiently and accurately complete the task of identifying and locating electric vehicle charging interfaces, providing support for the automatic charging operation of the charging robot.

[0051] II. Implementation methods of each module

[0052] 1. Data Acquisition Module

[0053] The data acquisition module mainly consists of a camera and a LiDAR sensor mounted on the charging robot. It acquires visual images and depth information of the electric vehicle. The camera obtains a two-dimensional image of the charging port, while the LiDAR acquires three-dimensional point cloud data of the vehicle's surroundings to determine the spatial depth and relative position of the charging port. The data acquisition module is activated after the charging robot starts, acquiring real-time images and depth data around the vehicle's charging port to provide foundational data for subsequent identification and localization.

[0054] 2. Preprocessing module

[0055] The preprocessing module processes the acquired raw images and depth data to improve image clarity and feature representation, including the following steps:

[0056] Image denoising: Gaussian filtering algorithm is used to denoise the acquired image and remove random noise from the image.

[0057] Illumination Correction: To address changes in ambient lighting, a normalization algorithm is used to correct the image's illumination, reducing the impact of lighting variations on recognition. The specific formula is:

[0058]

[0059] Where I represents the original image pixel value, I′ represents the corrected pixel value, and max(I) and min(I) are the maximum and minimum pixel values ​​in the image, respectively. Edge detection: The Canny edge detection algorithm is used to extract edges from the image to enhance the contour of the charging interface in the image, providing more significant features for subsequent recognition.

[0060] 3. Charging interface identification module

[0061] The charging interface recognition module is based on a deep learning model and uses a convolutional neural network (CNN) to perform target detection and recognition on the preprocessed image.

[0062] Model selection and training: The YOLO (You Only Look Once) object detection model is adopted, which has a compact structure and is suitable for real-time detection.

[0063] The model is trained using a large amount of charging interface image data and classifies target charging interfaces based on a deep learning model. For each possible category, the system calculates the probability of that category using the following formula:

[0064]

[0065] Where P(y|x) represents the probability of outputting target y under input image x, f1(x,y) is the output scoring function of the deep learning model, which is the scoring function used for target detection after extracting features from f(x,y), and y′ represents all possible categories in the target detection task.

[0066] Model loss function: The loss function of the YOLO model is used to optimize recognition accuracy, and is defined as follows:

[0067]

[0068] Feature extraction and convolution operations: During object detection, the convolutional layer calculates the feature map using the following formula to extract the features of the charging interface:

[0069]

[0070] The identified charging port location is output as two-dimensional coordinates for use by the positioning and attitude estimation module.

[0071] 4. Localization and Attitude Estimation Module

[0072] The positioning and attitude estimation module converts the two-dimensional position of the charging interface into three-dimensional spatial coordinates and calculates attitude information so that the control module can accurately dock with the charging interface.

[0073] 2D to 3D coordinate transformation: Using point cloud data from LiDAR, the position of the identified charging interface in the image is converted into 3D spatial coordinates.

[0074] Least squares fitting: The center position of the charging interface is fitted using the least squares method. The specific formula is as follows:

[0075] X = (A T A) -1 A T b

[0076] Attitude estimation: Based on the shape characteristics of the charging interface, calculate its attitude in three-dimensional space, including rotation angle and tilt, to ensure the accuracy of the docking angle of the robotic arm movement.

[0077] 5. Control Module

[0078] Based on the identification and localization results, the control module controls the charging robot's robotic arm to complete the docking operation. The control module employs PID control and the A* path planning algorithm.

[0079] PID control: Adjusts the robotic arm's movement trajectory in real time. The control formula is as follows:

[0080]

[0081] Path planning: The A* algorithm is used to plan the optimal motion trajectory of the robotic arm. The heuristic function is defined as follows:

[0082]

[0083] 6. Environmental Adaptation Module

[0084] The environmental adaptation module utilizes data augmentation and adaptive bias correction techniques to enhance the system's environmental adaptability. Data augmentation: During the model training phase, data augmentation techniques such as rotation and scaling are used to expand the sample size and improve the model's adaptability.

[0085] Adaptive troubleshooting and correction: A Kalman filter is used to smooth the positioning results. The Kalman filter formula is as follows:

[0086]

[0087] 7. Automatic Learning Module

[0088] The automatic learning module optimizes the recognition and docking process through reinforcement learning. The system uses the Deep Q-Learning algorithm (DQN) to train the model, and the DQN loss function is as follows:

[0089]

[0090] III. Examples

[0091] Example 1: Charging Interface Identification and Location under Standard Environment

[0092] In an environment with uniform lighting and vehicles parked in a standardized manner, the system uses the aforementioned modules to complete the identification, positioning, and docking operations of the charging interface with the robotic arm.

[0093] In this standard environment, the system can identify the charging interface and complete the docking within seconds, demonstrating excellent real-time performance and high precision, making it suitable for standardized scenarios such as parking lots.

[0094] Example 2: Charging Interface Identification in Complex Environments

[0095] In complex environments with uneven lighting, varying charging port angles, or irregular vehicle positions, this embodiment utilizes the adaptive deviation correction and path planning functions of the environmental adaptation module to improve system recognition and positioning performance.

[0096] The system uses Kalman filtering to correct for biases in the recognition results.

[0097]

[0098] The control module uses the A* algorithm to plan the path, avoid obstacles, and ensure that the robotic arm accurately reaches the charging interface.

[0099] Example 3: Charging interface identification and positioning for different vehicle models

[0100] To address the significant differences in the location and shape of charging ports across different vehicle models, the system employs a recognition method that combines template matching and deep learning.

[0101] Template Database: The system establishes a template database for charging interfaces of multiple vehicle models. Each template contains the characteristic shape, size and typical location of the charging interface.

[0102] Model and template matching: First, the location of the charging interface is initially located using a deep learning model. Then, the precise location and shape of the charging interface are determined by matching the recognition results with a template database.

[0103] Adaptive template update: The system automatically updates the template database during use, enabling the system to adapt to new vehicle models.

[0104] Example 4: Charging Interface Identification and Localization Based on Multimodal Data Fusion

[0105] In certain environments, single visual data may not be sufficient to meet the requirements of high-precision recognition. Therefore, this embodiment employs a data fusion approach using camera and LiDAR data.

[0106] Multimodal data acquisition: Cameras and LiDAR simultaneously acquire images and 3D point cloud data.

[0107] Image and point cloud fusion: After identifying the charging interface in the image, the identified two-dimensional coordinates are aligned with the LiDAR point cloud data to obtain a precise three-dimensional position.

[0108] Feature matching: By combining feature points from image and depth information, the ICP algorithm is used to further optimize the alignment results and achieve high-precision positioning.

[0109] Example 5: Automatic Charging Interface Recognition and Localization Optimized by Reinforcement Learning

[0110] To enable the system to be continuously optimized in different vehicle models and environments, this embodiment introduces an automatic learning module into the system and uses deep Q-learning for reinforcement learning.

[0111] Reward feedback mechanism: The system records the recognition accuracy and docking success rate in each recognition and docking as reward feedback.

[0112] DQN Training: Through Deep Q-Learning (DQN), the system automatically optimizes path planning and recognition accuracy in different environments. The DQN loss function is as follows:

[0113]

[0114] After a period of learning, the system can significantly improve the success rate of charging interface recognition and docking.

[0115] Example 6: Application of mobile charging robots in outdoor charging scenarios

[0116] In outdoor charging environments, there are significant challenges due to lighting conditions, uneven ground, and environmental interference. This embodiment optimizes the system's adaptability to outdoor scenarios: Image Enhancement: By adding preprocessing operations such as increasing image brightness, contrast, and saturation, the clarity of the image is improved under different lighting conditions. Adaptive Path Adjustment: When encountering uneven ground or obstacles, the system automatically adjusts the path, using the PID feedback mechanism of the control module to fine-tune the position of the robotic arm, ensuring docking stability. Multiple Positioning and Dynamic Tracking: During positioning and docking, the system periodically reacquires image and depth information, and combines this with Kalman filtering smoothing to achieve dynamic tracking of the vehicle charging interface.

[0117] The contents not described in detail in this description are existing technologies known to those skilled in the art. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An electric vehicle charging port identification and positioning system for a charging robot, characterized in that: The system includes: Data acquisition module: including cameras and lidar installed on the charging robot, used to collect visual images and depth information of electric vehicles; Preprocessing module: used to perform noise reduction, illumination correction and edge detection on the collected visual images and depth information to enhance the characteristic area of ​​the charging interface; Charging port identification module: Based on the deep learning target detection model, the charging port position of the electric vehicle is identified through the convolutional neural network. The convolutional layer of the convolutional neural network calculates the feature map through the following formula: Among them, f(x,y) is the convolution output feature, which is the result of the convolution operation in the feature extraction stage, I(x,y) is the input image, and w i,j is the convolution kernel weight, k is the convolution kernel size; Positioning and attitude estimation module: used to combine visual images and depth information to calculate the three-dimensional spatial coordinates and attitude of the charging interface. The calculation method is based on the least squares method to fit the three-dimensional coordinates. The calculation formula is: X=(A T A) -1 A T b Where X is the three-dimensional coordinate of the charging port, A is the conversion matrix, and b is the image coordinate data; Control module: used to control the movement of the charging robot's robotic arm based on the recognition and positioning results, so that its charging plug is aligned with and inserted into the charging port of the electric vehicle. The robotic arm adopts a six-axis robotic arm.

2. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The charging port recognition module uses a convolutional neural network model, which is trained using the following target detection algorithm: Among them, P(y|x) represents the probability of outputting target y under input image x; f1(x,y) is the output scoring function of the deep learning model, which is the scoring function used for target detection after extracting features from f(x,y); y′ represents all possible categories in the target detection task.

3. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The preprocessing module uses an illumination correction algorithm to normalize each pixel value in the image. The illumination correction algorithm is as follows: Where I represents the original image pixel value, I′ represents the corrected image pixel value, max(I) and min(I) are the maximum and minimum pixel values ​​in the image, respectively.

4. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The control module includes a real-time feedback control algorithm for dynamically adjusting the motion trajectory of the robot arm so that the charging plug can be accurately docked with the charging interface. The feedback control algorithm is as follows: Among them, u(t) is the control input, e(t) is the current deviation value, K p , K i and K d are the proportional, integral and differential control parameters respectively.

5. The electric vehicle charging port identification and positioning system for a charging robot according to claim 2, characterized in that: The deep learning model is a YOLO model, which is trained with a large amount of charging interface image data, outputs the location box coordinates of the charging interface in the image, and defines the following loss function to improve the model accuracy: in, and are the indicator functions for the presence and absence of the charging port, respectively, S is the number of grids, B is the number of candidate borders, and λ coord and λ noobj is the weight parameter.

6. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The system includes an environment adaptation module, which uses data enhancement technology to enhance the training data set, including rotation, scaling, translation and brightness adjustment operations, so that the charging port recognition module can adapt to different lighting, angles and appearance changes.

7. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The system includes an adaptive deviation correction module for automatically correcting the deviation of image recognition during the charging interface recognition process, and adopts a Kalman filter to smooth the output coordinates. The state update formula of the Kalman filter is as follows: in, represents the current state estimate, K k is the Kalman gain, z k is the measurement value, H is the measurement matrix, and R is the measurement noise covariance.

8. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The control module of the system realizes the optimal path selection of the robot arm through the path planning algorithm. The path planning algorithm is based on the A* algorithm and defines the heuristic function h(n) as the Euclidean distance between the target position and the current node position: Among them, (x g oal,y goal ,z goal ) is the target position coordinate, (x n ,y n ,z n ) are the current node coordinates.

9. The electric vehicle charging port identification and positioning system for a charging robot according to claim 1, characterized in that: The system includes an automatic learning module, which optimizes the recognition and positioning of the charging port through a reinforcement learning algorithm, using a deep Q learning algorithm, and its loss function is defined as follows: Among them, θ is the network parameter, s is the current state, a is the current action, r is the reward value, γ is the discount factor, and Q(s,a;θ) is the Q value function.

Images