Vehicle charging method, charging robot and computer readable storage medium

Abstract

The invention provides a vehicle charging method, a charging robot and a computer readable storage medium, and relates to the technical field of vehicles. The method comprises the following steps: acquiring a first image including a charging port of a to-be-charged vehicle; performing image recognition on the first image to obtain the current three-dimensional position of each charging hole; performing first registration processing according to the current three-dimensional position of each charging hole and the three-dimensional position of the template to obtain a first pose transformation relation between the current three-dimensional position of each charging hole and the three-dimensional position of the template; and according to the first pose transformation relation, the pose of the charging gun is adjusted, and the charging gun is controlled to be inserted into the charging port to charge the to-be-charged vehicle. According to the method, different types of image data are fused, the pose transformation relation is determined in a closed-loop control mode, the image is collected again after the pose of the charging gun is adjusted each time, rapid correction is carried out, it is ensured that each adjustment is close to the target position as much as possible, and the reliability and efficiency of automatic charging are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle charging method, a charging robot, and a computer-readable storage medium. Background Art

[0002] With the rapid development of the new energy vehicle market, users are increasingly demanding convenient and efficient charging methods. For example, charging, in practice, requires users to physically exit their vehicles and operate the charging equipment, regardless of weather conditions. This manual connection process takes a relatively long time, impacting the user experience and reducing charging efficiency.

[0003] To address these issues, existing technologies use robots carrying charging guns to charge new energy vehicles. However, this charging method relies on the vehicle's parking location to perform charging tasks, which is easily affected by external environmental factors. This leads to insufficient positioning accuracy of the vehicle's charging port, which in turn affects the success rate and efficiency of charging. Therefore, how to accurately locate the charging port and guide the robot to accurately complete the charging operation is one of the urgent problems in this field. Summary of the Invention

[0004] The purpose of this application is to address the deficiencies in the above-mentioned prior art and provide a vehicle charging method, a charging robot and a computer-readable storage medium, so as to solve the problem that the vehicle charging method in the prior art depends on the parking position of the vehicle to perform the charging task, which is easily affected by external environmental factors, resulting in insufficient positioning accuracy of the vehicle charging port, thereby affecting the success rate and efficiency of charging.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a vehicle charging method, the method comprising:

[0007] In response to a vehicle to be charged entering a charging area, acquiring a first image including a charging port of the vehicle to be charged, the charging port including at least one charging hole, the first image including a pixel image and a point cloud image;

[0008] Performing image recognition on the first image to obtain a current three-dimensional position of each of the charging ports;

[0009] Performing a first registration process based on the current three-dimensional position of each charging port and the three-dimensional position of a template of each charging port to obtain a first pose transformation relationship between the current three-dimensional position of each charging port and the three-dimensional position of the template, wherein the three-dimensional position of the template is the three-dimensional position of the charging port when the charging gun is at a reference three-dimensional position and the charging gun can be inserted into the charging port;

[0010] According to the first posture transformation relationship, the posture of the charging gun is adjusted, and the charging gun is controlled to be inserted into the charging port to charge the vehicle to be charged.

[0011] As a possible implementation, adjusting the posture of the charging gun according to the first posture transformation relationship, and controlling the charging gun to be inserted into the charging port to charge the vehicle to be charged, includes:

[0012] A. Using the first posture transformation relationship as the initial current posture transformation relationship;

[0013] B. adjusting the posture of the charging gun according to the current posture transformation relationship, and reacquiring a second image of the charging port of the vehicle to be charged after the adjustment;

[0014] C. performing image recognition on the second image to obtain a new three-dimensional position of each charging port;

[0015] D. performing a second registration process based on the new three-dimensional position of each charging hole and the three-dimensional position of the template to obtain a second pose transformation relationship between the new three-dimensional position of each charging hole and the three-dimensional position of the template;

[0016] E. Determine whether the second posture transformation relationship meets the stop adjustment condition. If so, end the loop and control the charging gun to be inserted into the charging port to charge the vehicle to be charged; if not, use the second posture transformation relationship as the new current posture transformation relationship and re-execute step A.

[0017] As a possible implementation manner, performing image recognition on the first image to obtain the current three-dimensional position of each charging port includes:

[0018] Inputting the pixel image into a pre-trained hole segmentation model, and having the hole segmentation model segment the pixel image to obtain segmentation information of at least one hole, the segmentation information including a category label, bounding box coordinates, and mask information;

[0019] Based on the segmentation information of each of the holes, the current three-dimensional position of the charging hole is determined.

[0020] As a possible implementation manner, determining the current three-dimensional position of the charging hole based on the segmentation information of each hole includes:

[0021] extracting the outline of each of the holes from the mask information of each of the holes;

[0022] Screening the contours of the holes to obtain a valid contour corresponding to at least one charging hole;

[0023] Determine the three-dimensional points in the point cloud image corresponding to the pixel points of each valid contour in the pixel image, and obtain a set of three-dimensional coordinates corresponding to each valid contour;

[0024] Based on the three-dimensional coordinate sets corresponding to the effective contours, the current three-dimensional position of each charging hole is determined.

[0025] As a possible implementation, screening the contours of the holes to obtain a valid contour corresponding to at least one charging hole includes:

[0026] According to the preset contour features of each charging hole, the contours of each hole are screened to obtain a valid contour corresponding to at least one charging hole, wherein the preset contour features include: contour size and relative positional relationship between the contours of multiple charging holes.

[0027] As a possible implementation, determining the current three-dimensional position of each charging port based on the three-dimensional coordinate set corresponding to each valid contour includes:

[0028] According to the three-dimensional coordinate sets corresponding to the effective contours, the central three-dimensional coordinates of each effective contour are determined, and the central three-dimensional coordinates are used as the current three-dimensional position of the charging hole.

[0029] As a possible implementation, performing a first registration process based on the current three-dimensional position of each charging hole and the three-dimensional position of the template of each charging hole to obtain a first pose transformation relationship between the current three-dimensional position of the charging hole and the three-dimensional position of the template includes:

[0030] Combining the current three-dimensional positions of all charging holes into a first ordered point set, and combining the three-dimensional positions of all templates into a second ordered point set;

[0031] The first posture transformation relationship is determined according to the first ordered point set and the second ordered point set.

[0032] As a possible implementation manner, determining the first posture transformation relationship according to the first ordered point set and the second ordered point set includes:

[0033] The first ordered point set and the second ordered point set are used as input parameters of a preset iterative closest point registration algorithm, and the first posture transformation relationship is calculated by the iterative closest point registration algorithm.

[0034] In a second aspect, another embodiment of the present application provides a charging robot, comprising a robotic arm and a processing device, wherein a charging gun and a camera are mounted at the end of the robotic arm, and a positional relationship between the charging gun and the camera is fixed;

[0035] The robotic arm is used to drive the photographing device to photograph the charging port of the vehicle to be charged, and the charging gun is used to charge the vehicle to be charged;

[0036] The processing device is used to execute the steps of the vehicle charging method described in any one of the first aspects.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle charging method as described in any one of the first aspects above are executed.

[0038] According to the vehicle charging method, charging robot, and computer-readable storage medium of the embodiments of the present application, in response to a vehicle to be charged entering a charging area, a first image of the charging port of the vehicle to be charged is acquired. The first image includes a pixel image and a point cloud image. Image recognition is performed on the first image to obtain the current three-dimensional position of each charging port. A first registration process is performed based on the current three-dimensional position of each charging port and the three-dimensional position of the template of each charging port to obtain a first pose transformation relationship between the current three-dimensional position of each charging port and the three-dimensional position of the template. Based on the first pose transformation relationship, the position of the charging gun is adjusted and the charging gun is controlled to be inserted into the charging port to charge the vehicle to be charged. According to the embodiments of the present application, the charging robot obtains more comprehensive and accurate environmental information and improves its understanding of the charging port and its surrounding environment by fusing different types of image data. The charging port area is accurately segmented and identified for registration processing. The pose transformation relationship between the current three-dimensional position of the charging port and the three-dimensional position of the template is determined, and the position of the charging gun is adjusted based on the pose transformation relationship. After each adjustment of the charging gun's pose, the image is recaptured and quickly corrected to ensure that each adjustment is as close to the target position as possible. In addition, this application uses a vision-guided charging robot for automatic charging. Users do not need to get off the vehicle and only need to park the vehicle in the charging area. The robot can automatically charge the vehicle with a charging gun under the guidance of the camera, greatly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic structural diagram of a charging robot provided in an embodiment of the present application is shown;

[0041] Figure 2 A schematic diagram of a vehicle charging method according to an embodiment of the present application is shown;

[0042] Figure 3 A schematic diagram of a flow chart of a method for determining the three-dimensional position of a charging port provided in an embodiment of the present application is shown;

[0043] Figure 4 A schematic diagram showing a flow chart of another method for determining the three-dimensional position of a charging port provided in an embodiment of the present application is shown;

[0044] Figure 5 A schematic diagram showing a contour segmentation result provided by an embodiment of the present application is shown;

[0045] Figure 6 A schematic diagram of a flow chart of a method for determining a first pose transformation relationship provided in an embodiment of the present application is shown;

[0046] Figure 7 A flow chart of a posture adjustment method provided in an embodiment of the present application is shown;

[0047] Figure 8 A schematic structural diagram of a processing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0049] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0050] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0051] Figure 1 The schematic diagram of the structure of a charging robot provided by an embodiment of the present application is shown. Figure 1 As shown, robot 1 includes a robotic arm and processing equipment. The end of the robotic arm is equipped with a charging gun 3 and a camera, such as a camera 2. The positional relationship between charging gun 3 and camera 2 remains fixed. When a vehicle to be charged is parked in the charging area, robot 1 first drives camera 2 to the camera position, takes a picture of the vehicle's charging port 4, and calculates its accurate position and posture.

[0052] It should be noted that since the posture relationship between the charging gun 3 and the camera 2 is fixed, it means that the relative position, angle and other parameter information between the charging gun 3 and the camera 2 are known and constant, so that the processing equipment can directly determine the position and posture of the charging gun relative to the charging port of the vehicle to be charged based on the image obtained by the shooting device without the need for additional calibration or adjustment. Therefore, by using a fixed posture relationship, high-precision alignment processing can be achieved to ensure that the charging gun is accurately positioned every time.

[0053] Optionally, a robotic arm is used to drive a camera 2 to photograph the charging port 4 of the vehicle being charged. The robotic arm has sufficient degrees of freedom for flexible operation, allowing the robotic arm to precisely move the charging gun 3 and camera 2 to a specified position to accommodate the charging port location of different vehicle models. The camera is used to photograph the charging port of the vehicle being charged to determine its exact location and posture. The captured image data is then combined with image recognition technology to calculate the coordinates of the charging port relative to the robot by analyzing the captured image data. The charging gun is used to charge the vehicle being charged. Once the location of the charging port is determined by the camera, the robotic arm adjusts its posture to accurately insert the charging gun into the charging port, completing the charging task.

[0054] Optionally, when a vehicle to be charged enters a designated charging area, the processing device sends a capture command to a camera. The camera first responds to the capture command by capturing a first image of the vehicle's charging port. This first image includes not only a pixel image but also a point cloud image to provide depth information. The charging port is provided with at least one charging port. The processing device performs advanced image recognition processing on the captured first image to accurately locate the current three-dimensional position of each charging port. Based on the current three-dimensional position of each charging port and a predefined three-dimensional template position of the charging port, i.e., the ideal position of each charging port when the charging gun is at a reference three-dimensional position and ready for insertion into the charging port, the processing device performs a first registration process to determine a first pose transformation relationship between the currently detected charging port position and the ideal template three-dimensional position. Furthermore, the processing device uses the obtained first pose transformation relationship to precisely adjust the posture of the charging gun so that it is accurately aligned with the charging port of the vehicle to be charged. The processing device then controls the safe and reliable insertion of the charging gun into the charging port, thereby initiating charging of the vehicle to be charged.

[0055] Based on this, the charging robot provided by the embodiment of the present application can automatically charge the vehicle. This automated process ensures that the charging task can be completed efficiently and accurately even without human intervention, greatly improving the user experience and operational efficiency.

[0056] The following combines the above Figure 1 The content described in the charging robot shown is used to explain in detail the vehicle charging method provided in the embodiment of the present application.

[0057] Figure 2 This is a flow chart of a vehicle charging method provided in an embodiment of the present application. Figure 2 As shown, the method specifically includes the following steps:

[0058] S201: In response to a vehicle to be charged entering a charging area, acquiring a first image including a charging port of the vehicle to be charged.

[0059] Optionally, the charging robot is equipped with a vehicle entry detection mechanism to detect whether the vehicle to be charged has entered the charging area, for example, using a ground sensor coil, camera monitoring, lidar, etc. to identify the presence of the vehicle to be charged in the charging area.

[0060] Optionally, when a vehicle to be charged is detected entering a predetermined charging area, the charging robot activates and prepares to begin the charging process. Specifically, the charging robot needs to determine the exact location of the charging port. To this end, the processing device may send a shooting instruction to a camera, control a robotic arm to drive the camera to move to the area of the charging port of the vehicle to be charged, and use the camera to capture an image including the charging port, thereby obtaining a first image including the charging port of the vehicle to be charged.

[0061] Optionally, the charging port includes at least one charging hole, and the first image includes a pixel image and a point cloud image. The pixel image is also an RGB image, which can provide rich color and texture information, helping to identify the specific shape, color and other visual features of the charging port. The point cloud image is also a depth image. The point cloud image not only contains the color information of the vehicle to be charged, but also provides spatial coordinate information, that is, the precise position of each point in three-dimensional space, such as x, y, and z coordinates, which is crucial for determining the three-dimensional position of the charging port.

[0062] S202: Perform image recognition on the first image to obtain the current three-dimensional position of each charging port.

[0063] Optionally, after acquiring a first image (including a pixel image and a point cloud image) containing the charging port of the vehicle to be charged, the first image may be preprocessed, such as by noise removal and registration. Noise removal refers to preprocessing the collected point cloud data to eliminate any noise, and registration refers to point cloud registration, which is necessary if multiple sensors are used or if data from multiple scans needs to be merged, to ensure that all data is in the same coordinate system.

[0064] Optionally, features of the charging port and charging hole are extracted from the pixel image and, combined with the depth information provided by the point cloud image, the 2D features are mapped back into 3D space. This allows the specific 3D coordinates of each charging hole to be located from the point cloud data. The charging hole region representing the charging port is then segmented from the entire point cloud data. For each charging hole, the data points are fitted to estimate the precise 3D position of the charging hole center, thereby obtaining the current 3D position of each charging hole.

[0065] S203 : Perform a first registration process based on the current three-dimensional position of each charging hole and the three-dimensional position of the template of each charging hole to obtain a first pose transformation relationship between the current three-dimensional position of each charging hole and the three-dimensional position of the template.

[0066] Optionally, the template 3D position is the 3D position of the charging port when the charging gun is in the reference 3D position and can be inserted into the charging port. This refers to the ideal coordinates of the charging port relative to the charging gun when the charging gun is in the ideal reference position. This template 3D position is typically a fixed value based on design specifications or obtained through multiple precise measurements. It represents the spatial coordinates where the charging port should be located under optimal docking conditions, i.e., when the charging gun can be directly inserted without additional adjustment.

[0067] Optionally, before locating the charging port of a randomly parked charging vehicle, a camera is required to capture a baseline image of the charging port. This baseline image includes both a point cloud image and a color image of the charging port. Specifically, the baseline image contains not only a color image of the charging port and its surroundings, but also corresponding point cloud data, providing three-dimensional spatial information for each pixel. The edge pixels of all circular holes within the charging port are annotated in the color image. Since the position of each point in the color image can be directly mapped to the same position in the point cloud image, the edge pixels can be associated one-to-one with the corresponding three-dimensional points in the point cloud image. Then, using the three-dimensional point set of the selected circular hole edges, a geometric fitting algorithm is used to fit an ideal circle that most closely resembles the actual shape. Based on the fitted three-dimensional circle, the three-dimensional coordinates of the center of each circular hole can be accurately calculated. The calculated three-dimensional coordinates of the centers of all circular holes are saved as template data, thereby obtaining the template three-dimensional position of each charging port.

[0068] Optionally, a registration algorithm can be used to compare and align the current 3D position of each charging port with a pre-stored standard template 3D position of each charging port. By comparing the actual detected charging port position with the template position, a first pose transformation relationship is determined so that the current 3D position of the charging port matches the template 3D position as closely as possible. The first pose transformation relationship describes the rotation and translation required to transform the currently detected 3D position of the charging port to the template 3D position. In other words, by optimizing the current 3D position of each charging port, the robot arm determines how to move the charging gun so that the charging gun is correctly aligned with the charging port, thereby ensuring the accuracy of the charging port position.

[0069] S204: Adjust the posture of the charging gun according to the first posture transformation relationship, and control the charging gun to be inserted into the charging port to charge the vehicle to be charged.

[0070] Optionally, based on the first pose transformation relationship, the pose of the charging gun mounted on the robotic arm is adjusted so that its position and orientation in space meet the requirements, allowing it to be smoothly inserted into the charging port. After the pose adjustment is completed, the charging gun is controlled to advance along a predetermined path and insert into the charging port, thereby starting to charge the vehicle.

[0071] Optionally, the first pose transformation relationship obtained by the registration process contains the necessary rotation and translation parameters, and these parameters describe how to transform the charging gun from its current posture to a posture that can correctly dock with the charging port. Therefore, path planning is performed based on the transformation parameters contained in the first pose transformation relationship to determine a safe and effective motion path, so that the robotic arm can smoothly move the charging gun to the target position, and drive the robotic arm to accurately adjust the position and posture of the charging gun according to the predetermined path. When the charging gun approaches the calculated target position, the charging gun insertion speed is slowed down and the posture is adjusted more finely to ensure that the charging gun is aligned with the charging port, and the insertion action is performed and the charging program is started to start powering the vehicle to be charged.

[0072] Based on this, according to the vehicle charging method provided in the embodiment of the present application, a charging robot is used to obtain more comprehensive and accurate environmental information, and the ability to understand the charging port and its surrounding environment is improved by fusing different types of image data. The charging port area is accurately segmented and identified for registration processing, and the posture transformation relationship between the current three-dimensional position of the charging port and the three-dimensional position of the template is determined, and then the posture of the charging gun is adjusted according to the posture transformation relationship. And after each adjustment of the charging gun posture, the image is recaptured and quickly corrected to ensure that each adjustment is as close to the target position as possible. In addition, the present application is based on a visually guided charging robot for automatic charging. The user does not need to get off the vehicle and only needs to park the vehicle in the charging area. The robot can automatically charge the vehicle with the charging gun under the guidance of the camera, which greatly improves the user experience.

[0073] Figure 3 The flowchart of a method for determining the three-dimensional position of a charging port provided by an embodiment of the present application is shown. Figure 3 As shown, the above step S202 performs image recognition on the first image to obtain the current three-dimensional position of each charging port, which specifically includes the following steps:

[0074] S301: Input a pixel image into a pre-trained hole segmentation model, and use the hole segmentation model to segment the pixel image to obtain segmentation information of at least one hole.

[0075] Optionally, the hole segmentation model is a model pre-trained by a machine learning algorithm, such as a convolutional neural network in deep learning, and is used to identify and segment specific types of objects in an image, such as charging holes. Moreover, the hole segmentation model can not only identify different areas in the image, but also distinguish which areas belong to charging holes. Specifically, after obtaining the pixel image of the charging port of the vehicle to be charged, the pixel image is fed into the hole segmentation model as input. The hole segmentation model analyzes each pixel in the pixel image and classifies it as a "charging hole" or "non-charging hole" according to the characteristics of each pixel. That is, by segmenting the pixel image, the pixel image is divided into multiple parts, each part corresponding to a different category.

[0076] Optionally, the segmentation information includes a category label, bounding box coordinates, and mask information. The category label indicates whether a certain area is classified as a charging hole, for example, it can be represented by a digital label. The bounding box coordinates refer to the position and size of a rectangular box around each detected charging hole, which indicates the approximate position and size of the charging hole. The mask information is information that describes the shape of the charging hole more accurately than the bounding box coordinates. It is a binary image of the same size as the original image, in which the charging hole area is marked as 1 and the rest is marked as 0, thereby clearly marking the exact outline of the charging hole.

[0077] S302: Determine the current three-dimensional position of the charging port based on the segmentation information of each port.

[0078] Alternatively, since the spatial position of the charging port cannot be directly determined from a pixel image alone, a point cloud image containing depth information is needed to determine the current three-dimensional position of the charging port. Since the point cloud data provides the actual physical distance of each pixel relative to the camera, the specific location of the charging port on the two-dimensional pixel image can be determined based on the segmentation information. This two-dimensional pixel location information is then used to query the corresponding point cloud data, extract the depth value at the corresponding location, and convert the coordinates on the two-dimensional plane into three-dimensional spatial coordinates, thereby obtaining the precise location of the charging port in the real world.

[0079] Figure 4 FIG2 shows a flow chart of another method for determining the three-dimensional position of a charging port provided in an embodiment of the present application. Figure 4 As shown, the above step S302 determines the current three-dimensional position of the charging port based on the segmentation information of each port, and specifically includes the following steps:

[0080] S401 : Extract the outline of each hole from the mask information of each hole.

[0081] Optionally, the mask information is binary information, where the charging hole area is marked as 1 and the rest is marked as 0, thereby providing a clear boundary definition. The outline of each charging hole is extracted from the mask information by using a computer vision algorithm, such as OpenCV. For example, referring to Figure 5 The schematic diagram of the contour segmentation result shown is that the contour is a closed curve composed of a series of points, representing the shape of the edge of the charging hole.

[0082] S402: Screen the contours of each hole to obtain a valid contour corresponding to at least one charging hole.

[0083] It should be noted that not all detected contours are valid charging holes. Some invalid contours may exist due to noise or misidentification. Therefore, it is necessary to screen the contours of each hole to obtain a valid contour that truly represents the charging hole. For example, contour screening is performed through size filtering, shape analysis, and position verification. Among them, size filtering refers to excluding contours that are obviously too large or too small based on the approximate size range of known charging holes. Shape analysis refers to checking whether the contour is close to a circle or other expected shape. For charging holes, a more regular geometric shape is usually expected. Position verification refers to considering that charging holes generally appear in specific areas of the vehicle, and contours can be further filtered through position information.

[0084] Optionally, the contours of each charging hole are screened according to preset contour features of each charging hole to obtain a valid contour corresponding to at least one charging hole, wherein the preset contour features include: contour size and relative positional relationship between contours of multiple charging holes.

[0085] Exemplarily, the contour size refers to the size or area of each detected contour. For the charging port, its actual physical size is known or at least has an approximate range. Therefore, during the screening process, contours that are obviously too large or too small can be excluded by setting a reasonable threshold range. For example, a minimum size limit and a maximum size limit. Specifically, any contours smaller than the minimum size limit can be excluded because they may represent noise or other non-target objects. Similarly, contours larger than the maximum size limit do not meet the standards for charging ports and can also be excluded, which can effectively reduce the possibility of misjudgment and improve recognition accuracy.

[0086] For example, considering that there may be multiple charging ports on a vehicle, such as fast charging and slow charging ports, the relative positions between them are usually fixed, and this layout also has certain rules between different models. Therefore, by analyzing the relative positions between the contours, it is possible to further confirm which contours correspond to the real charging ports. Specifically, distance analysis, direction and arrangement pattern, combination verification, etc. can be used to further confirm which contours belong to the real charging ports. Among them, distance analysis refers to calculating the mutual distance between all potential contours and comparing them with the known standard distance between charging ports. If it is found that the distance between certain contours is consistent with expectations, it is more likely to be a real charging port. Direction and arrangement pattern means that in addition to simple distance measurement, the arrangement of charging ports can also be considered, such as whether they are arranged in a straight line, rectangular distribution, and other specific patterns, to help determine the valid contours. Combination verification refers to combining other prior knowledge, such as whether a charging port is usually located above or to the left of another charging port, etc., to further screen out valid contour combinations that meet the conditions.

[0087] Based on this, combining contour dimension analysis with relative position relationship analysis can not only improve the accuracy of identifying a single charging port, but also ensure that when there are multiple charging ports, the position of each port can be correctly distinguished and identified.

[0088] S403 : Determine the three-dimensional points in the point cloud image corresponding to the pixel points of each valid contour in the pixel image, and obtain a set of three-dimensional coordinates corresponding to each valid contour.

[0089] For example, the depth information provided by the point cloud image is used to convert the two-dimensional pixel points on each valid contour into points in three-dimensional space. Specifically, for each pixel point (x, y), by querying its depth value z in the point cloud image, the actual three-dimensional coordinates (x, y, z) of the point in the world coordinate system can be calculated. For each valid contour, the above process is repeated, and the three-dimensional coordinates of all points on the valid contour are collected to form a three-dimensional coordinate set. This three-dimensional coordinate set represents the spatial distribution of each charging port in the real world.

[0090] S404: Determine the current three-dimensional position of each charging port based on the three-dimensional coordinate set corresponding to each valid contour.

[0091] Alternatively, to simplify the representation, the center of each charging port can be calculated as its 3D position. For example, the centroid of the 3D coordinate set (i.e., the average of all point coordinates) can be calculated, or other statistical methods can be used to find the most representative point. Furthermore, the 3D coordinate set can be used to further analyze features such as the principal axis direction of the contour to estimate the charging port's posture.

[0092] Optionally, according to the three-dimensional coordinate set corresponding to each effective contour, the central three-dimensional coordinate of each effective contour is determined, and the central three-dimensional coordinate is used as the current three-dimensional position of the charging port.

[0093] For example, for each charging port that is confirmed as a valid contour, there is a corresponding three-dimensional coordinate set, and this three-dimensional coordinate set contains the spatial coordinates (x, y, z) of all points on the contour. Determining the central three-dimensional coordinates is also determining the center of mass. The center of mass is the center of mass of all points inside the object, which can usually be understood as the average value of the coordinates of all points. By calculating the center of mass, a single point representing the entire contour can be obtained, which is usually considered the center of the contour.

[0094] For example, once the centroid coordinates of each valid contour are calculated, they can be used as the current 3D position of the corresponding charging port. Because the centroid can well represent the position of the entire contour and is a single point, it also facilitates subsequent calculations and operations, such as comparing with template positions or guiding the movement of a robotic arm.

[0095] Based on this, the current three-dimensional position of the charging port is determined by calculating the central three-dimensional coordinates of each valid contour, ensuring the precise position of the charging port in the real world, thereby providing accurate data support for subsequent operations such as alignment processing and posture adjustment of the charging gun.

[0096] Figure 6 FIG2 shows a flow chart of a method for determining a first pose transformation relationship provided by an embodiment of the present application. Figure 6 As shown, the above step S203 performs a first registration process based on the current three-dimensional position of each charging hole and the three-dimensional position of the template of each charging hole to obtain the first pose transformation relationship between the current three-dimensional position of each charging hole and the three-dimensional position of the template, which specifically includes the following steps:

[0097] S601: Combine the current three-dimensional positions of all charging ports into a first ordered point set, and combine the three-dimensional positions of all templates into a second ordered point set.

[0098] For example, the first ordered point set is a set of three-dimensional coordinates of the actually detected charging ports, where the location of each charging port is represented by a three-dimensional coordinate (x, y, z). All three-dimensional coordinates are arranged in a certain order, such as from left to right or from top to bottom, to form an ordered point set. The second ordered point set is another set of preset charging port template positions. These template positions are the spatial coordinates where the charging ports should be located under ideal conditions. They are also given in the form of three-dimensional coordinates and arranged in the same order.

[0099] It should be noted that the corresponding points in the first ordered point set and the second ordered point set have the same meaning, that is, the first point in the first ordered point set corresponds to the first point in the second ordered point set, and so on. This ensures that the comparison between each pair of points is meaningful during alignment.

[0100] S602: Determine a first pose transformation relationship according to the first ordered point set and the second ordered point set.

[0101] Optionally, the first pose transformation relationship describes how to adjust the position and posture of the charging gun so as to accurately align it with the charging port. The purpose of determining the first pose transformation relationship is to find a rigid body transformation including rotation and translation so that the first ordered point set can match the second ordered point set as much as possible.

[0102] Optionally, the first ordered point set and the second ordered point set are used as input parameters of a preset iterative closest point registration algorithm, and the first pose transformation relationship is obtained by calculating the iterative closest point registration algorithm.

[0103] For example, the first ordered point set and the second ordered point set are passed as input to the Iterative Closest Point (ICP) algorithm, which is a classic algorithm widely used in point cloud registration and aims to minimize the distance error between the two point sets. Specifically, an initial transformation matrix is initialized. The initial transformation matrix is usually the unit matrix, which means there is no rotation and position. Then, corresponding points are found. For each point in the first ordered point set, the nearest point in the second ordered point set is found as the corresponding point. Based on the corresponding point pairs found, a rigid transformation that minimizes the distance between the point pairs is calculated through singular value decomposition (SVD), and the position of the first ordered point set is updated. The above process is repeated until the stopping condition is met, such as reaching the maximum number of iterations or the error is less than a certain threshold. Based on this, after multiple iterations, the ICP algorithm will output an optimal first pose transformation relationship, which defines the best way to transform from the first ordered point set to the second ordered point set.

[0104] Specifically, taking the first ordered point set as the source point set and the second ordered point set as the target point set as an example, first, for each point in the source point set, find the nearest point in the target point set as the corresponding point, then calculate the center of mass of the source point set and the target point set respectively, and translate all points relative to their respective centers of mass so that the origin of the new coordinate system is located at the center of mass. A covariance matrix is constructed based on the translated point set, singular value decomposition is performed on the covariance matrix, and a rotation matrix is calculated based on the singular value decomposition result, and the final translation vector is calculated using the rotation matrix and the center of mass position before translation. In this way, an optimal rotation matrix and translation vector can be obtained through singular value decomposition, so that the source point set can be aligned with the target point set as much as possible.

[0105] Based on this, the ICP algorithm is used to accurately calculate the optimal rigid body transformation between the two point sets, ensuring the accuracy of charging port identification and positioning in the automatic charging application scenario.

[0106] Figure 7 FIG2 shows a flow chart of a posture adjustment method provided in an embodiment of the present application. Figure 7 As shown, the above step S204 adjusts the posture of the charging gun according to the first posture transformation relationship, and controls the charging gun to be inserted into the charging port to charge the vehicle to be charged, which specifically includes the following steps:

[0107] S701: Taking the first pose transformation relationship as the initial current pose transformation relationship.

[0108] Exemplarily, the first pose transformation relationship obtained by ICP algorithm registration is used as the starting point, and the first pose transformation relationship is used as the initial current pose transformation relationship. The current pose transformation relationship contains the rotation and translation information required to convert the current position and posture of the charging gun to the ideal position that can be aligned with the charging port.

[0109] S702: Adjust the posture of the charging gun according to the current posture transformation relationship, and reacquire a second image of the charging port of the vehicle to be charged after the adjustment.

[0110] Exemplarily, based on the current posture transformation relationship, the robotic arm is driven to move the charging gun to adjust the posture of the charging gun, so that the position and posture of the charging gun change accordingly, and after the adjustment is completed, the charging port area of the vehicle to be charged is photographed again to obtain a second image, so as to check whether the new position of the charging gun after adjustment is close enough to the target based on the second image, or whether further fine-tuning is required.

[0111] S703: Perform image recognition on the second image to obtain new three-dimensional positions of each charging port.

[0112] For example, similar to the processing of the first image, after obtaining a second image (including a pixel image and a point cloud image) containing the charging port of the vehicle to be charged, the second image can be preprocessed, such as noise removal and registration. Furthermore, the features of the charging port and the charging hole are extracted from the pixel image of the second image, and combined with the depth information provided by the point cloud image, the two-dimensional features are mapped back to three-dimensional space. In this way, the specific three-dimensional coordinates of each charging hole can be located from the point cloud data. The charging hole area representing the charging port is then segmented from the entire point cloud data. For each charging hole, the data points are fitted to estimate the precise three-dimensional position of the charging hole center, and then the new three-dimensional position of each charging hole is obtained. The new three-dimensional coordinates provide the latest position information of the charging hole relative to the current viewing angle.

[0113] S704: Perform a second registration process based on the new three-dimensional position of each charging hole and the three-dimensional position of the template to obtain a second posture transformation relationship between the new three-dimensional position of each charging hole and the three-dimensional position of the template.

[0114] Exemplarily, the ICP registration algorithm is continued to be used to compare and align the obtained new three-dimensional positions of each charging port with the pre-stored standard template three-dimensional positions of each charging port. By comparing the actually detected charging port positions with the template positions, a second posture transformation relationship is determined. The second posture transformation relationship indicates the additional adjustments that need to be made to achieve the optimal docking state. By comparing the differences between the new and old positions, the final position and posture of the charging gun are further refined, so that the new three-dimensional position of the charging port matches the template three-dimensional position as much as possible.

[0115] S705. Determine whether the second posture transformation relationship meets the stop adjustment condition. If so, end the loop and control the charging gun to be inserted into the charging port to charge the vehicle to be charged; if not, use the second posture transformation relationship as the new current posture transformation relationship and re-execute step S701.

[0116] Exemplarily, the processing device evaluates whether the new second pose transformation relationship satisfies a stopping condition, i.e., whether the new pose meets a predetermined accuracy requirement or stopping criterion. The stopping condition includes, but is not limited to, the difference between two consecutive transformations being less than a threshold, the error being below a set limit, or the maximum number of iterations being reached.

[0117] For example, if the stop adjustment condition is met, it indicates that the charging gun is close enough to the ideal position and can be safely controlled to be inserted and charging can begin. If it is not met, the new posture transformation relationship is used as the basis for the next round of iteration, and the adjustment is continued in step S701 until the stop condition is met.

[0118] It should be noted that the present application uses the ICP registration algorithm to determine the first pose transformation relationship between the current three-dimensional position of the charging port and the template three-dimensional position of the charging port when the template pose is recorded. This is a rough registration of the charging port pose. On the basis of completing the rough registration, the first pose transformation relationship is used as the initial current pose transformation relationship. The charging robot continues to drive the camera to move the charging port to take a picture, and obtains a second image, that is, a new point cloud data of the charging port. The new point cloud data and the point cloud data when the template pose is recorded are then used as inputs of the ICP registration algorithm for iterative fine registration, and finally the accurate pose of the charging port is determined, so that the charging gun can be accurately placed in the charging port based on the guidance of the robot to complete automatic charging. Therefore, the present application further performs fine registration processing on the basis of rough registration. The user only needs to park the vehicle in a rough position in the designated area, and the robot can automatically charge the new energy vehicle with the charging gun under the guidance of the camera. In this way, the user can complete the energy replenishment operation without getting off the vehicle. Compared with the traditional manual charging method, it is more convenient and quick, greatly improving the user experience.

[0119] Based on this, the embodiments of the present application provide a closed-loop feedback mechanism that gradually approaches the ideal charging gun position through coarse and fine registration. Even if there are errors in the initial estimation, they can be gradually corrected through multiple iterations. This can significantly improve the success rate and efficiency of automatic charging, especially in complex environments or when the alignment is not perfect. Furthermore, through continuous adjustment and verification, it can ensure that the charging gun is accurately inserted into the charging port, achieving an efficient and reliable automatic charging process.

[0120] The embodiment of the present application further provides a processing device 800, such as Figure 8 FIG. 8 is a schematic diagram of the structure of a processing device 800 provided in an embodiment of the present application, comprising: a processor 801, a memory 802, and optionally, a bus 803. The memory 802 stores machine-readable instructions executable by the processor 801. When the processing device 800 is running, the processor 801 communicates with the memory 802 via the bus 803. When the machine-readable instructions are executed by the processor 801, the method steps of the vehicle charging method described in any one of the above items are performed.

[0121] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps of the vehicle charging method as described above are executed.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0123] In addition, the functional units in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0124] The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A vehicle charging method, characterized in that: include: In response to a vehicle to be charged entering a charging area, acquiring a first image including a charging port of the vehicle to be charged, the charging port including at least one charging hole, the first image including a pixel image and a point cloud image; Performing image recognition on the first image to obtain a current three-dimensional position of each of the charging ports; Performing a first registration process based on the current three-dimensional position of each charging port and the three-dimensional position of a template of each charging port to obtain a first pose transformation relationship between the current three-dimensional position of each charging port and the three-dimensional position of the template, wherein the three-dimensional position of the template is the three-dimensional position of the charging port when the charging gun is at a reference three-dimensional position and the charging gun can be inserted into the charging port; According to the first posture transformation relationship, the posture of the charging gun is adjusted, and the charging gun is controlled to be inserted into the charging port to charge the vehicle to be charged.

2. The method according to claim 1, characterized in that The step of adjusting the posture of the charging gun according to the first posture transformation relationship and controlling the charging gun to be inserted into the charging port to charge the vehicle to be charged includes: A. Using the first posture transformation relationship as the initial current posture transformation relationship; B. adjusting the posture of the charging gun according to the current posture transformation relationship, and reacquiring a second image of the charging port of the vehicle to be charged after the adjustment; C. performing image recognition on the second image to obtain a new three-dimensional position of each charging port; D. performing a second registration process based on the new three-dimensional position of each charging hole and the three-dimensional position of the template to obtain a second pose transformation relationship between the new three-dimensional position of each charging hole and the three-dimensional position of the template; E. Determine whether the second posture transformation relationship meets the stop adjustment condition. If so, end the loop and control the charging gun to be inserted into the charging port to charge the vehicle to be charged; if not, use the second posture transformation relationship as the new current posture transformation relationship and re-execute step A.

3. The method according to claim 1, characterized in that The performing image recognition on the first image to obtain the current three-dimensional position of each charging port includes: Inputting the pixel image into a pre-trained hole segmentation model, and having the hole segmentation model segment the pixel image to obtain segmentation information of at least one hole, the segmentation information including a category label, bounding box coordinates, and mask information; Based on the segmentation information of each of the holes, the current three-dimensional position of the charging hole is determined.

4. The method according to claim 3, characterized in that The determining the current three-dimensional position of the charging hole based on the segmentation information of each hole includes: extracting the outline of each of the holes from the mask information of each of the holes; Screening the contours of the holes to obtain a valid contour corresponding to at least one charging hole; Determine the three-dimensional points in the point cloud image corresponding to the pixel points of each valid contour in the pixel image, and obtain a set of three-dimensional coordinates corresponding to each valid contour; Based on the three-dimensional coordinate sets corresponding to the effective contours, the current three-dimensional position of each charging hole is determined.

5. The method according to claim 4, characterized in that The screening of the contours of the holes to obtain a valid contour corresponding to at least one charging hole includes: According to the preset contour features of each charging hole, the contours of each hole are screened to obtain a valid contour corresponding to at least one charging hole, wherein the preset contour features include: contour size and relative positional relationship between the contours of multiple charging holes.

6. The method according to claim 4, characterized in that The determining of the current three-dimensional position of each charging hole based on the three-dimensional coordinate set corresponding to each valid contour includes: According to the three-dimensional coordinate sets corresponding to the effective contours, the central three-dimensional coordinates of each effective contour are determined, and the central three-dimensional coordinates are used as the current three-dimensional position of the charging hole.

7. The method according to claim 1, characterized in that The first registration process is performed based on the current three-dimensional position of each charging hole and the three-dimensional position of the template of each charging hole to obtain a first pose transformation relationship between the current three-dimensional position of the charging hole and the three-dimensional position of the template, including: Combining the current three-dimensional positions of all charging holes into a first ordered point set, and combining the three-dimensional positions of all templates into a second ordered point set; The first posture transformation relationship is determined according to the first ordered point set and the second ordered point set.

8. The method according to claim 7, characterized in that The determining the first posture transformation relationship according to the first ordered point set and the second ordered point set includes: The first ordered point set and the second ordered point set are used as input parameters of a preset iterative closest point registration algorithm, and the first posture transformation relationship is calculated by the iterative closest point registration algorithm.

9. A charging robot, characterized in that: The charging robot includes a robotic arm and a processing device, wherein a charging gun and a photographing device are installed at the end of the robotic arm, and the positional relationship between the charging gun and the photographing device is fixed; The robotic arm is used to drive the photographing device to photograph the charging port of the vehicle to be charged, and the charging gun is used to charge the vehicle to be charged; The processing device is used to execute the steps of the vehicle charging method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the vehicle charging method according to any one of claims 1 to 8.

Images