Vehicle charging method, charging robot and computer readable storage medium

Abstract

The application provides a vehicle charging method, a charging robot and a computer readable storage medium, and relates to the technical field of vehicles. The method comprises the following steps: acquiring a first image containing a charging port of a vehicle to be charged; performing image recognition on the first image to obtain the current three-dimensional position of each charging hole; performing first registration processing according to the current three-dimensional position of each charging hole and the template three-dimensional position to obtain the first pose transformation relationship between the current three-dimensional position of each charging hole and the template three-dimensional position; adjusting the pose of the charging gun according to the first pose transformation relationship, and controlling the charging gun to be inserted into the charging port to charge the vehicle to be charged. The application determines the pose transformation relationship by fusing different types of image data in a closed-loop control mode, reacquires images and performs rapid correction after adjusting the pose of the charging gun each time, ensures that each adjustment is as close to the target position as possible, and significantly improves the reliability and efficiency of automatic charging.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a vehicle charging method, a charging robot, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of the new energy vehicle market, users' demand for convenient and efficient charging methods is increasing. Taking charging as an example, in practical applications, regardless of weather conditions, users need to get out of their vehicles to operate the charging equipment. This manual charging connection process takes a relatively long time, which not only affects the user experience but also reduces charging efficiency to some extent.

[0003] To address the aforementioned issues, existing technologies involve using robots carrying charging guns to charge new energy vehicles. However, this charging method relies on the vehicle's parking location and is easily affected by external environmental factors, leading to insufficient positioning accuracy of the charging port and consequently impacting charging success rate and efficiency. Therefore, accurately locating the charging port and guiding the robot to precisely complete the charging operation is one of the urgent problems to be solved in this field. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing a vehicle charging method, a charging robot, and a computer-readable storage medium, in order to solve the problem that the vehicle charging method in the prior art relies on the vehicle's parking location to perform the charging task, which is easily affected by external environmental factors, resulting in insufficient positioning accuracy of the vehicle charging port, and thus affecting the success rate and efficiency of charging.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, embodiments of this application provide a vehicle charging method, the method comprising:

[0007] In response to a vehicle entering the charging area, a first image containing the charging port of the vehicle is acquired, wherein the charging port includes at least one charging hole, and the first image includes a pixel image and a point cloud image.

[0008] Image recognition is performed on the first image to obtain the current three-dimensional position of each charging port;

[0009] According to the current three-dimensional positions of the charging holes and the template three-dimensional positions of the charging holes, a first registration process is performed to obtain a first pose transformation relationship between the current three-dimensional positions of the charging holes and the template three-dimensional positions, wherein the template three-dimensional positions are the three-dimensional positions of the charging holes when the charging gun is at a reference three-dimensional position and the charging gun can be inserted into the charging hole;

[0010] According to the first pose transformation relationship, the pose of the charging gun is adjusted, and the charging gun is controlled to be inserted into the charging hole to charge the vehicle to be charged.

[0011] As a possible implementation, according to the first pose transformation relationship, the pose of the charging gun is adjusted, and the charging gun is controlled to be inserted into the charging hole to charge the vehicle to be charged, comprising:

[0012] A. The first pose transformation relationship is taken as an initial current pose transformation relationship;

[0013] B. According to the current pose transformation relationship, the pose of the charging gun is adjusted, and a second image of the charging hole of the vehicle to be charged is reacquired after adjustment;

[0014] C. Image recognition is performed on the second image to obtain new three-dimensional positions of the charging holes;

[0015] D. According to the new three-dimensional positions of the charging holes and the template three-dimensional positions, a second registration process is performed to obtain a second pose transformation relationship between the new three-dimensional positions of the charging holes and the template three-dimensional positions;

[0016] E. It is determined whether the second pose transformation relationship satisfies a stop adjustment condition, if yes, the loop is ended, and the charging gun is controlled to be inserted into the charging hole to charge the vehicle to be charged; if not, the second pose transformation relationship is taken as a new current pose transformation relationship, and step A is re-executed.

[0017] As a possible implementation, the image recognition is performed on the first image to obtain the current three-dimensional positions of the charging holes, comprising:

[0018] The pixel image is input into a pre-trained hole segmentation model, and the pixel image is segmented by the hole segmentation model to obtain segmentation information of at least one hole, the segmentation information comprising a class label, a bounding box coordinate and mask information;

[0019] Based on the segmentation information of each hole, the current three-dimensional positions of the charging holes are determined.

[0020] As a possible implementation manner, the determining of the current three-dimensional position of the charging hole based on the segmentation information of each hole comprises:

[0021] extracting the contour of each hole from the mask information of each hole;

[0022] screening the contour of each hole to obtain an effective contour corresponding to at least one charging hole;

[0023] determining the three-dimensional point corresponding to the pixel point of each effective contour in the pixel image in the point cloud image to obtain a three-dimensional coordinate set corresponding to each effective contour;

[0024] determining the current three-dimensional position of each charging hole based on the three-dimensional coordinate set corresponding to each effective contour.

[0025] As a possible implementation manner, the screening of the contour of each hole to obtain an effective contour corresponding to at least one charging hole comprises:

[0026] screening the contour of each hole according to a preset contour feature of each charging hole to obtain an effective contour corresponding to at least one charging hole, wherein the preset contour feature comprises a contour size and a relative position relationship between the contours of a plurality of charging holes.

[0027] As a possible implementation manner, the determining of the current three-dimensional position of each charging hole based on the three-dimensional coordinate set corresponding to each effective contour comprises:

[0028] determining the center three-dimensional coordinate of each effective contour according to the three-dimensional coordinate set corresponding to each effective contour, and taking the center three-dimensional coordinate as the current three-dimensional position of the charging hole.

[0029] As a possible implementation manner, the performing of the first registration processing according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole to obtain the first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of the charging hole comprises:

[0030] combining the current three-dimensional positions of all charging holes into a first ordered point set, and combining the template three-dimensional positions into a second ordered point set;

[0031] determining the first pose transformation relationship according to the first ordered point set and the second ordered point set.

[0032] As a possible implementation manner, the determining of the first pose transformation relationship according to the first ordered point set and the second ordered point set comprises:

[0033] The first ordered point set and the second ordered point set are taken as input parameters of a preset iterative closest point registration algorithm, and the first pose transformation relationship is calculated by the iterative closest point registration algorithm.

[0034] In a second aspect, another embodiment of the present application provides a charging robot, which comprises a mechanical arm and a processing device, a charging gun and a shooting device are installed at the end of the mechanical arm, and the pose relationship between the charging gun and the shooting device is fixed and unchanged.

[0035] The mechanical arm is used to drive the shooting device to shoot the charging port of the vehicle to be charged, and the charging gun is used to charge the vehicle to be charged.

[0036] The processing device is used to execute the steps of the vehicle charging method according to any one of the first aspect.

[0037] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the vehicle charging method according to any one of the first aspect.

[0038] According to the vehicle charging method, the charging robot and the computer readable storage medium provided by the embodiments of the present application, in response to the vehicle to be charged driving into the charging area, a first image containing the charging port of the vehicle to be charged is obtained, the first image includes a pixel image and a point cloud image, image recognition is performed on the first image to obtain the current three-dimensional position of each charging hole, and first registration processing is performed according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole to obtain the first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of each charging hole, and then the pose of the charging gun is adjusted according to the first pose transformation relationship, and the charging gun is inserted into the charging port to charge the vehicle to be charged. According to the embodiments of the present application, the charging robot obtains more comprehensive and accurate environmental information, and improves the understanding ability of the charging port and the surrounding environment by fusing different types of image data, accurately segments and identifies the charging port area for registration processing, determines the pose transformation relationship between the current three-dimensional position and the template three-dimensional position of the charging hole, and then adjusts the pose of the charging gun according to the pose transformation relationship. And after adjusting the pose of the charging gun each time, the image is reacquired and quickly corrected to ensure that each adjustment is as close to the target position as possible. In addition, the charging robot based on visual guidance automatically charges, the user does not need to get off the vehicle and only needs to park the vehicle in the charging area, and the robot can automatically charge the vehicle with the charging gun under the guidance of the camera, which greatly improves the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor under the guidance of the content of the present application.

[0040] Figure 1 A structural schematic diagram of a charging robot is shown;

[0041] Figure 2 A flow schematic diagram of a vehicle charging method is shown;

[0042] Figure 3 A flow schematic diagram of a charging hole three-dimensional position determination method is shown;

[0043] Figure 4 A flow schematic diagram of another charging hole three-dimensional position determination method is shown;

[0044] Figure 5 A schematic diagram of a contour segmentation result is shown;

[0045] Figure 6 A flow schematic diagram of a first pose transformation relationship determination method is shown;

[0046] Figure 7 A flow schematic diagram of a pose adjustment method is shown;

[0047] Figure 8 A structural schematic diagram of a processing device is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only play the purpose of illustration and description, and do not limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportion. The flowchart shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowchart or one or more operations can be removed from the flowchart under the guidance of the content of the present application.

[0049] In addition, the described embodiments are only some embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0050] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0051] Figure 1 A structural schematic diagram of a charging robot is shown. Referring to Figure 1 As shown, the robot 1 includes a mechanical arm and a processing device, wherein the end of the mechanical arm is provided with a charging gun 3 and a shooting device, for example, a camera 2, and the pose relationship between the charging gun 3 and the camera 2 is fixed and unchangeable. When the vehicle to be charged is parked in the charging area, the robot 1 first drives the camera 2 to run to the shooting position to take a photo of the charging port 4 of the vehicle to be charged, and then calculates the accurate pose of the charging port 4.

[0052] It should be noted that since the pose relationship between the charging gun 3 and the camera 2 is fixed, it means that the relative position, angle and other parameter information between the charging gun 3 and the camera 2 are known and constant, so that the processing device can directly determine the position and attitude of the charging gun relative to the charging port of the vehicle to be charged according to the image obtained by the shooting device, without additional calibration or adjustment. Therefore, by using the fixed pose relationship, high-precision registration processing can be realized to ensure accurate positioning of the charging gun every time.

[0053] Optionally, the mechanical arm is used to drive the camera 2 to take a photo of the charging port 4 of the vehicle to be charged, and the mechanical arm has sufficient degrees of freedom to operate flexibly, so as to accurately move the charging gun 3 and the camera 2 to the specified position through the mechanical arm, so as to adapt to the charging port positions of different vehicle models. The shooting device is used to take a photo of the charging port of the vehicle to be charged, so as to determine the exact position and attitude of the charging port, and then combine the image recognition technology to calculate the coordinates of the charging port relative to the robot by analyzing the photographed image data. The charging gun is used to charge the vehicle to be charged. Once the position of the charging port is determined by the shooting device, the mechanical arm will adjust its attitude to make the charging gun accurately inserted into the charging port, and complete the charging task.

[0054] Optionally, when the vehicle to be charged enters the designated charging area, the processing device sends a photographing instruction to the photographing device, and the photographing device first acquires a first image containing the charging port of the vehicle to be charged in response to the photographing instruction, wherein the first image includes not only a pixel image but also a point cloud image to provide depth information. At least one charging hole is arranged in the charging port. The processing device performs high-level image recognition processing on the acquired first image to accurately locate the current three-dimensional positions of the charging holes, and performs first registration processing based on the current three-dimensional positions of the charging holes and the pre-defined three-dimensional position of the charging hole template, i.e., the ideal positions of the charging holes when the charging gun is at the reference three-dimensional position and ready for insertion into the charging port, to determine the first pose transformation relationship between the positions of the currently detected charging holes and the ideal three-dimensional positions of the template. Further, the processing device adjusts the pose of the charging gun accurately based on the obtained first pose transformation relationship, so that the charging gun accurately points to the charging port of the vehicle to be charged, and further controls the charging gun to be safely and reliably inserted into the charging port, thereby starting to charge the vehicle to be charged.

[0055] Therefore, the charging robot provided by the embodiments of the present application can automatically charge the vehicle, and the automatic process ensures that the charging task can be efficiently and accurately completed without human intervention, thereby greatly improving the user experience and operation efficiency.

[0056] The vehicle charging method provided by the embodiments of the present application will be described in detail below in combination with the content described in the charging robot shown in the above Figure 1 The vehicle charging method provided by the embodiments of the present application will be described in detail below in combination with the content described in the charging robot shown in the above

[0057] Figure 2 A flowchart of a vehicle charging method provided by the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the method specifically includes the following steps: Figure 2

[0058] S201, in response to the vehicle to be charged entering the charging area, a first image containing the charging port of the vehicle to be charged is acquired.

[0059] Optionally, the charging robot has a vehicle entry detection mechanism to detect whether the vehicle to be charged has entered the charging area, for example, using a ground inductance coil, a camera monitoring, a laser radar, etc. to identify the presence of the vehicle to be charged in the charging area.

[0060] Optionally, when it is detected that the vehicle to be charged enters the predetermined charging area, the charging robot starts and prepares to begin the charging process. Specifically, the charging robot needs to determine the exact position of the charging port, for this purpose, the processing device can send a photographing instruction to the photographing device, and control the mechanical arm to move the photographing device to the charging port area of the vehicle to be charged, and use the photographing device to capture an image containing the charging port, so as to obtain the first image containing the charging port of the vehicle to be charged.​

[0061] Optionally, the charging port comprises at least one charging hole, and the first image comprises a pixel image and a point cloud image. The pixel image, i.e. the RGB image, can provide rich color and texture information, which helps to identify the specific shape, color and other visual features of the charging port. The point cloud image, i.e. the depth image, not only contains the color information of the vehicle to be charged, but also provides spatial coordinate information, i.e. the exact position of each point in three-dimensional space, such as x, y, z coordinates, which is crucial for determining the three-dimensional position of the charging port.

[0062] S202, image recognition is performed on the first image to obtain the current three-dimensional position of each charging hole.

[0063] Optionally, after obtaining the first image (including pixel image and point cloud image) containing the charging port of the vehicle to be charged, the first image can be preprocessed, such as noise removal and registration. Noise removal refers to preprocessing the collected point cloud data to eliminate possible noise. Registration refers to the point cloud registration operation if multiple sensors are used or the data of multiple scans needs to be combined, to ensure that all data are in the same coordinate system.

[0064] Optionally, the features of the charging port and the charging hole are extracted from the pixel image, and combined with the depth information provided by the point cloud image to map the two-dimensional features back to the three-dimensional space. In this way, the specific three-dimensional coordinates of each charging hole can be located from the point cloud data. Then, the charging hole area representing the charging port is segmented from the entire point cloud data. For each charging hole, the precise three-dimensional position of the charging hole center is estimated by fitting the data points, and the current three-dimensional position of each charging hole is obtained.

[0065] S203, according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole, first registration processing is performed to obtain the first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of each charging hole.

[0066] Optionally, the template three-dimensional position is the three-dimensional position of the charging hole when the charging gun is at the reference three-dimensional position and the charging gun can be inserted into the charging port, i.e. the ideal coordinates of the charging hole relative to the charging gun when the charging gun is at the ideal reference position. This template three-dimensional position is usually a fixed value based on design specifications or obtained by multiple accurate measurements, representing the spatial coordinates where the charging hole should be located under the best docking conditions, i.e. the charging gun can be directly inserted without additional adjustment.

[0067] Optionally, before locating the charging port of the charging vehicle parked at a random position, the camera needs to first take a reference image of the charging port, which includes a point cloud image and a color image of the charging port, that is, the reference image not only contains the color image of the charging port and its surrounding environment, but also contains the corresponding point cloud data, which provides the three-dimensional spatial information of each pixel point. In the color image, mark the edge pixels of all the round holes in the charging port, and since the position of each point in the color image can be directly mapped to the same position in the point cloud image, the edge pixels can be one-to-one corresponding to the corresponding three-dimensional points in the point cloud image. Then, using the three-dimensional point set of the selected round hole edge, an ideal circle closest to the actual shape is fitted through a geometric fitting algorithm, and based on the fitted three-dimensional circle, the three-dimensional coordinates of the center of each round hole can be accurately calculated, and the calculated three-dimensional coordinates of the centers of all round holes are saved as template data, so as to obtain the template three-dimensional position of each charging hole.

[0068] Optionally, the current three-dimensional position of each charging hole obtained can be compared and registered with the pre-stored standard template three-dimensional position of each charging hole by using a registration algorithm. By comparing the actual detected charging hole position with the template position, a first pose transformation relationship is determined, so that the current three-dimensional position of the charging hole matches the template three-dimensional position as much as possible. The first pose transformation relationship is used to describe the rotation and translation required for the current detected charging hole three-dimensional position to be converted to the template three-dimensional position, that is, by optimizing the current three-dimensional position of each charging hole, the movement information of the charging gun is determined to ensure the accuracy of the charging hole position.

[0069] S204, adjust the pose of the charging gun according to the first pose transformation relationship, and control the charging gun to insert into the charging port to charge the vehicle to be charged.

[0070] Optionally, based on the first pose transformation relationship, the pose of the charging gun installed on the mechanical arm is adjusted, so that the position and direction of the charging gun in space meet the requirements, so as to be able to smoothly insert into the charging port. And after completing the pose adjustment of the charging gun, control the charging gun to advance according to the predetermined path and insert into the charging port to start charging the vehicle to be charged.

[0071] Optionally, the first pose transformation relationship obtained by the registration processing contains necessary rotation and translation parameters, and these parameters describe how to transform the charging gun from the current pose to the pose that can correctly dock with the charging port. Therefore, based on the transformation parameters contained in the first pose transformation relationship, a path planning is performed to determine a safe and effective motion path, so that the mechanical arm can smoothly move the charging gun to the target position, and drive the mechanical arm to accurately adjust the position and pose of the charging gun according to the predetermined path. When the charging gun approaches the calculated target position, the charging gun insertion speed is slowed down and the pose is adjusted more finely to ensure that the charging gun is aligned with the charging port, and the insertion action is performed and the charging program is started to begin to power the vehicle to be charged.

[0072] Therefore, according to the vehicle charging method provided by the embodiments of the present application, more comprehensive and accurate environmental information is obtained by the charging robot, and the understanding ability of the charging port and its surrounding environment is improved by fusing different types of image data, the charging port area is accurately segmented and identified for registration processing to determine the pose transformation relationship between the current three-dimensional position of the charging hole and the template three-dimensional position, and then the pose of the charging gun is adjusted according to the pose transformation relationship. And after adjusting the pose of the charging gun each time, the image is reacquired and quickly corrected to ensure that each adjustment is as close to the target position as possible. In addition, the charging robot based on visual guidance provided by the embodiments of the present application performs automatic charging, and the user does not need to get off the vehicle and only needs to park the vehicle in the charging area. The robot can automatically charge the vehicle with the charging gun under the guidance of the camera, greatly improving the user experience.

[0073] Figure 3 A flowchart of a charging hole three-dimensional position determination method provided by an embodiment of the present application is shown. Referring to Figure 3 As shown in FIG. 2, the above-mentioned step S202 performs image recognition on the first image to obtain the current three-dimensional position of each charging hole, which specifically includes the following steps:

[0074] S301, input the pixel image into the hole segmentation model obtained by pre-training, and segment the pixel image by the hole segmentation model to obtain segmentation information of at least one hole.

[0075] Optionally, the hole segmentation model is a pre-trained model through a machine learning algorithm, such as a convolutional neural network in deep learning, for identifying and segmenting specific types of objects, such as charging holes, in an image. Moreover, the hole segmentation model can not only identify different regions in the image, but also distinguish which regions belong to charging holes. Specifically, when a pixel image of the charging port of the vehicle to be charged is obtained, the pixel image is input into the hole segmentation model, and the hole segmentation model analyzes each pixel point in the pixel image and classifies it as a "charging hole" or a "non-charging hole" according to the characteristics of each pixel point, that is, the pixel image is divided into multiple parts, each part corresponds to a different category, by segmenting the pixel image.

[0076] Optionally, the segmentation information includes a category label, a bounding box coordinate, and mask information. The category label indicates whether a certain region is classified as a charging hole, for example, a digital label can be used to represent it. The bounding box coordinate indicates the position and size of a rectangular box around each detected charging hole, that is, it represents the approximate position and size of the charging hole. The mask information is more accurate information describing the shape of the charging hole than the bounding box coordinate, and is a binary image of the same size as the original image, in which the charging hole region is marked as 1 and the rest is marked as 0, thereby explicitly indicating the exact outline of the charging hole.

[0077] S302, based on the segmentation information of each hole, determining the current three-dimensional position of the charging hole.

[0078] Optionally, since the spatial position of the charging hole cannot be directly obtained from the pixel image, it is necessary to determine the current three-dimensional position of the charging hole in combination with the point cloud image containing depth information. Since the point cloud data provides the actual physical distance of each pixel point relative to the shooting device, based on the segmentation information, the specific position of the charging hole on the two-dimensional pixel image can be determined, and then the two-dimensional pixel position information is used to query the corresponding point cloud data to extract the depth value at the corresponding position, and the coordinates on the two-dimensional plane are converted into three-dimensional space coordinates, thereby obtaining the accurate position of the charging hole in the real world.

[0079] Figure 4 A flowchart of another method for determining the three-dimensional position of a charging hole is shown. Referring to Figure 4 The above step S302 determines the current three-dimensional position of the charging hole based on the segmentation information of each hole, and specifically includes the following steps:

[0080] S401, extracting the outline of each hole from the mask information of each hole.

[0081] Optionally, the mask information is binary information, in which the charging hole area is marked as 1, and the rest is marked as 0, thereby providing a clear boundary definition. By using a computer vision algorithm, such as OpenCV, the contour of each charging hole is extracted from the mask information. For example, referring to the schematic diagram of the contour segmentation result shown in FIG. 8, the contour is a closed curve composed of a series of points, which represents the shape of the charging hole edge. Figure 5

[0082] S402, screening the contour of each hole to obtain at least one effective contour corresponding to the charging hole.

[0083] It should be noted that not all detected contours are effective charging holes, and some invalid contours may exist due to noise or false recognition. Therefore, the contour of each hole needs to be screened to obtain an effective contour that truly represents the charging hole. For example, the contour is screened by size filtering, shape analysis, and position verification. Among them, size filtering refers to excluding those obviously too large or too small contours according to the approximate size range of the known charging hole. Shape analysis refers to checking whether the contour is close to a circle or other expected shape. For charging holes, a regular geometric shape is usually expected. Position verification refers to considering that charging holes generally appear in a specific area of the vehicle, and the contour can be further screened by position information.

[0084] Optionally, the contour of each hole is screened according to a preset contour feature of each charging hole to obtain at least one effective contour corresponding to the charging hole, wherein the preset contour feature includes a contour size and a relative position relationship between the contours of a plurality of charging holes.

[0085] For example, the contour size refers to the size or area of each detected contour. For charging holes, the actual physical size is known or at least has a rough range. Therefore, in the screening process, a reasonable threshold range can be set to exclude those obviously too large or too small contours. For example, a minimum size limit and a maximum size limit. Specifically, any contour smaller than the minimum size limit can be excluded because it may represent noise or other non-target objects. Similarly, the contour larger than the maximum size limit does not meet the standard of the charging hole and can also be excluded, thereby effectively reducing the possibility of misjudgment and improving the accuracy of identification.

[0086] ​Exemplarily, considering that there can be multiple charging holes on a vehicle, such as fast charging and slow charging holes, the relative positions between them are usually fixed, and such a layout also has certain regularity among different vehicle models, so it can be further confirmed which contours correspond to the real charging holes by analyzing the relative positions between the contours. Specifically, it can be further confirmed which contours belong to the real charging holes through distance analysis, direction and arrangement mode, combination verification and the like. Among them, distance analysis refers to calculating the mutual distance between all potential contours and comparing it with the standard distance between known charging holes, if it is found that the distance between certain contours is consistent with the expectation, it is more likely to be a real charging hole. Direction and arrangement mode refers to considering the arrangement mode of the charging hole in addition to simple distance measurement, such as whether it is arranged in a straight line, rectangular distribution or other specific mode, so as to help determine the effective contour. Combination verification refers to further filtering out the effective contour combination that meets the conditions in combination with other priori knowledge, for example, a certain charging hole is usually located above or to the left of another charging hole.

[0087] Based on this, the combination of contour size analysis and relative position relationship analysis can not only improve the accuracy of single charging hole recognition, but also ensure that when there are multiple charging holes, each hole can be correctly distinguished and recognized.

[0088] S403, determine the three-dimensional points corresponding to the pixel points of each effective contour in the pixel image in the point cloud image, and obtain a three-dimensional coordinate set corresponding to each effective contour.

[0089] Exemplarily, the depth information provided by the point cloud image is used to convert the two-dimensional pixel points on each effective contour into points in the three-dimensional space. Specifically, for each pixel point (x, y), the actual three-dimensional coordinates (x, y, z) of the point in the world coordinate system can be calculated by querying the depth value z of the point in the point cloud image. For each effective contour, repeat the above process to collect the three-dimensional coordinates of all points on the effective contour to form a three-dimensional coordinate set, which represents the spatial distribution of each charging hole in the real world.

[0090] S404, determine the current three-dimensional position of each charging hole based on the three-dimensional coordinate set corresponding to each effective contour.

[0091] Optionally, in order to simplify the representation, the center position of each charging hole can be calculated as its three-dimensional position, for example, the centroid of the three-dimensional coordinate set, that is, the average value of all point coordinates, or other statistical methods are used to find the most representative point. Further, the main axis direction and other features of the contour can also be analyzed based on the three-dimensional coordinate set to estimate the attitude of the charging hole.

[0092] Optionally, the center three-dimensional coordinates of each effective contour are determined according to the three-dimensional coordinate set corresponding to each effective contour, and the center three-dimensional coordinates are taken as the current three-dimensional position of the charging hole.

[0093] Illustratively, for each charging hole confirmed as an effective contour, there is a corresponding three-dimensional coordinate set, and the three-dimensional coordinate set contains the spatial coordinates (x, y, z) of all points on the contour. The center three-dimensional coordinates, i.e., the center of mass, are determined. The center of mass is the mass center of all points inside the object, which can be generally understood as the average of all point coordinates, and by calculating the center of mass, a best point representing the entire contour can be obtained, which is generally regarded as the center of the contour.

[0094] Illustratively, once the center of mass coordinates of each effective contour are calculated, these center of mass coordinates can be taken as the current three-dimensional position of the corresponding charging hole. Since the center of mass can well represent the position of the entire contour, and the center of mass is a single point, it is also convenient for subsequent calculation and operation, such as comparison with the template position or guidance of the movement of the mechanical arm.

[0095] Based on this, the current three-dimensional position of the charging hole is determined by calculating the center three-dimensional coordinates of each effective contour, which ensures that the accurate position of the charging hole in the real world is obtained, thereby providing accurate data support for subsequent operations such as registration processing and pose adjustment of the charging gun.

[0096] Figure 6 A flowchart of a first pose transformation relationship determination method provided by an embodiment of the present application is shown. Referring to Figure 6 As shown in FIG. 6, the above step S203 performs first registration processing according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole, to obtain the first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of each charging hole, and specifically includes the following steps:

[0097] S601, the current three-dimensional positions of all charging holes are combined into a first ordered point set, and the template three-dimensional positions of all charging holes are combined into a second ordered point set.

[0098] Illustratively, the first ordered point set is a set composed of three-dimensional coordinates of actually detected charging holes, where the position of each charging hole is represented as a three-dimensional coordinate (x, y, z), and all three-dimensional coordinates are arranged in a certain order, such as from left to right or from top to bottom, to form an ordered point set. The second ordered point set is another set composed of preset charging hole template positions, which are the spatial coordinates where the charging holes should be located in an ideal state, and are also given in the form of three-dimensional coordinates and arranged in the same order.

[0099] It should be noted that the corresponding points in the first ordered point set and the second ordered point set have the same meaning, i.e. the first point in the first ordered point set corresponds to the first point in the second ordered point set, and so on, so as to ensure that the comparison between each pair of points is meaningful when registration is performed.

[0100] S602, determining a first pose transformation relationship according to the first ordered point set and the second ordered point set.

[0101] Optionally, the first pose transformation relationship describes how to adjust the position and pose of the charging gun so as to accurately align the charging port, and the purpose of determining the first pose transformation relationship is to find a rigid body transformation including rotation and translation, so that the first ordered point set can match the second ordered point set as much as possible.

[0102] Optionally, the first ordered point set and the second ordered point set are taken as input parameters of a preset iterative closest point registration algorithm, and the first pose transformation relationship is calculated through the iterative closest point registration algorithm.

[0103] Exemplarily, the first ordered point set and the second ordered point set are taken as inputs and passed to an iterative closest point (ICP) registration algorithm, which is a classic algorithm widely used in point cloud registration and aims to minimize the distance error between two point sets. Specifically, an initial transformation matrix is initialized, which is usually a unit matrix, i.e. it means no rotation and position, then the corresponding points are found, for each point in the first ordered point set, the nearest point in the second ordered point set is found as the corresponding point. Based on the found corresponding point pair, a rigid transformation that minimizes the distance between the point pairs is calculated through singular value decomposition (SVD), and the position of the first ordered point set is updated, and the above process is repeated until a stop condition is met, for example, the maximum number of iterations is reached or the error is less than a certain threshold. Based on this, after multiple iterations, the ICP algorithm outputs an optimal first pose transformation relationship, which defines the best way to transform from the first ordered point set to the second ordered point set.

[0104] Specifically, taking the first ordered point set as the source point set and the second ordered point set as the target point set as an example, first, for each point in the source point set, find the nearest point in the target point set as the corresponding point, then calculate the centroids of the source point set and the target point set respectively, and translate all the points relative to the respective centroids so that the new coordinate system origin is located at the centroid. Based on the translated point set, a covariance matrix is constructed, singular value decomposition is performed on the covariance matrix, and a rotation matrix is calculated according to the singular value decomposition result, and a final translation vector is calculated using the rotation matrix and the centroid position before translation. In this way, through singular value decomposition, an optimal rotation matrix and translation vector can be obtained, so that the source point set can be aligned with the target point set as much as possible.

[0105] Based on this, the best rigid body transformation between the two point sets is accurately calculated through the ICP algorithm, ensuring the accuracy of the recognition and positioning of the charging hole in the automatic charging application scenario.

[0106] Figure 7 A flowchart of a pose adjustment method provided by an embodiment of the application is shown. Referring to Figure 7 As shown in FIG. 7, the above step S204 adjusts the pose of the charging gun according to the first pose transformation relationship, and controls the charging gun to be inserted into the charging port to charge the vehicle to be charged, and specifically includes the following steps:

[0107] S701, taking the first pose transformation relationship as an initial current pose transformation relationship.

[0108] Exemplarily, the first pose transformation relationship obtained by using the ICP algorithm for registration is taken as a starting point, and the first pose transformation relationship is taken as an initial current pose transformation relationship, which contains the rotation and translation information required to convert from the current position and attitude of the charging gun to the ideal position that can align with the charging hole.

[0109] S702, adjusting the pose of the charging gun according to the current pose transformation relationship, and reacquiring a second image of the charging port of the vehicle to be charged after adjustment.

[0110] Exemplarily, based on the current pose transformation relationship, the mechanical arm is driven to move the charging gun to adjust the pose of the charging gun, so that the position and attitude of the charging gun change correspondingly, and after the adjustment is completed, the charging port area of the vehicle to be charged is photographed again to obtain a second image, so as to check whether the new position of the charging gun after adjustment is close enough to the target or whether further fine adjustment is needed.

[0111] S703, performing image recognition on the second image to obtain new three-dimensional positions of each charging hole.

[0112] Exemplarily, similar to the processing of the first image, after obtaining the second image (including pixel image and point cloud image) containing the charging port of the vehicle to be charged, the second image can be pre-processed first, such as noise removal and registration, etc. Further, the features of the charging port and the charging holes are extracted from the pixel image of the second image, and combined with the depth information provided by the point cloud image, the two-dimensional features are mapped back to the three-dimensional space, so that the specific three-dimensional coordinates of each charging hole can be located from the point cloud data. Then, the charging hole area representing the charging port is segmented from the entire point cloud data, and for each charging hole, the accurate three-dimensional position of the center of the charging hole is estimated by fitting the data points, and then the new three-dimensional position of each charging hole is obtained, which provides the latest position information of the charging hole relative to the current view angle.

[0113] S704, according to the new three-dimensional position of each charging hole and the template three-dimensional position, a second registration processing is performed to obtain a second pose transformation relationship between the new three-dimensional position of each charging hole and the template three-dimensional position.

[0114] Exemplarily, the ICP registration algorithm is still used to compare and register the obtained new three-dimensional position of each charging hole with the pre-stored standard template three-dimensional position of each charging hole, and by comparing the actual detected charging hole position with the template position, the second pose transformation relationship is determined, which indicates the additional adjustment that needs to be made to achieve the best docking state. In this way, by comparing the differences between the new and old positions, the final position and attitude of the charging gun are further refined, so that the new three-dimensional position of the charging hole matches the template three-dimensional position as much as possible.

[0115] S705, it is determined whether the second pose transformation relationship satisfies a stop adjustment condition, if yes, the loop is ended, and the charging gun is controlled to be inserted into the charging port to charge the vehicle to be charged; if not, the second pose transformation relationship is taken as a new current pose transformation relationship, and step S701 is re-executed.

[0116] Exemplarily, the processing device will evaluate whether the new second pose transformation relationship satisfies the stop adjustment condition, that is, whether the new pose meets the predetermined accuracy requirement or stop standard. Among them, the stop adjustment condition includes but is not limited to that the difference between two consecutive transformations is less than a certain threshold, the error is lower than the set limit, or the maximum iteration number has been reached, etc.

[0117] Exemplarily, if the stop adjustment condition is met, it indicates that the charging gun has reached a position close enough to the ideal position, and the charging gun can be safely controlled to be inserted and start charging. If not, the new pose transformation relationship is used as the basis for the next iteration, and step S701 is returned to continue adjustment until the stop condition is met.

[0118] It should be noted that the ICP registration algorithm is used to determine the first pose transformation relationship between the current three-dimensional position of the charging port and the template three-dimensional position of the charging port when the template pose is recorded, which is a coarse registration of the charging port pose. On the basis of completing the coarse registration, the first pose transformation relationship is taken as the initial current pose transformation relationship, the charging robot continues to move the camera to take pictures of the charging port, and a second image is obtained, that is, new point cloud data of the charging port is obtained. Then, the new point cloud data and the point cloud data when the template pose is recorded are taken as inputs of the ICP registration algorithm for iterative fine registration, and finally the accurate pose of the charging port is determined, so as to accurately put the charging gun into the charging port based on the robot guidance, and complete automatic charging. Therefore, the application further performs fine registration processing on the basis of coarse registration, and the user only needs to park the vehicle at a rough position in a specified area, and the robot can automatically charge the new energy vehicle under the guidance of the camera. In this way, the user can conveniently complete the energy supplementing operation without getting off the vehicle, which is more convenient and faster than the traditional manual charging method, and greatly improves the user experience.

[0119] Based on this, the embodiment of the application provides a closed-loop feedback mechanism, which gradually approaches the ideal charging gun position through coarse registration and fine registration. Even if there is an error in the initial estimation, it can be gradually corrected through multiple iterations. Especially in the face of complex environment or incomplete matching, the success rate and efficiency of automatic charging can be significantly improved. And through continuous adjustment and verification, the charging gun can be accurately inserted into the charging port, realizing efficient and reliable automatic charging process.

[0120] The embodiment of the application also provides a processing device 800, as shown in Figure 8 The structure schematic diagram of the processing device 800 provided by the embodiment of the application includes a processor 801, a memory 802, and optionally a bus 803. The memory 802 stores machine readable instructions executable by the processor 801. When the processing device 800 is running, the processor 801 communicates with the memory 802 through the bus 803. The machine readable instructions are executed by the processor 801 to perform the method steps of the vehicle charging method according to any one of the above.

[0121] The embodiment of the application also provides a computer readable storage medium, which stores a computer program. When the processor runs the computer program, the method steps of the vehicle charging method according to any one of the above are executed.

[0122] Those skilled in the art can clearly understand the specific working process of the system and the device described above for the convenience and brevity of description, which can refer to the corresponding process in the method embodiment, and will not be repeated herein. In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and the actual implementation can have another division, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some communication interface, device or module, which can be electrical, mechanical or other forms.

[0123] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. When the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0124] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A vehicle charging method characterized by, The method comprises the following steps: in response to a to-be-charged vehicle entering a charging area, a first image containing a charging port of the to-be-charged vehicle is acquired, the charging port comprises at least one charging hole, and the first image comprises a pixel image and a point cloud image; the pixel image is input into a pre-trained hole segmentation model, the pixel image is segmented by the hole segmentation model to obtain segmentation information of at least one hole, and the segmentation information comprises a category label, a bounding box coordinate and mask information; the contour of each hole is extracted from the mask information of each hole; the contour of each hole is screened to obtain an effective contour corresponding to at least one charging hole; the pixel points of each effective contour in the pixel image are determined in the corresponding three-dimensional points in the point cloud image to obtain a three-dimensional coordinate set corresponding to each effective contour; based on the three-dimensional coordinate set corresponding to each effective contour, the current three-dimensional position of each charging hole is determined; first registration processing is performed according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole, and a first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of each charging hole is obtained, wherein the template three-dimensional position is the three-dimensional position of the charging hole when the charging gun is at a reference three-dimensional position and the charging gun can be inserted into the charging port; the pose of the charging gun is adjusted according to the first pose transformation relationship, and the charging gun is controlled to be inserted into the charging port to charge the to-be-charged vehicle.

2. The method of claim 1, wherein, The method of adjusting the pose of the charging gun according to the first pose transformation relationship and controlling the charging gun to be inserted into the charging port to charge the to-be-charged vehicle comprises: A. the first pose transformation relationship is taken as an initial current pose transformation relationship; B. the pose of the charging gun is adjusted according to the current pose transformation relationship, and a second image of the charging port of the to-be-charged vehicle is re-acquired after the adjustment; C. image recognition is performed on the second image to obtain new three-dimensional positions of each charging hole; D. second registration processing is performed according to the new three-dimensional positions of each charging hole and the template three-dimensional position to obtain a second pose transformation relationship between the new three-dimensional positions of each charging hole and the template three-dimensional position; E. it is determined whether the second pose transformation relationship satisfies a stop adjustment condition, if yes, the cycle is ended, and the charging gun is controlled to be inserted into the charging port to charge the to-be-charged vehicle; if not, the second pose transformation relationship is taken as a new current pose transformation relationship, and step A is re-executed.

3. The method of claim 1, wherein, The method of screening the contour of each hole to obtain an effective contour corresponding to at least one charging hole comprises: the contour of each hole is screened according to preset contour features of each charging hole to obtain an effective contour corresponding to at least one charging hole, wherein the preset contour features comprise a contour size and a relative position relationship between the contours of a plurality of charging holes.

4. The method of claim 1, wherein, The method of determining the current three-dimensional position of each charging hole based on the three-dimensional coordinate set corresponding to each effective contour comprises: According to the three-dimensional coordinate set corresponding to each effective contour, a center three-dimensional coordinate of each effective contour is determined, and the center three-dimensional coordinate is taken as a current three-dimensional position of the charging hole.

5. The method of claim 1, wherein, The first registration processing is performed according to the current three-dimensional position of each charging hole and the template three-dimensional position of each charging hole, to obtain a first pose transformation relationship between the current three-dimensional position and the template three-dimensional position of the charging hole, including: The current three-dimensional positions of all charging holes are combined into a first ordered point set, and the template three-dimensional positions are combined into a second ordered point set. The first pose transformation relationship is determined according to the first ordered point set and the second ordered point set.

6. The method of claim 5, wherein, The first pose transformation relationship is determined according to the first ordered point set and the second ordered point set, including: The first ordered point set and the second ordered point set are taken as input parameters of a preset iterative closest point registration algorithm, and the first pose transformation relationship is calculated by the iterative closest point registration algorithm.

7. A charging robot characterized by, The charging robot includes a mechanical arm and a processing device, an end of the mechanical arm is provided with a charging gun and a shooting device, and a pose relationship between the charging gun and the shooting device is fixed and unchanged; The mechanical arm is used to drive the shooting device to shoot a charging port of the vehicle to be charged, and the charging gun is used to charge the vehicle to be charged; The processing device is used to execute the steps of the vehicle charging method in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by the processor to execute the steps of the vehicle charging method in any one of claims 1-6.

Images