Charging pile pose calculation method and device, computer equipment and storage medium

Abstract

The invention relates to a charging pile pose calculation method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a charging pile image, inputting the charging pile image into a pre-trained charging pile target positioning model, and outputting a target positioning frame through the charging pile target positioning model; performing feature extraction based on the target positioning frame to obtain a feature point coordinate; based on a predetermined camera internal reference matrix and a distortion coefficient, correcting the feature point coordinates to obtain corrected feature point coordinates; and based on the correction feature point coordinates, predetermined three-dimensional feature coordinates of the charging pile and a conversion relation between a camera coordinate system and a machine coordinate system, determining a relative pose with the charging pile. By adopting the method, the pose can be identified in a complex environment.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, computer equipment, and storage medium for calculating the pose of a charging pile. Background Technology

[0002] With the development of automation technology, more and more scenarios rely on machines to complete tasks automatically. For example, in the field of lawnmowers or robot vacuums, after the task is set, the machine will automatically return to the charging station to recharge when the battery is low. At this time, it is necessary to detect the position of the charging station. The position detection of the transmission mainly relies on two types of technical solutions:

[0003] 1. Positioning based on traditional sensors. Distance and angle information between the machine and the charging station are obtained using LiDAR and ultrasonic sensors. Combined with preset charging station size parameters, the relative pose is calculated. This method is suitable for open, unobstructed, and simple environments.

[0004] 2. Localization based on basic machine vision. This method uses industrial cameras to capture images of charging piles, extracts the outline of the charging piles through algorithms such as edge detection and color thresholding, and then calculates the pose using a geometric model. This method is commonly used in scenarios with fixed lighting and a simple background.

[0005] However, neither of the above two methods can identify poses in complex environments, and their accuracy is low. Summary of the Invention

[0006] Therefore, it is necessary to provide a method, device, computer equipment, and storage medium for calculating the pose of a charging pile to address the aforementioned technical problems.

[0007] Firstly, this disclosure provides a method for calculating the pose of a charging pile. The method includes:

[0008] Acquire an image of a charging pile, input the image into a pre-trained charging pile target localization model, and output a target localization box through the charging pile target localization model;

[0009] Feature extraction is performed based on the target localization box to obtain the coordinates of the feature points;

[0010] Based on a predetermined camera intrinsic parameter matrix and distortion coefficients, the coordinates of the feature points are corrected to obtain the corrected feature point coordinates.

[0011] Based on the coordinates of the corrected feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system, the relative pose with the charging pile is determined.

[0012] In one embodiment, the method further includes:

[0013] Acquire charging pile images captured in various environmental scenarios, annotate the target bounding boxes of the charging piles contained in the charging pile images captured in the various environmental scenarios, and obtain training data;

[0014] The training data is augmented, and the augmented training data is divided into a training set, a validation set, and a test set.

[0015] The target detection model is trained using the training set, validation set, and test set to obtain the charging pile target localization model.

[0016] In one embodiment, the target location bounding box includes: a first shape category and a second shape category; the step of extracting features based on the target location bounding box to obtain feature point coordinates includes:

[0017] Based on the target positioning bounding box, a local target bounding box is obtained by cropping. Edge detection is performed on the local target bounding box to determine the target shape category to which the local target bounding box belongs.

[0018] In response to the local target box being of a first shape category, the coordinates of multiple local feature points within the local target box are determined based on the shape feature information of the first shape category;

[0019] In response to the target positioning box being of the second shape category, the coordinates of multiple local feature points in the local target box are determined based on the shape feature information of the second shape category;

[0020] The coordinate information of the target point in the local target within the target positioning frame is determined, and the coordinate information is used to perform original image coordinate mapping on the coordinates of the multiple local feature points to obtain the feature point coordinates.

[0021] In one embodiment, the camera intrinsic parameter matrix includes: focal length and principal point coordinates; the distortion coefficients include: radial distortion coefficients and tangential distortion coefficients; the step of correcting the feature point coordinates based on the predetermined camera intrinsic parameter matrix and distortion coefficients to obtain corrected feature point coordinates includes:

[0022] Using the focal length and the principal point coordinates, the feature point coordinates are normalized to obtain normalized coordinates;

[0023] Based on the radial distortion coefficient and the tangential distortion coefficient, the normalized coordinates are distorted to obtain the coordinates of the corrected feature points.

[0024] In one embodiment, determining the relative pose with respect to the charging pile based on the coordinates of the corrected feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system includes:

[0025] Based on the coordinates of the corrected feature points and the pre-determined three-dimensional feature coordinates of the charging pile, the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system are determined.

[0026] Based on the transformation relationship between the camera coordinate system and the machine coordinate system, and the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system, the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system are determined.

[0027] The relative pose with respect to the charging pile is determined based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system.

[0028] In one embodiment, the relative pose includes: three-dimensional distance and angle; determining the relative pose with respect to the charging pile based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system includes:

[0029] Based on the translation vector, the three-dimensional distance to the charging pile is determined;

[0030] The rotation matrix is ​​transformed using Euler angles to determine the angle with the charging pile.

[0031] The method further includes: performing regression charging based on the three-dimensional distance to the charging pile and the angle to the charging pile.

[0032] Secondly, this disclosure also provides a charging pile pose calculation device. The device includes:

[0033] The model processing module is used to acquire charging pile images, input the charging pile images into a pre-trained charging pile target localization model, and output target localization boxes through the charging pile target localization model.

[0034] The feature point extraction module is used to extract features and map the original image coordinates based on the target localization box to obtain the feature point coordinates;

[0035] The feature point correction module is used to correct the coordinates of the feature points based on a predetermined camera intrinsic parameter matrix and distortion coefficients, so as to obtain the corrected feature point coordinates.

[0036] The pose determination module is used to determine the relative pose with respect to the charging pile based on the coordinates of the correction feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system.

[0037] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in any of the above method embodiments.

[0038] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.

[0039] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0040] In the above embodiments, coarse positioning is achieved through a charging pile target positioning model. The model, trained with multi-scene data, can automatically identify charging piles and output target positioning boxes, effectively eliminating background (weeds, sky) interference and solving the problems of susceptibility to occlusion and lighting effects found in traditional methods. Feature point coordinates are corrected using camera intrinsic parameter matrices and distortion coefficients to eliminate errors caused by lens optical distortion. By combining three-dimensional feature coordinates to calculate relative pose, the pose estimation error can be ultimately controlled within 2°, and the detection blind zone reduced to 3cm, solving the problems of low accuracy and large blind zones found in traditional methods. The final output relative pose (three-dimensional distance + three attitude angles) can directly provide movement control commands for intelligent agricultural machinery (such as lawnmowers) (e.g., moving forward 0.5 meters, adjusting to the left by 2°), ensuring the machine quickly and accurately docks with the charging pile, achieving fully automatic recharging, and improving the intelligence level of the equipment and user experience. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a charging pile pose calculation method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating the model training steps in one embodiment;

[0044] Figure 3 This is a schematic diagram of the charging pile structure in one embodiment;

[0045] Figure 4This is a flowchart illustrating step S104 in one embodiment;

[0046] Figure 5 This is a flowchart illustrating step S106 in one embodiment;

[0047] Figure 6 This is a flowchart illustrating step S108 in one embodiment;

[0048] Figure 7 This is a flowchart illustrating step S506 in one embodiment;

[0049] Figure 8 This is a structural block diagram of a charging pile pose calculation device in one embodiment;

[0050] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0052] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0053] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0054] As described in the background section, traditional technologies, such as those used in lawnmowers, often suffer from limitations. The lidar installed on these devices is easily obscured by weeds and fallen leaves, resulting in missing point cloud data and an inability to accurately locate charging stations. Furthermore, ultrasonic sensors are susceptible to interference from environmental noise (such as wind and mechanical noise), leading to ranging errors exceeding 5%, which fails to meet the accuracy requirements for recharging. Therefore, their detection pose is poorly adaptable to complex environments. Additionally, existing technologies often rely on single graphic features (such as contours or single marker points) for detection, resulting in dual blind spots: unclear features at long distances and limited field of view at close range, leading to ineffective positioning within certain distance ranges. Machine vision, relying solely on contour features, cannot capture complete feature contours when the machine is too close to the charging station, resulting in a blind spot of approximately 30cm.

[0055] The above-mentioned technical problems have not been resolved. In one embodiment, such as... Figure 1 As shown, a method for calculating the pose of a charging pile is provided. Taking the application of this method to a lawnmower as an example, the method includes the following steps:

[0056] S102, acquire the charging pile image, input the charging pile image into the pre-trained charging pile target localization model, and output the target localization box through the charging pile target localization model.

[0057] The charging pile image typically refers to the original image (which may include backgrounds such as lawn, weeds, and sky, with resolution depending on camera parameters, such as 1920×1080) captured by the camera of a smart agricultural machine (such as a smart lawnmower) during automatic recharging. The pre-trained charging pile target localization model can be a deep learning model trained on the YOLOv5 architecture. It can be trained using a multi-scene charging pile image dataset (including sunny / cloudy, occluded / unobstructed, near / far-range, etc.) and can identify the location of the charging pile in the image in real time, outputting the coordinates of the localization box. The target localization box is typically a rectangle output by the model that surrounds the charging pile in the image, with coordinates in (x...)... min y min x max y max ), where (x min y min (x) represents the pixel coordinates of the top-left corner of the rectangle in the original image. max y max () represents the bottom right pixel coordinates, used to pinpoint the area of ​​the charging station in the image.

[0058] Specifically, when intelligent agricultural machinery (such as lawnmowers) activates its automatic recharging mode, its onboard camera captures real-time images of the surrounding environment. When a charging pile is present in the image (i.e., "acquiring the charging pile image"), this image is input into a pre-trained charging pile target localization model. The model uses a deep learning algorithm (based on charging pile features learned during training, such as circular charging ports and rectangular outlines) to quickly identify and locate the charging pile in the image, ultimately outputting the coordinates of a rectangular bounding box ("target localization box") surrounding the charging pile. This box accurately pinpoints the charging pile's location within the original image. Furthermore, it should be noted that the entire process must meet real-time performance requirements (model inference frame rate ≥ 50fps) and high accuracy (target detection mAP ≥ 95%) to adapt to dynamic scenarios involving machinery movement.

[0059] S104, feature extraction is performed based on the target location box to obtain the coordinates of the feature points.

[0060] Feature extraction involves using machine vision algorithms to extract key shape features within the target location bounding box of a charging station, ultimately obtaining feature points.

[0061] Specifically, the charging piles in the target positioning frame can be identified, and after identification, feature extraction can be performed to determine the feature points, and then the coordinates of the feature points can be determined.

[0062] S106, Based on the predetermined camera intrinsic parameter matrix and distortion coefficients, the coordinates of the feature points are corrected to obtain the corrected feature point coordinates.

[0063] The pre-determined camera intrinsic parameter matrix is ​​typically a parameter matrix (denoted as K) representing the camera's internal optical and geometric characteristics, including focal length and principal point coordinates. This matrix is ​​obtained in advance through camera calibration experiments (such as the Zhang Zhengyou calibration method). Distortion coefficients, usually parameters describing the camera lens's optical distortion (radial and tangential distortion), are also obtained through camera calibration and are used to correct pixel position deviations caused by lens defects. Feature point coordinates are typically the uncorrected pixel coordinates of charging pile feature points (such as the center of a circular feature or the vertices of a rectangular feature) extracted from the target bounding box in the original image. Corrected feature point coordinates are the pixel coordinates of feature points corrected using the camera intrinsic parameter matrix and distortion coefficients. This eliminates the influence of lens distortion, more closely approximating the actual physical position, and serves as accurate input for subsequent pose calculations.

[0064] Specifically, the intrinsic parameter matrix and distortion coefficients obtained beforehand through camera calibration are acquired. Next, the feature point coordinates extracted in the previous steps are corrected, converting the pixel coordinates into normalized coordinates in the camera coordinate system. The normalized coordinates are then corrected using the distortion coefficients to obtain the corrected normalized coordinates. Finally, the corrected normalized coordinates are converted back to pixel coordinates to obtain the corrected feature point coordinates.

[0065] S108, based on the coordinates of the correction feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system, determine the relative pose with the charging pile.

[0066] The pre-determined 3D feature coordinates of the charging pile typically represent the 3D coordinates (in the format (X, Y, Z)) of a fixed number of feature points (centers of circular features or vertices of rectangular features) on the charging pile in the "charging pile coordinate system," predefined by the physical dimensions and structural design of the charging pile (e.g., setting the center of the charging pile as the origin and determining fixed values ​​based on the actual positions of the feature points). The transformation relationship between the camera coordinate system and the machine coordinate system describes the fixed relationship between the camera and the machine's relative position and orientation. The relative pose of the lawnmower relative to the charging pile defines its spatial position and attitude. Position is described by a 3D distance (x, y, z) (corresponding to the distance between the three coordinate axes), and attitude is described by yaw, roll, and pitch (corresponding to rotational deviations in the three directions).

[0067] Specifically, the projection relationship between 2D and 3D coordinates can be established through the EPNP algorithm: that is, the coordinates of the corrected feature points and the 3D feature coordinates are substituted into the algorithm, combined with the camera intrinsic parameter matrix, to solve for the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system, and thus obtain the pose of the camera relative to the charging pile.

[0068] Finally, coordinate system transformation is performed: using the fixed transformation relationship between the camera and the machine, the rotation matrix and translation vector are inversely transformed (the rotation matrix is ​​transposed, and the translation vector is offset accordingly) to obtain the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system; then the rotation matrix is ​​transformed through ZYX Euler angles to obtain the three attitude angles yaw, roll, and pitch, and finally the complete relative pose of the machine relative to the charging pile (three-dimensional distance + three attitude angles) is determined, providing a precise control basis for the machine to automatically return to charging dock.

[0069] In the above-mentioned charging pile pose calculation method, coarse positioning is achieved through a charging pile target localization model. This model, trained with multi-scene data, can automatically identify charging piles and output target positioning boxes, effectively eliminating background (weeds, sky) interference and solving the problems of susceptibility to occlusion and lighting effects found in traditional methods. Feature point coordinates are corrected using camera intrinsic parameter matrices and distortion coefficients to eliminate errors caused by lens optical distortion. By combining three-dimensional feature coordinates to calculate relative pose, the pose estimation error can be ultimately controlled within 2°, and the detection blind zone reduced to 3cm, solving the problems of low accuracy and large blind zones found in traditional methods. The final output relative pose (three-dimensional distance + three attitude angles) can directly provide movement control commands for intelligent agricultural machinery (such as lawnmowers) (e.g., moving forward 0.5 meters, adjusting to the left by 2°), ensuring the machine quickly and accurately docks with the charging pile, achieving fully automatic recharging, and improving the intelligence level of the equipment and user experience.

[0070] In one embodiment, such as Figure 2 As shown, the method further includes:

[0071] S202, acquire charging pile images captured in various environmental scenarios, label the target bounding boxes of the charging piles contained in the charging pile images captured in the various environmental scenarios, and obtain training data.

[0072] The various environmental scenarios cover all possible real-world scenarios that intelligent agricultural machinery (such as intelligent lawnmowers) may encounter during automatic recharging, including weather conditions (sunny, cloudy, backlighting), occlusion (weeds, fallen leaves, no occlusion), and distance range (short, medium, long distance), ensuring comprehensive scenario coverage. Charging pile images: Original images containing charging piles, captured by the image acquisition module (such as a camera) on the intelligent agricultural machinery under the aforementioned environmental scenarios, serve as the foundational data source for model training. Target bounding box annotation: Rectangular bounding boxes are manually or semi-automatically drawn on the charging pile images to accurately enclose the charging piles in the images, while recording the key coordinate information of the bounding boxes to clarify the location of the charging piles in the images. Training data: The charging pile image dataset after target bounding box annotation includes the original images and corresponding annotation information (target bounding box coordinates), used to train the charging pile target localization model (such as YOLOv5n), enabling the model to learn the characteristics of charging piles in different scenarios and possess accurate recognition capabilities.

[0073] Specifically, the "multiple environmental scenarios" to be collected are determined, including different weather conditions (sunny days with ample light, cloudy days with dim light, and strong backlight interference), different occlusion conditions (lawns and weeds partially obscuring the charging pile, fallen leaves covering the surface of the charging pile, and open scenes without any obstructions), and different shooting distances (close-up, medium-range, and long-range shooting between the machine and the charging pile), to ensure that the collected images can simulate various situations in actual use. Using the image acquisition module (such as an industrial camera) on intelligent agricultural machinery, a certain number of images containing the charging pile are taken in each of the above-planned environmental scenarios to ensure sufficient data samples for each scenario and avoid model training biased towards a single scenario. Then, target bounding boxes are labeled for all the captured charging pile images: using professional data labeling tools (such as LabelImg), the labelers manually draw rectangles on each image to ensure that the rectangles accurately surround the main body of the charging pile (without omitting key features of the charging pile, and without including too much background area). At the same time, the tool automatically records the coordinate information of the rectangles and generates a labeling file (such as XML or TXT format) corresponding to the original image. Finally, the annotated original images are associated with their corresponding annotation files one by one to form complete training data, which provides a foundation for the subsequent training of the charging pile target localization model. This ensures that the model can learn the image features and location information of charging piles in different scenarios and has the ability to accurately identify charging piles and output target localization boxes in complex environments.

[0074] S204, perform augmentation processing on the training data, and divide the augmented training data into a training set, a validation set, and a test set.

[0075] Augmentation processing typically involves various transformations and optimizations of the original labeled training data (charging pile images and corresponding bounding box annotations). The aim is to expand data sample diversity, improve model generalization ability, and avoid overfitting. The training set, comprising the largest proportion, is the core training dataset for charging pile target localization models (such as YOLOv5n). Its purpose is to allow the model to learn the image features and location associations of charging piles in different scenarios. The validation set is used to monitor the model training process and adjust model parameters. It allows for real-time evaluation of model accuracy, avoiding overfitting or underfitting, and ensuring model generalization ability. The test set is an independent dataset used for the final evaluation of model performance. It has never participated in model training or parameter tuning and objectively reflects the model's recognition performance in real-world scenarios.

[0076] Specifically, augmentation processing is performed on the training data with completed target bounding box annotations. The operations revolve around "preserving the core features of charging piles and increasing scene diversity," and include three key processing methods:

[0077] Geometric deformation: Randomly flip the image horizontally and vertically, and simultaneously adjust the target box coordinates (e.g., after horizontal flipping, the target box x-coordinate is calculated in reverse according to the image width) to simulate different shooting angles of the machine;

[0078] Lighting adjustment: Randomly change the brightness (brighten or darken) and contrast of the image to simulate different lighting conditions such as sunny day, cloudy day, and backlight;

[0079] Noise addition: Gaussian noise is added to the image to simulate environmental interference during camera capture, improving the model's resistance to interference. All enhancement operations must ensure that the bounding box annotations are synchronized with image deformation to guarantee the accuracy of the annotation coordinates.

[0080] Next, the augmented training data (including images and corresponding labeled files) can be divided into training, validation, and test sets according to a preset 7:2:1 ratio (this is just an example; other ratios are also possible): Training set: 70% of the augmented data (referring to the augmented training data) is selected as the core samples for model training and parameter learning; Validation set: 20% of the augmented data is selected and used after each iteration during model training. The model performance is evaluated by calculating the target detection mAP (mean accuracy). Training is stopped when the accuracy does not improve for 10 consecutive iterations, and the optimal model weights are saved; Test set: 10% of the augmented data is selected and used for final performance testing after model training is completed. This verifies the model's recognition accuracy and real-time performance (e.g., inference frame rate ≥ 50fps, mAP ≥ 95%) in unseen samples, ensuring that the model meets the requirements of practical applications.

[0081] During the segmentation process, it is necessary to ensure that the data is evenly distributed, that is, each environmental scenario (such as occlusion, backlight, and different distances) is covered in all three datasets, avoiding the concentration of a single scenario in a certain dataset, and ensuring that the evaluation results are objective and reliable.

[0082] S206, Use the training set, validation set and test set to train the target detection model and obtain the charging pile target positioning model.

[0083] The object detection model can be the YOLOv5n lightweight deep learning model.

[0084] Specifically, YOLOv5n was selected as the basic object detection model, and a training environment was built based on the PyTorch deep learning framework to ensure that the hardware computing power met the model training requirements (e.g., supporting GPU acceleration to improve training efficiency). Next, model training configuration was performed: image data from the training set and corresponding bounding box annotation files were input into the model, and training hyperparameters (such as learning rate, iteration epochs, batch size, etc.) were set. Simultaneously, validation set data was configured as the evaluation basis for the training process, with the core evaluation metric being object detection mAP (mean accuracy), requiring a final mAP ≥ 95%. Then, iterative model training was initiated: the model continuously learned the association between image features of charging piles (such as circular and rectangular key structures) and bounding box coordinates through training set samples. After each round of training, the validation set was automatically used for accuracy evaluation; if there was no improvement in validation set mAP for 10 consecutive rounds, it indicated that the model had reached optimal performance, training was stopped, and the current optimal model weight file was saved. Finally, model testing and verification are conducted: the image data of the test set is input into the trained model to test the model's inference frame rate (≥50fps) and target detection accuracy, and to verify the model's recognition performance in new samples that have not been trained; if the test results meet the preset performance requirements, the model is the final usable charging pile target positioning model, which can be used for charging pile positioning in the automatic recharging scenario of intelligent agricultural machinery.

[0085] In this embodiment, images are collected under various environmental conditions, covering scenarios that may be encountered during actual charging, such as sunny / cloudy days, weed-covered / unobstructed conditions, and near / far distances. This exposes the model to diverse interference factors during the training phase. Data augmentation (flipping, lighting adjustment, and noise addition) further expands the sample diversity, preventing the model from becoming dependent on a single environment. The resulting charging pile target localization model effectively resists interference from weed obstruction, lighting changes, and environmental noise, solving the problem of traditional localization methods (such as LiDAR and basic vision) failing in complex lawn environments. Precise bounding box annotations provide a clear learning basis for the model, ensuring that the model can accurately capture the location features of charging piles and avoid positioning deviations caused by ambiguous annotations. The training set, validation set, and test set are divided in a 7:2:1 ratio. The training set is used for the model to learn core features, the validation set monitors the training effect in real time and stops overfitting training in time (stopping after 10 consecutive rounds of no improvement in accuracy), and the test set objectively evaluates the model's performance on unseen samples. With these three safeguards, a high accuracy of mAP ≥ 95% for object detection is finally achieved, providing a reliable localization foundation for subsequent feature extraction and pose calculation.

[0086] In one embodiment, the target positioning frame includes: a first shape category and a second shape category. For example... Figure 3As shown, the first shape category can be circular, and the second shape category can be rectangular. In addition, it should be noted that the shapes of various charging piles may differ. Therefore, the first shape category and the second shape category are not absolutely limited in some embodiments of this disclosure. The first shape category and the second shape category only need to have obvious features that can be identified and are not the same. Figure 3 The circular corners of the charging pile can be identified as the first shape category. The rectangle in the middle can be identified as the second shape category.

[0087] like Figure 4 As shown, the step of extracting feature points based on the target localization box to obtain feature point coordinates includes:

[0088] S302, based on the target positioning box, crop to obtain a local target box, perform edge detection on the local target box, and determine the target shape category to which the local target box belongs.

[0089] In this context, the local target bounding box typically refers to the image region cropped from the original image based on the target localization box. This region contains only the charging station itself and a small amount of background, with the aim of eliminating interference from a large amount of irrelevant background in the original image, making subsequent feature extraction more accurate and efficient.

[0090] Specifically, the coordinates of the target bounding box output by the model in the previous step are retrieved. Then, using the cropping function provided by an image processing library (such as OpenCV) or other cropping functions, the region defined by this rectangle is precisely cropped from the original image. The resulting image after cropping is the "local target box". This new image uses the charging pile as the main content, and the background information is greatly compressed, simplifying the data for subsequent processing. To reduce computation and noise interference, the cropped local target box image is usually converted into a grayscale image. Next, smoothing processes such as Gaussian filtering may be applied to further eliminate minor noise in the image to obtain clearer edges. Then, the Canny edge detection algorithm is used to extract edges from the processed image. The Canny algorithm can effectively find the true object edges in the image by setting high and low thresholds and generate a binary image containing only black and white edge pixels. Contour search is performed on the binary image obtained after edge detection (such as using OpenCV's findContours function) to find all continuous edge lines. Based on the found continuous lines, the target shape category to which the local target box belongs is determined. Typically, the Canny edge detection algorithm can detect rectangular or other non-circular shapes. If one edge is detected and its contour fit is approximately rectangular, the target shape can be classified as rectangular. The SimpleBlobDetector algorithm can also be used, setting a radius range for the circle to detect the center coordinates (x_c, y_c) of the circular charging port as feature points. For example, if the SimpleBlobDetector algorithm detects four feature points, these four feature points can be considered to be located at the four edges of the charging station. Figure 3 As shown, the identified target shape category can be determined to be circular. It should be understood that the above is merely an illustrative example. In practical applications, the rules for identifying the target shape category may differ depending on the shape or characteristics of the charging pile. Some embodiments of this disclosure do not impose restrictions on the rules for identifying the target shape category.

[0091] S304, in response to the local target box being a first shape category, the coordinates of multiple local feature points in the local target box are determined based on the shape feature information of the first shape category.

[0092] S306, in response to the target positioning box being a second shape category, the coordinates of multiple local feature points in the local target box are determined based on the shape feature information of the second shape category.

[0093] Shape feature information is typically used to define and identify the geometric properties of a specific shape. For a circle, its feature information mainly includes the center and radius. For a rectangle, its feature information mainly includes the coordinates of its four vertices.

[0094] Specifically, when the local target bounding box belongs to the first shape category, the coordinates of multiple local feature points within the local target bounding box can be determined based on the shape feature information of the first shape category. The same applies to the second shape category. For example, if the first shape category is a circle, a circle detection algorithm, such as the SimpleBlobDetector in OpenCV, is mainly used. The parameters of the blob detector are configured, such as filtering out excessively dark or bright areas, setting a circularity threshold (closer to 1 indicates a more rounded shape), and setting an area range, to accurately identify circular charging ports. The configured detector is applied to the edge-detected local target bounding box image. The algorithm automatically finds circular regions that meet the parameter conditions and calculates the center coordinates and radius of each circle. The detected center coordinates (relative to the local target bounding box coordinates) are output as the "local feature point coordinates" of that shape. In some embodiments of this disclosure, four feature points can also be selected as local feature point coordinates based on the circle's center and circular symmetry.

[0095] Responding to the second shape category being rectangle, the primary approach employed is contour detection and minimum bounding rectangle fitting. On the local target bounding box image after edge detection, a contour finding algorithm (such as OpenCV's findContours) is used to identify all continuous edge contours. Based on predefined conditions (such as area size, contour complexity, etc.), the contour most likely to be the rectangular shell of the charging pile is selected from the found contours. For the selected target contour, the minimum bounding rectangle is calculated using the minAreaRect function. This function returns the rectangle's center coordinates, width, height, and rotation angle. Based on the rectangle information returned by minAreaRect, the precise coordinates of the four vertices of the rectangle (relative to the local target bounding box) are calculated using trigonometric functions. These four vertex coordinates are output as the local feature point coordinates of the shape.

[0096] S308, determine the coordinate information of the target point in the local target within the target positioning frame, and perform original image coordinate mapping on the coordinates of the multiple local feature points based on the coordinate information to obtain the feature point coordinates.

[0097] Among them, the original image coordinate mapping is the operation of converting the local coordinates of feature points into the global coordinates of the original image. The core is to compensate for the offset by using the coordinate information of the target point to ensure the uniformity of the coordinate reference.

[0098] Specifically, this explanation will take the coordinates of the top-left vertex of the local target as an example.

[0099] The coordinates of the top-left vertex of the local target can be extracted from the target bounding box. These coordinates represent the reference position of the local target bounding box in the original image (the coordinates of the target point, X1, Y1), which is the key offset for subsequent mapping calculations. Next, the coordinates of multiple local feature points extracted in the previous steps are obtained. These coordinates are based on the local coordinates of the local target bounding box itself (such as the local coordinates of the center of a circular feature (u', v') and the local coordinates of the vertices of a rectangular feature (u', v')). They only represent the relative position of the feature points within the local target bounding box and are not associated with the global coordinate reference of the original image. Then, the original image coordinate mapping is performed: following the logic of "local coordinates + offset", the coordinates of each local feature point are added to the coordinates of the target point. The mapping formula is: u = u' + X1, v = v' + Y1 (where u and v are the global coordinates of the feature point, u' and v' are the local coordinates, and X1 and Y1 are the coordinates of the target point). Finally, the mapped feature point coordinates are output: Through the above calculations, all local feature points are converted into global pixel coordinates of the original image, ensuring that subsequent steps (such as distortion correction based on camera intrinsics and EPnP pose calculation) use a unified and accurate coordinate reference, avoiding calculation errors caused by coordinate reference confusion. The core of the entire process is to compensate for the offset between local and global coordinates by using the coordinate information of the target point, integrating the scattered local feature point coordinates into the unified coordinate system of the original image, providing a reliable coordinate foundation for subsequent high-precision pose detection.

[0100] In this embodiment, a local target bounding box is obtained through "target localization box clipping," directly excluding irrelevant backgrounds such as lawns, weeds, and sky from the original image. This allows edge detection and feature extraction to focus solely on the core area of ​​the charging pile, avoiding mis-extraction of features caused by background noise (such as misjudging weed edges as charging pile features). This solves the problem of low feature extraction accuracy caused by complex backgrounds in traditional vision methods. Feature extraction is performed using the features inherent to the charging pile. Typically, at long distances, the first shape category (circle) is identified, leveraging the high recognition rate of circular features at long distances to ensure wide coverage. At close distances, the second shape category (rectangle) is identified, adapting to the contour features after the field of view is focused. The two categories work together to cover the entire distance range. This design completely solves the dual blind spot problem of "blurred features at long distances and limited field of view at close distances" caused by traditional methods relying on a single feature.

[0101] In one embodiment, the camera intrinsic parameter matrix includes: focal length and principal point coordinates; the distortion coefficients include: radial distortion coefficients and tangential distortion coefficients; as shown below. Figure 5 As shown, the step of correcting the feature point coordinates based on a predetermined camera intrinsic parameter matrix and distortion coefficients to obtain corrected feature point coordinates includes:

[0102] S402, using the focal length and the principal point coordinates, the feature point coordinates are normalized to obtain normalized coordinates.

[0103] S404, Based on the radial distortion coefficient and the tangential distortion coefficient, the normalized coordinates are distorted to obtain the coordinates of the corrected feature points.

[0104] Specifically, according to the normalization formula, the pixel coordinates are converted into normalized coordinates in the camera coordinate system (format (x,y)). The core of the formula is to eliminate the influence of pixel scale and principal point offset: For example: If the pixel coordinates of a feature point are (u = 380, v = 260), the principal point coordinates are (c_x = 960, c_y = 540), and the focal length is (f_x = 1000, f_y = 1000), then the normalized coordinates are (x = (380-960) / 1000 = -0.58), (y = (260-540) / 1000 = -0.28).

[0105] Calculate the radial distortion factor r 2 The formula is r 2 =x 2 +y 2 (x and y are normalized coordinates), used to quantify the influence of radial distortion; substituting them into the distortion correction formula, the radial and tangential distortions in the x and y directions are corrected respectively, resulting in the corrected normalized coordinates (x′, y′):

[0106] x′=x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0107] y′=y(1+k1r 2 +k2r 4 +k3r 6 )+2p2xy+p1(r 2 +2y 2 )

[0108] The corrected normalized coordinates (x′, y′) are denormalized and converted back to pixel coordinate format to obtain the final corrected feature point coordinates (u′, v′), with the formula: u′=f x ·x′+c x v′=f y ·y′+c y

[0109] For example: Based on the above normalized coordinates (x = -0.58, y = -0.28), if the distortion coefficients k1 = -0.01, k2 = 0.001, k3 = 0, p1 = 0.0005, p2 = 0, where k1, k2, and k3 can be radial distortion coefficients, and p1 and p2 can be tangential distortion coefficients, then r can be calculated. 2 =(-0.58) 2 +(-0.28) 2 =0.4288. Substituting this into the correction formula and then inversely normalizing it, we can obtain the pixel coordinates after distortion removal.

[0110] In this embodiment, the camera lens exhibits radial distortion (such as the fisheye effect, where points at the image edges shift outwards or inwards) and tangential distortion (linear offset caused by lens mounting deviations) due to its optical characteristics. These distortions cause the positions of feature points on the image to deviate from their true physical locations, directly affecting the accuracy of pose calculation. By correcting based on radial and tangential distortion coefficients, these offsets can be reversed, allowing the feature point coordinates to return to the ideal imaging position without distortion, restoring the true geometry and relative position of the charging pile features, and solving the problem of "large pose calculation errors caused by image distortion".

[0111] In one embodiment, such as Figure 6 As shown, determining the relative pose with respect to the charging pile based on the coordinates of the corrected feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system includes:

[0112] S502, based on the coordinates of the corrected feature points and the pre-determined three-dimensional feature coordinates of the charging pile, determine the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system.

[0113] The rotation matrix (R) is typically a 3x3 orthogonal matrix used to describe the rotational attitude of the camera coordinate system relative to the charging pile coordinate system. It contains rotational information along three axes (roll, pitch, and yaw). The translation vector (t) is typically a 3x1 vector used to describe the positional offset of the camera coordinate system origin relative to the charging pile coordinate system origin. Its three components represent the distances along the X, Y, and Z axes, respectively.

[0114] Specifically, collect all the "corrected feature point coordinates" obtained in the previous step to form an Nx2 matrix, where N is the number of feature points. From the predefined 3D model of the charging pile, find the 3D coordinates of the feature points that correspond one-to-one with the above 2D image points, forming an Nx3 matrix. Key point: The points in these two sets must correspond strictly one-to-one. You can use the PnP solver function (e.g., cv2.solvePnP) provided by a computer vision library (such as OpenCV). Pass the Nx2 matrix, the Nx3 matrix, and the camera intrinsic matrix as input parameters to this function. The algorithm will calculate the rotation matrix (R) and translation vector (t) that can most accurately project the 3D world points onto the corresponding 2D image points based on the principle of perspective projection. For example, the EPnP algorithm will automatically generate 3 virtual control points, C1, C2, and C3 (these 3D coordinates are in the charging pile coordinate system), such that the 4 feature points P... wi All of these can be represented as a linear combination of these three control points:

[0115] P wi =α i1 C1+α i2 C2+α i3 C3(i=1,2,3,4)

[0116] Where α i1 ,α i2 ,α i3 The weighting coefficients (satisfying α) i1 +α i2 +α i3 =1, ensuring the rationality of linear combination), calculated from the coordinates of the feature points. Calculate the 3D coordinates of the control points in the camera coordinate system. Let the 3D coordinates of the control points in the camera coordinate system be C′1, C′2, C′3 (unknown, needs to be solved). According to the camera projection model, each feature point P... wi Coordinates P′ in the camera coordinate system wi (i.e. R·P) wi +t) can also be expressed as a linear combination of control points C′1, C′2, C′3:

[0117] P′ wi =α i1 C′1+α i2 C′2+α i3 C′3

[0118] At the same time, P′ wi Projecting onto an image is p iEquations can be established by combining the camera intrinsic parameter K. By simultaneously solving the equations for the four feature points, the coordinates of C′1, C′2, and C′3 can be obtained (this step is the core of EPnP, simplifying the calculation through linear algebra and avoiding complex iterations). Given the control points C1, C2, and C3 in the charging pile coordinate system and the control points C′1, C′2, and C′3 in the camera coordinate system, the relationship between them satisfies:

[0119] C′ j =R·C j +t (j=1,2,3)

[0120] This is a problem of "given the correspondence between two sets of 3D points, find R and t", which can be solved using Singular Value Decomposition (SVD):

[0121] Calculate the centroid (center point) of the two sets of control points:

[0122] After removing the influence of the centroid, we obtain decentralized coordinates:

[0123] Constructing a matrix Perform SVD decomposition on H: H = UΣV T ;

[0124] Rotation matrix R = VU T (If the determinant is negative, it needs to be corrected to ensure the right-handed coordinate system);

[0125] Translation vector

[0126] S504, based on the transformation relationship between the camera coordinate system and the machine coordinate system, and the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system, determine the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system.

[0127] The transformation relationship between the camera coordinate system and the machine coordinate system can be predetermined through calibration, including the rotation matrix R. cm (Camera rotation angle relative to the machine) and translation vector t cm (Camera position offset relative to the machine center), used to associate the spatial attitude of the camera and the machine, remains unchanged throughout. The rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system: pose parameters (denoted as R) obtained through the EPnP algorithm. cw t cw ),R cw Describe the camera's orientation relative to the charging station, t cw Describe the 3D position of the camera relative to the charging station. The rotation matrix and translation vector of the machine coordinate system relative to the charging station coordinate system: the final target parameters (denoted as R).mw t mw R mw Describes the machine's attitude relative to the charging station, t mw Describing the three-dimensional distance between the machine and the charging station is the core basis for the machine's automatic recharging control.

[0128] Specifically, the coordinate system rotation relationship satisfies the transmission logic of "machine → camera → charging station", therefore R mw Requires R cm inverse transform and R cw Combined calculations; due to R cm It is an orthogonal matrix, and its inverse is equal to its transpose. The final formula is: The calculation is performed by multiplying two 3×3 rotation matrices to obtain the rotation matrix of the machine relative to the charging station.

[0129] Solve for the translation vector t of the machine relative to the charging station. mw The translation vector needs to consider the superposition of the "camera offset relative to the machine" and the "camera offset relative to the charging station," and it needs to be... Correcting rotational effects;

[0130] The final formula is: First calculate t cw With t cm The difference (to eliminate the influence of camera mounting position offset), then compared with... Multiply (to correct for directional deviations caused by rotation) to obtain a 3×1 translation vector of the machine relative to the charging station.

[0131] S506, Based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system, determine the relative pose with the charging pile.

[0132] Specifically, abstract matrix and vector parameters can be converted into intuitive "distance + angle" information that machines can directly use, thereby determining relative pose.

[0133] In this embodiment, by combining the input of "corrected feature point coordinates (precise 2D image coordinates) + charging pile 3D feature coordinates (known 3D world coordinates)," the EPnP algorithm is used to directly establish the relationship between the image and the real world, calculating the rotation matrix and translation vector of the camera relative to the charging pile. This step completely solves the problem of "spatial pose cannot be obtained from images alone." Through 2D-3D perspective projection constraints, the accuracy of the pose relationship calculation between the camera and the charging pile is ensured (e.g., rotation angle error ≤ 1°), providing a precise intermediate bridge for subsequent machine pose derivation. Since the installation positions of the camera and the machine have a fixed offset (e.g., the camera is mounted on top of the machine, and the machine center is the control reference), directly using the camera pose as the machine pose would lead to huge errors. By introducing the "camera-machine coordinate system transformation relationship" (pre-calibrated rotation matrix and translation vector), the camera pose relative to the charging pile is transformed to obtain the machine pose parameters relative to the charging pile. This design eliminates the systematic errors caused by "camera and machine installation deviations" and solves the problem of "coordinate system datum confusion leading to pose transfer distortion" in traditional methods, ensuring the accurate transfer of pose parameters from the camera to the machine.

[0134] In one embodiment, such as Figure 7 As shown, the relative pose includes: three-dimensional distance and angle; determining the relative pose with respect to the charging pile based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system includes:

[0135] S602, Based on the translation vector, determine the three-dimensional distance to the charging pile.

[0136] S604, Perform Euler angle transformation on the rotation matrix to determine the angle with the charging pile.

[0137] Specifically, t mw =[x,y,z] T The three components correspond to the three-dimensional distance between the machine and the charging station:

[0138] x: The distance of the machine in the x-axis direction of the charging pile coordinate system (e.g., x = 0.7 meters, which means that the machine is 0.7 meters in front of the charging pile's return charging interface);

[0139] y: The distance of the machine in the y-axis direction of the charging pile coordinate system (e.g., y = 0.2 meters, which means the machine is 0.2 meters to the right of the charging pile); this position information is directly used for the translational movement control of the machine (e.g., "move forward 0.7 meters" or "adjust to the left 0.2 meters").

[0140] z: The distance of the machine in the z-axis direction of the charging pile coordinate system (e.g., z = -0.1 meters, which means that the machine is 0.1 meters lower than the charging pile's return charging interface).

[0141] Extracting attitude information (based on rotation matrix R) mw ): Using the ZYX Euler angle transformation formula, the abstract R... mw Converted to three intuitive attitude angles, the formula is as follows (r ij For R mw (element in row i, column j):

[0142]

[0143] Roll: The tilt angle of the machine around its own x-axis (e.g., roll = -2°, meaning the left side of the machine is lower and the right side is higher); Pitch: The tilt angle of the machine around its own y-axis (e.g., pitch = 1°, meaning the front of the machine is higher and the rear is lower); Yaw: The yaw angle of the machine around its own z-axis (e.g., yaw = 3°, meaning the machine deviates 3° from the charging station's orientation to the right). This attitude information is directly used for the machine's rotation adjustment control (e.g., "adjust yaw angle to the left by 3°", "correct roll angle by 2°").

[0144] The method further includes: performing regression charging based on the three-dimensional distance to the charging pile and the angle to the charging pile.

[0145] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0146] Based on the same inventive concept, this disclosure also provides a charging pile pose calculation device for implementing the charging pile pose calculation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the charging pile pose calculation device provided below can be found in the limitations of the charging pile pose calculation method described above, and will not be repeated here.

[0147] In one embodiment, such as Figure 8As shown, a charging pile pose calculation device 700 is provided, including: a model processing module 702, a feature point extraction module 704, a feature point correction module 706, and a pose determination module 708, wherein:

[0148] The model processing module 702 is used to acquire charging pile images, input the charging pile images into a pre-trained charging pile target localization model, and output target localization boxes through the charging pile target localization model.

[0149] The feature point extraction module 704 is used to extract features and map the original image coordinates based on the target positioning box to obtain the feature point coordinates;

[0150] The feature point correction module 706 is used to correct the coordinates of the feature points based on a predetermined camera intrinsic parameter matrix and distortion coefficients to obtain corrected feature point coordinates.

[0151] The pose determination module 708 is used to determine the relative pose with respect to the charging pile based on the coordinates of the correction feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system.

[0152] In one embodiment of the device, the device further includes:

[0153] The training data acquisition module is used to acquire images of charging piles taken in various environmental scenarios, and to annotate the target boxes of the charging piles contained in the images of charging piles taken in various environmental scenarios to obtain training data.

[0154] The training data partitioning module is used to perform augmentation processing on the training data and divide the augmented training data into a training set, a validation set, and a test set.

[0155] The model training module is used to train the target detection model using the training set, validation set, and test set to obtain the charging pile target localization model.

[0156] In one embodiment of the device, the target positioning bounding box includes: a first shape category and a second shape category; the feature point extraction module 704 includes:

[0157] The target shape category determination module is used to crop based on the target positioning box to obtain a local target box, perform edge detection on the local target box, and determine the target shape category to which the local target box belongs;

[0158] The local feature point coordinate determination module is used to determine the coordinates of multiple local feature points in the local target box based on the shape feature information of the first shape category when the local target box is of a first shape category; and to determine the coordinates of multiple local feature points in the local target box based on the shape feature information of the second shape category when the target positioning box is of a second shape category.

[0159] The coordinate mapping module is used to determine the coordinate information of the target point in the target positioning frame, and to perform original image coordinate mapping on the coordinates of the multiple local feature points based on the coordinate information to obtain the feature point coordinates.

[0160] In one embodiment of the device, the camera intrinsic parameter matrix includes: focal length and principal point coordinates; the distortion coefficients include: radial distortion coefficients and tangential distortion coefficients. The feature point correction module 706 includes:

[0161] The normalization module is used to normalize the coordinates of the feature points using the focal length and the principal point coordinates to obtain normalized coordinates.

[0162] The correction module is used to perform distortion correction on the normalized coordinates based on the radial distortion coefficient and the tangential distortion coefficient to obtain the coordinates of the corrected feature points.

[0163] In one embodiment of the device, the pose determination module 708 is further configured to: determine the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system based on the coordinates of the corrected feature points and the pre-determined three-dimensional feature coordinates of the charging pile; determine the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system based on the transformation relationship between the camera coordinate system and the machine coordinate system and the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system; and determine the relative pose with respect to the charging pile based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system.

[0164] In one embodiment of the device, the relative pose includes: a three-dimensional distance and an angle; the pose determination module 708 is further configured to determine the three-dimensional distance to the charging pile based on the translation vector; and to perform Euler angle transformation on the rotation matrix to determine the angle to the charging pile. The device also includes a regression charging module, configured to perform regression charging based on the three-dimensional distance to the charging pile and the angle to the charging pile.

[0165] Each module in the aforementioned charging pile pose calculation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0166] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a charging pile pose calculation method.

[0167] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0169] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0170] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0171] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this disclosure may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.

Claims

1. A method for calculating the pose of a charging pile, characterized in that, The method includes: Acquire an image of a charging pile, input the image into a pre-trained charging pile target localization model, and output a target localization box through the charging pile target localization model; Feature extraction is performed based on the target localization box to obtain the coordinates of the feature points; Based on a predetermined camera intrinsic parameter matrix and distortion coefficients, the coordinates of the feature points are corrected to obtain the corrected feature point coordinates. Based on the coordinates of the corrected feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system, the relative pose with the charging pile is determined.

2. The method according to claim 1, characterized in that, The method further includes: Acquire charging pile images captured in various environmental scenarios, annotate the target bounding boxes of the charging piles contained in the charging pile images captured in the various environmental scenarios, and obtain training data; The training data is augmented, and the augmented training data is divided into a training set, a validation set, and a test set. The target detection model is trained using the training set, validation set, and test set to obtain the charging pile target localization model.

3. The method according to claim 1, characterized in that, The target location bounding box includes: a first shape category and a second shape category; the feature extraction based on the target location bounding box to obtain feature point coordinates includes: Based on the target positioning bounding box, a local target bounding box is obtained by cropping. Edge detection is performed on the local target bounding box to determine the target shape category to which the local target bounding box belongs. In response to the local target box being of a first shape category, the coordinates of multiple local feature points within the local target box are determined based on the shape feature information of the first shape category; In response to the target positioning box being of the second shape category, the coordinates of multiple local feature points in the local target box are determined based on the shape feature information of the second shape category; The coordinate information of the target point in the local target within the target positioning frame is determined, and the coordinate information is used to perform original image coordinate mapping on the coordinates of the multiple local feature points to obtain the feature point coordinates.

4. The method according to claim 1, characterized in that, The camera intrinsic parameter matrix includes: focal length and principal point coordinates; the distortion coefficients include: radial distortion coefficients and tangential distortion coefficients; the step of correcting the feature point coordinates based on the predetermined camera intrinsic parameter matrix and distortion coefficients to obtain corrected feature point coordinates includes: Using the focal length and the principal point coordinates, the feature point coordinates are normalized to obtain normalized coordinates; Based on the radial distortion coefficient and the tangential distortion coefficient, the normalized coordinates are distorted to obtain the coordinates of the corrected feature points.

5. The method according to claim 1, characterized in that, The step of determining the relative pose with respect to the charging pile based on the coordinates of the corrected feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system includes: Based on the coordinates of the corrected feature points and the pre-determined three-dimensional feature coordinates of the charging pile, the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system are determined. Based on the transformation relationship between the camera coordinate system and the machine coordinate system, and the rotation matrix and translation vector of the camera coordinate system relative to the charging pile coordinate system, the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system are determined. The relative pose with respect to the charging pile is determined based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system.

6. The method according to claim 5, characterized in that, The relative pose includes: three-dimensional distance and angle; determining the relative pose with respect to the charging pile based on the rotation matrix and translation vector of the machine coordinate system relative to the charging pile coordinate system includes: Based on the translation vector, the three-dimensional distance to the charging pile is determined; The rotation matrix is ​​transformed using Euler angles to determine the angle with the charging pile. The method further includes: performing regression charging based on the three-dimensional distance to the charging pile and the angle to the charging pile.

7. A charging pile pose calculation device, characterized in that, The device includes: The model processing module is used to acquire charging pile images, input the charging pile images into a pre-trained charging pile target localization model, and output target localization boxes through the charging pile target localization model. The feature point extraction module is used to extract features and map the original image coordinates based on the target localization box to obtain the feature point coordinates; The feature point correction module is used to correct the coordinates of the feature points based on a predetermined camera intrinsic parameter matrix and distortion coefficients, so as to obtain the corrected feature point coordinates. The pose determination module is used to determine the relative pose with respect to the charging pile based on the coordinates of the correction feature points, the pre-determined three-dimensional feature coordinates of the charging pile, and the transformation relationship between the camera coordinate system and the machine coordinate system.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images