Pose obtaining method and device, charging equipment and storage medium

Abstract

The invention relates to the technical field of charging, in particular to a pose obtaining method and device, charging equipment and a storage medium. The method comprises the steps of performing enhancement processing on a charging port image based on an illumination scene when the charging port image is acquired to obtain a target image; cutting out M sub-images in one-to-one correspondence with M terminals included in the charging port from the target image; the sub-image comprises a terminal image of the corresponding terminal; processing the M sub-images by adopting a semantic segmentation model to obtain corresponding M mask images; based on the M mask images, determining coordinates of the M terminals in the corresponding M mask images; based on the coordinates of the M terminals in the corresponding M mask images, obtaining the coordinates of the M terminals in the target image; and determining the pose of the charging port based on the coordinates of the M terminals in the target image and the three-dimensional model of the charging port.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of charging, and particularly relates to a method and device for obtaining a pose, a charging device and a storage medium. BACKGROUND

[0002] Under the background of the rapid development of intelligent transportation and new energy vehicles, the automation and intelligence of charging devices have become a key research direction. Charging devices need to have high-precision positioning capability for electric vehicle charging ports to realize automatic docking and efficient charging. Visual positioning technology is widely used due to its non-contact and strong real-time performance, and involves multiple key technical links such as image processing, target detection and pose estimation. SUMMARY

[0003] Embodiments of the present application provide a method and device for obtaining a pose, a charging device and a storage medium, and the following content is specifically disclosed: The embodiment of the present application provides a method for obtaining a pose, which comprises: performing enhancement processing on a charging port image based on an illumination scene when the charging port image is collected to obtain a target image; the charging port image is an image collected by a camera on a charging port; M sub-images corresponding to M terminals included in the charging port are cropped from the target image; wherein M is a positive integer; a sub-image includes a terminal image corresponding to a terminal; a semantic segmentation model is used to process the M sub-images to obtain M mask images corresponding to the M sub-images; based on the M mask images, coordinates of the M terminals in the M mask images are determined; based on the coordinates of the M terminals in the M mask images, coordinates of the M terminals in the target image are obtained; and based on the coordinates of the M terminals in the target image and a three-dimensional model of the charging port, a pose of the charging port is determined.

[0004] Based on the aforementioned technical methods, firstly, the charging port image is enhanced according to the lighting scene during image acquisition to obtain the target image, thereby improving image quality and enhancing the accuracy of subsequent processing. Secondly, M sub-images corresponding one-to-one with the M terminals included in the charging port are cropped from the target image, facilitating refined processing for each terminal. Then, a semantic segmentation model is used to process the M sub-images to obtain corresponding M mask images, thereby achieving high-precision segmentation of the terminals. Further, the coordinates of the terminals in their respective mask images are determined based on the M mask images, and mapped onto the target image in combination with bounding box information. Finally, based on the coordinates in the target image and the 3D model of the charging port, the PnP algorithm is used to accurately estimate the charging port pose. Through the "cropping-segmentation-aggregation" process, the complex overall segmentation task is decomposed into refined processing of individual terminals, thereby significantly improving the accuracy and robustness of charging port pose estimation. This not only effectively overcomes the problems of false detection and missed detection, but also generates more accurate terminal contour coordinates through sub-image-level semantic segmentation, providing high-quality input for subsequent pose calculation. This application enables stable and reliable pose estimation in complex lighting scenarios. In some embodiments, the method further includes: processing the charging port image into a first grayscale image; determining the global brightness and brightness variance of the first grayscale image based on the first grayscale image; wherein the global brightness is used to characterize the exposure level of the charging port image; the brightness variance is used to characterize the richness of texture in the charging port image; and determining the lighting scene when the charging port image was acquired based on the global brightness and brightness variance.

[0005] Based on the aforementioned technical methods, by converting the charging port image to grayscale and extracting its global brightness and brightness variance as quantitative indicators, accurate classification of the lighting conditions of the acquired image can be achieved, thus providing a basis for subsequent adaptive lighting enhancement processing. Selecting appropriate image enhancement methods according to different lighting conditions ensures the adaptability and generalization ability of the image processing workflow.

[0006] In some embodiments, the lighting scene is one of a low-light scene, a normal scene, a strong-light scene, and a low-contrast normal-light scene. Based on the lighting scene when the charging port image is acquired, the charging port image is enhanced to obtain a target image, including: when the lighting scene is a low-light scene, processing the charging port image using multi-frame image fusion technology to obtain the target image; when the lighting scene is a strong-light scene, processing the charging port image using specular highlight suppression technology to obtain the target image; and when the lighting scene is a low-contrast normal-light scene, processing the charging port image using texture enhancement technology to obtain the target image.

[0007] Based on the aforementioned technical methods, by employing targeted image enhancement techniques such as multi-frame fusion, specular suppression, and texture enhancement under different lighting scenarios, the image quality degradation caused by lighting variations can be effectively addressed. For example, in low-light scenarios, multi-frame fusion helps reduce noise while preserving key features; in high-light scenarios, specular suppression prevents reflective areas from interfering with segmentation results; and in low-contrast scenarios, texture enhancement improves the visibility of edge markers. Thus, the organic combination of lighting perception and enhancement strategies significantly improves image processing performance, thereby ensuring the stability and accuracy of subsequent localization and segmentation tasks.

[0008] In some embodiments, cropping M sub-images corresponding one-to-one with the M terminals included in the charging port from the target image includes: determining a target detection model corresponding to the lighting scene based on the lighting scene when the charging port image is acquired; processing the target image based on the target detection model corresponding to the lighting scene to obtain M bounding box information; and cropping M sub-images from the target image based on the M bounding box information; wherein, the target detection model corresponding to the lighting scene is a model trained on images acquired under the lighting scene and has model parameters and / or model structure corresponding to the lighting scene.

[0009] Based on the aforementioned technical methods, by using specially trained detection models under different lighting conditions, it is possible to more accurately identify and locate each terminal inside the charging port, thereby generating more precise bounding box information. This lighting-adaptive model design enables charging devices to maintain high detection performance under complex lighting conditions, reducing the occurrence of missed detections and false detections. Furthermore, since each model is optimized for specific lighting conditions, it exhibits stronger adaptability and robustness in practical applications.

[0010] In some embodiments, a semantic segmentation model is used to process M sub-images to obtain corresponding M mask images, including: determining a semantic segmentation model corresponding to the lighting scene based on the lighting scene when the charging port image is acquired; and processing the M sub-images based on the semantic segmentation model corresponding to the lighting scene to obtain corresponding M mask images; wherein, the semantic segmentation model corresponding to the lighting scene is a model trained on images acquired under the lighting scene and has model parameters and / or model structure corresponding to the lighting scene.

[0011] Based on the aforementioned techniques, the segmentation accuracy of the charging port terminals can be effectively improved by using a specially trained semantic segmentation model under different lighting conditions. This lighting-adaptive model design enables the charging device to stably output high-quality mask images under various lighting conditions, thus providing a reliable data foundation for subsequent coordinate calculations and pose estimation. Furthermore, this method can dynamically adjust model parameters according to different lighting conditions to adapt to the image characteristics of different scenarios.

[0012] In some embodiments, determining the coordinates of M terminals in the corresponding M mask images based on M mask images includes: for each mask image, using an edge detection algorithm to extract a set of edge contour points of the terminal image; determining the integrity level of the terminal image based on the set of edge contour points; the integrity level is used to indicate the contour integrity of the terminal image; determining the search radius and minimum number of points for a clustering algorithm based on the integrity level; and executing a clustering algorithm based on the search radius and minimum number of points to extract the coordinates of the terminals in the corresponding mask images from the set of edge contour points.

[0013] Based on the aforementioned technical means, by introducing edge detection and contour integrity assessment mechanisms, the contours of terminal images can be quantitatively analyzed, and the parameter settings of the clustering algorithm can be dynamically adjusted accordingly, thereby improving the accuracy and robustness of center point estimation. Specifically, the classification of integrity levels enables the charging device to select the most suitable clustering strategy based on different contour states, avoiding estimation bias caused by contour damage, thus providing more accurate basic data for subsequent pose calculations.

[0014] In some embodiments, determining the integrity level of a sub-image based on a set of edge contour points includes: determining the notch rate and burr density of a terminal image based on the set of edge contour points; wherein the notch rate is used to indicate the degree of interruption of the continuity of the terminal contour; the burr density is used to indicate the degree of noise interference to the terminal contour; and determining the integrity level of the sub-image based on the notch rate and burr density.

[0015] Based on the aforementioned technical methods, by defining two quantitative indicators—notch rate and burr density—the contour integrity of a terminal image can be accurately assessed, thereby determining whether it is in a complete, moderate, or incomplete state. This integrity assessment method based on quantitative indicators not only objectively reflects the quality status of the terminal image but also provides a clear basis for parameter adjustment in subsequent clustering algorithms, thus improving the stability and reliability of center point estimation.

[0016] In some embodiments, obtaining the coordinates of the M terminals in the target image based on the coordinates of the M terminals in the corresponding M mask images includes: determining the position of the sub-image in the target image based on the M bounding box information; and determining the coordinates of the M terminals in the target image based on the position of the sub-image in the target image and the coordinates of the M terminals in the corresponding M mask images.

[0017] Based on the aforementioned techniques, by combining bounding box information with the terminal coordinates in the mask image, a precise mapping of the terminal position to the target image can be achieved. This mechanism, based on accurate coordinate transformation, ensures the consistency of the positional information of each terminal throughout the entire image space, thereby providing reliable data support for subsequent 3D reconstruction and pose estimation.

[0018] In some embodiments, the pose of the charging port is determined based on the coordinates of the M terminals in the target image and the 3D model of the charging port, including: constructing a 2D-3D point correspondence set based on the global coordinates of the M terminals in the target image and their predefined 3D coordinates in the 3D model of the charging port; and determining the pose of the charging port using the PnP algorithm based on the 2D-3D point correspondence set. Based on the above technical means, by constructing the mapping relationship between the terminal coordinates in the target image and the terminal coordinates in the three-dimensional model, and combining it with the PnP algorithm, the six-degree-of-freedom pose of the charging port can be solved efficiently and accurately.

[0019] In some embodiments, the method may further include: adding target annotation information to a historical charging port image, the target annotation information being used to define the relative positional relationship between the M terminals included in the charging port; and performing enhancement processing on the historical charging port image based on the target annotation information, the enhanced historical charging port image being used to train a semantic segmentation model and / or an object detection model.

[0020] Based on the aforementioned technical means, by adding target annotation information that defines the relative positional relationship between terminals to historical charging port images, and performing data augmentation based on this structured information, the training effect of semantic segmentation models and object detection models can be effectively improved.

[0021] This application provides a device for obtaining pose, comprising: a first processing unit, configured to enhance the charging port image based on the lighting scene when the charging port image is acquired, to obtain a target image; a second processing unit, configured to crop out M sub-images from the target image that correspond one-to-one with the M terminals included in the charging port; a third processing unit, configured to process the M sub-images using a semantic segmentation model to obtain corresponding M mask images; a first determining unit, configured to determine the coordinates of the M terminals in the corresponding M mask images based on the M mask images; a second determining unit, configured to obtain the coordinates of the M terminals in the target image based on the coordinates of the M terminals in the corresponding M mask images; and a third determining unit, configured to determine the pose of the charging port based on the coordinates of the M terminals in the target image and a three-dimensional model of the charging port.

[0022] This application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer's processor, causes the computer to perform the pose acquisition method as described in the first aspect and any one thereof. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a method for obtaining a pose according to an embodiment of this application; Figure 2 This is a schematic diagram of a charging port structure provided in an embodiment of this application; Figure 3 This is a flowchart illustrating another method for obtaining pose provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a device for obtaining pose according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a charging device provided in an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0025] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0026] Against the backdrop of the rapid development of intelligent transportation and new energy vehicles, the automation and intelligence of charging equipment have become key research directions. Charging equipment needs to have high-precision positioning capabilities for electric vehicle charging ports to achieve automatic docking and efficient charging. Visual positioning technology is widely used due to its advantages such as non-contact operation and strong real-time performance, involving several key technical aspects such as image processing, target detection, and pose estimation.

[0027] In this context, charging port positioning refers to the need for high-precision pose estimation of the charging port to enable high-precision automatic charging device insertion. The pose of the charging port can be represented using 6 degrees of freedom. For example, the pose can include coordinates and orientation, where coordinates can be represented using coordinates such as (x, y, z) and orientation can be represented using Euler angles such as (α, β, γ).

[0028] In existing technologies, the charging port image is typically captured by a camera, and its pose is determined by image recognition algorithms. However, such methods generally suffer from low reliability, specifically poor recognition accuracy under complex lighting conditions. Furthermore, existing recognition methods often rely on high-cost hardware (such as high-resolution cameras and multi-view vision systems), resulting in high costs and hindering their widespread adoption in home charging scenarios.

[0029] To address this, this application provides a method for obtaining the pose. First, based on the lighting scene when the charging port image is acquired, the charging port image is enhanced to adapt to the image quality under different environmental conditions, thereby improving the subsequent recognition accuracy. Second, by cropping out the sub-images corresponding to each terminal and combining them with a semantic segmentation model, the boundary information of each terminal can be extracted more accurately. Furthermore, the local coordinates of the terminal are determined based on the mask image and mapped back to the target image to obtain the global coordinates of the terminal, making the positioning result more stable and reliable. Finally, by combining the three-dimensional model of the charging port with the PnP algorithm, a high-precision estimation of the 6D pose of the charging port is achieved.

[0030] This application significantly improves recognition accuracy and stability in complex environments through adaptive enhancement and refined sub-image segmentation. It has low hardware requirements, requiring only a monocular camera to achieve high-precision pose estimation, greatly reducing costs and making it suitable for cost-sensitive scenarios such as home use. Furthermore, this application optimizes the entire process from image preprocessing and feature extraction to pose solving, demonstrating excellent robustness.

[0031] This application provides a method for obtaining pose, applied to a charging device. The charging device identifies the coordinates of a charging port and automatically connects to it. The charging device includes a camera and a connector. The camera captures images of the charging port; these images are images captured by the camera of the charging port. The connector, guided by the charging device, performs physical movement and docking operations, ultimately establishing a stable electrical connection with the charging port for charging.

[0032] In this embodiment of the application, the charging device may be, for example, a charging robotic arm, a charging robot, a mobile charging pile, etc.

[0033] In this application embodiment, the charging port can be, for example, an AC charging port or DC charging port for a vehicle, a charging port for an electric ship, a charging port for an electric aircraft, a charging port for a terminal device, or a charging port for an electrical appliance, etc., and this application does not make specific limitations in this regard. Among them, vehicles include, but are not limited to, electric cars, electric motorcycles, electric forklifts, electric buses, etc.

[0034] The following description, taking the AC charging port of a vehicle as an example, uses specific embodiments and accompanying drawings to illustrate a method, apparatus, charging device, and storage medium for obtaining pose provided by this application. Figure 1 This is a flowchart illustrating a method for obtaining a pose according to an embodiment of this application, as shown below. Figure 1 As shown, the method may include: S101, based on the lighting scene when the charging port image is acquired, enhance the charging port image to obtain the target image.

[0035] The lighting scene refers to the ambient lighting conditions when acquiring the charging port image. Different lighting scenes affect the quality of the charging port image, thus affecting the subsequent pose recognition performance. For example, the lighting scene can be one of the following: weak light scene, normal light scene, strong light scene, or low-contrast normal light scene. Weak light refers to an ambient light intensity less than a first preset value when acquiring the charging port image; strong light refers to an ambient light intensity greater than a second preset value; normal light scene refers to an ambient light intensity between the first and second preset values ​​when acquiring the charging port image. A low-contrast normal light scene refers to an ambient light intensity between the first and second preset values ​​when acquiring the charging port image, and where the image contrast is below a threshold. The first preset value is less than the second preset value.

[0036] For example, charging port images captured in underground parking garages or at night are in low-light conditions, resulting in low overall brightness, significant noise, and poor detail. Charging port images captured outdoors on cloudy or overcast days are in low-contrast normal-light conditions; while the overall brightness is moderate, edges and contours show little difference from the background, leading to insufficient contrast. Charging port images captured outdoors in direct sunlight are in strong-light conditions, exhibiting overexposure, reflections, or highlights, resulting in abnormal saturation or loss of detail.

[0037] In some embodiments, before executing S101, the charging device can acquire images of the vehicle using a camera to obtain a vehicle image. The charging device can then crop the charging port image from the vehicle image based on an object detection algorithm. Existing technologies can be referenced, and will not be elaborated further.

[0038] In some embodiments, before executing S101, the charging device may first determine the lighting scene when acquiring the charging port image. For example, the charging device determines the lighting scene based on the charging port image. Specifically, the charging device processes the charging port image into a grayscale image, determines the global brightness and brightness variance of the grayscale image, and determines the light intensity based on the global brightness and brightness variance of the grayscale image.

[0039] The charging port image can be a color image, such as a red-green-blue (RGB) image. The grayscale image is converted from the charging port image and is a single-channel image containing only brightness information. It is typically represented by an 8-bit pixel depth, with each pixel's value being its brightness value. In some embodiments, the charging device can use a conversion formula such as brightness value = 0.299 × R + 0.587 × G + 0.114 × B to convert the charging port image into a grayscale image. Here, R is the value of the red channel of the pixel; G is the value of the green channel; and B is the value of the blue channel. It should be understood that the grayscale image and the charging port image have the same dimensions. For example, if the charging port image has dimensions W × H, the grayscale image also has dimensions W × H. Here, W is the width of the image, and H is the height. In some embodiments, the charging port image can be preprocessed before being converted to grayscale. Preprocessing includes, but is not limited to, distortion correction, noise reduction, and image straightening.

[0040] The global brightness of a grayscale image is used to characterize the exposure level of the charging port image. For example, global brightness is the average brightness value of all pixels in the grayscale image, reflecting the overall brightness of the charging port image. For instance, a charging port image taken in low-light conditions corresponds to a lower global brightness, while a charging port image taken in bright light conditions corresponds to a higher global brightness. Global brightness can be used to determine whether a charging port image is overexposed or underexposed, and is one of the important parameters in illumination assessment.

[0041] For example, global brightness can be calculated using formula (1).

[0042] , formula (1).

[0043] in, W is the global brightness, H is the width of the grayscale image, and H is the height of the grayscale image. It is the grayscale value of pixel (x, y). (x, y) are the coordinates of the pixel in the grayscale image.

[0044] The luminance variance of a grayscale image is used to characterize the richness of texture in a charging port image. For example, luminance variance measures the degree of brightness variation between different regions in a charging port image. A higher luminance variance indicates more texture detail in the charging port image, while a lower luminance variance indicates a smoother or less detailed image. In some embodiments, luminance variance may also be referred to as local luminance variance.

[0045] For example, the sliding window method can be used to calculate the luminance variance based on formula (2).

[0046] , formula (2).

[0047] in, It represents the brightness variance, where N is the number of sliding windows, and Var(window) is the brightness variance. i) Let be the variance of the brightness values ​​of the pixels in the i-th window. The window can be, for example, an 8-pixel × 8-pixel rectangular window. In this embodiment, the charging device traverses the grayscale image by sliding on the grayscale image, obtains the variance of the brightness values ​​of the pixels in each window, and calculates the brightness variance based on formula (2).

[0048] In some embodiments, the charging device determines the lighting scene based on global brightness, brightness variance, and a preset threshold. For example, if the global brightness is less than a first brightness threshold, the lighting scene is determined to be a low-light scene. If the global brightness is greater than the first brightness threshold but less than a second brightness threshold, the lighting scene is determined to be a normal-light scene. If the global brightness is greater than the second brightness threshold, the lighting scene is determined to be a high-light scene. The first brightness threshold is less than the second brightness threshold. For example, the first brightness threshold is 50 lux, and the second brightness threshold is 200 lux. For example, if the brightness variance is less than a preset threshold, it is determined to be a low-contrast scene. The preset threshold can be, for example, 100 lux. For example, if 50 lux ≤ L... global ≤200 lux and C local In cases where the light intensity is less than 100 lux, the lighting scene is determined to be a low-contrast normal light scene. For example, an image of a charging port taken in a uniformly dark area caused by aging garage lights corresponds to a low-contrast normal light scene.

[0049] In some embodiments, the grayscale image can be pre-denoised before calculating the global brightness and brightness variance. For example, Gaussian filtering can be used to denoise the grayscale image to remove noise. This Gaussian filtering can be, for example, a Gaussian filter with a standard deviation δ of 0.8 and a Gaussian kernel of 3×3.

[0050] In this embodiment of the application, after obtaining the light intensity, the charging device can perform targeted enhancement processing on the charging port image based on the light intensity, which can effectively improve the accuracy and robustness of subsequent pose recognition.

[0051] In some embodiments, when the lighting scene is a low-light scene, a multi-frame image fusion technique is used to process the charging port image to obtain the target image. This multi-frame image fusion technique fuses the brightness values ​​of multiple charging port images to obtain a charging port image with more uniform brightness, clearer details, and less noise. The multiple frames can be, for example, 3 or 5 frames; this embodiment does not specifically limit this. The core of the multi-frame image fusion technique is to aggregate the brightness values ​​of pixels at the same coordinate in multiple frames. This aggregation can be averaging or weighted averaging, etc. The multi-frame image fusion technique can effectively reduce image noise while preserving key features, improving image quality and providing more reliable input for subsequent target detection and segmentation.

[0052] Taking a 3-frame image as an example, the charging device uses formula (3) to fuse the brightness values ​​based on the 3 grayscale images corresponding to the 3-frame images. For example, the brightness values ​​of pixels with the same coordinates in the 3 grayscale images are added together and then averaged. Among them, the 3 grayscale images have the same size, such as W×H, and are aligned at the pixel level.

[0053] , formula (3).

[0054] in, It is the brightness value of the merged pixel. The pixels in the first grayscale image brightness value, It is the pixel in the second grayscale image. brightness value, It is the pixel in the third frame of the grayscale image. The brightness value. Pixels in the first frame of the grayscale image. Pixels in the second grayscale image and pixels in the third frame grayscale image The coordinates in the corresponding grayscale images are consistent. After fusion, a completely new grayscale image is obtained. The charging device can process this new grayscale image to obtain the target image. For example, the charging device maps this new grayscale image to an RGB image to obtain the target image. The mapping process can refer to existing technologies and will not be elaborated here.

[0055] In some embodiments, when the lighting scene is a strong light scene, the image may suffer from overexposure, resulting in distortion of bright areas and missing terminal outlines. Therefore, highlight suppression technology can be used to process the charging port image to obtain the target image. Highlight suppression technology is a method of suppressing the brightness value of overexposed areas in the image by suppressing the value of the brightness channel.

[0056] For example, the charging device converts the RGB format charging port image from the RGB color space to the Hue-Saturation-Value (HSV) color space, separating the hue, saturation, and luminance channels. Then, the charging device performs adaptive threshold truncation on the luminance channel. For example, setting the luminance threshold to 230 sets the luminance value of all pixels with a luminance value higher than this threshold to 230, thereby effectively suppressing the brightness of overexposed areas. In some embodiments, the charging device can also simultaneously increase the overall value of the saturation channel by 15% to compensate for the potential decrease in color saturation caused by luminance limiting, avoiding color casts in highlight areas and maintaining natural and realistic colors. Finally, the charging device converts the processed image from the HSV color space back to the RGB color space to obtain the target image.

[0057] In some embodiments, when the lighting scene is a low-contrast normal light scene, although the brightness of the charging port image is normal, the terminal edges are blurred due to insufficient image texture. Therefore, texture enhancement technology is used to process the charging port image to obtain the target image. Texture enhancement technology is an image processing method used to improve the outline of the terminals in the charging port image. A terminal refers to a metal contact piece or hole inside the charging port that performs a specific electrical function. Terminals can also be called charging holes, sockets, charging terminals, etc.

[0058] For example, the charging device performs a linear grayscale transformation on the grayscale image, linearly mapping its grayscale value range to the [0, 255] interval to fully enhance the overall contrast of the image. Next, the charging device extracts edge features from the charging port image. For instance, the charging device uses the Sobel operator to perform a convolution operation on the grayscale image processed in the previous step, calculating its gradients in the horizontal and vertical directions respectively. The charging device combines the gradient results in the two directions into an edge map to highlight edge information. Simultaneously, the charging device uses Gaussian filtering to smooth the grayscale image processed in the first step. Finally, the charging device superimposes and fuses the edge map obtained in the second step with the smoothed image obtained in the third step to obtain a new grayscale image, and based on this new grayscale image, obtains the target image. For example, the charging device maps this new grayscale image to an RGB image to obtain the target image. The mapping process can refer to existing technologies and will not be elaborated further.

[0059] After obtaining the target image, the charging device can then execute S102.

[0060] S102, crop out the sub-image corresponding to the terminals included in the charging port from the target image.

[0061] The charging port image may include terminal images corresponding to the terminals. In this embodiment, the charging device can crop out a sub-image containing only the terminals from the target image. That is, the sub-image is a terminal image cropped from the target image. For example, the charging port includes M terminals, and the charging device can crop out M corresponding sub-images from the target image.

[0062] In this embodiment, each terminal should be displayed in the charging port image. For example, after acquiring the charging port image, the charging device can detect whether the charging port image includes M terminal images. If the charging port image includes M terminal images, the charging device can execute S101. If the number of terminal images included in the charging port image is less than M, the charging device can acquire the charging port image again until a charging port image including M terminal images is acquired.

[0063] In some embodiments, an object detection model can be used to crop sub-images from a target image. The object detection model detects terminals in the target image and outputs the bounding box information of the terminals. The bounding box can be, for example, the smallest bounding rectangle including the terminal, and the bounding box information is used to characterize the position of the smallest bounding rectangle in the target image. The bounding box information can be, for example, (x, y, w, h), where (x, y) characterizes the coordinates of the bounding box in the target image, for example, the coordinates of the center point of the bounding box in the target image. (w, h) characterizes the size of the bounding box. The bounding box information can also be, for example, (x1, y1, x2, y2), where (x1, y1) represents the X and Y coordinates of the top-left vertex of the bounding box in the target image; and (x2, y2) represents the X and Y coordinates of the bottom-right vertex of the bounding box in the target image.

[0064] After using an object detection model to output bounding box information, the charging device can crop out a sub-image containing only the terminals from the target image based on the bounding box information.

[0065] In some embodiments, different lighting scenarios correspond to different target detection models. For example, low-light scenarios, normal-light scenarios, strong-light scenarios, and low-contrast normal-light scenarios each have their own target detection models. The target detection model corresponding to the lighting scenario has model parameters and / or model structure matched to the corresponding lighting scenario. In other words, in this embodiment, different target detection models are trained for different lighting scenarios, achieving a synergistic improvement in accuracy, efficiency, and robustness. This not only allows each model to focus on learning features specific to the scene, but also significantly improves the recognition accuracy and reliability in different lighting scenarios.

[0066] The object detection model can be, for example, the YOLO series models such as YOLOv4, YOLOv5, YOLOv7, YOLOv8, etc., or it can be a convolutional neural network. The following section will use the YOLOv5 object detection model as an example for introduction.

[0067] The YOLOv5 model parameters and structure are matched to those for low-light scenarios. For example, the anchor box size is larger and the confidence threshold is smaller. Because the signal-to-noise ratio of charging port images acquired in low-light scenarios is low, the terminal images are usually small. Appropriately increasing the anchor box size and lowering the confidence threshold helps improve the recall rate of candidate boxes, avoids missed detections due to weak features, and ensures that more real terminals are successfully detected. For example, in low-light scenarios, the anchor box size is adjusted from the preset original size (10×13, 16×30, 33×23) to (12×15, 18×35, 36×26), and the confidence threshold is reduced from the original threshold of 0.5 to 0.35. Meanwhile, the YOLOv5 model corresponding to low-light scenes has a low-light feature enhancement convolutional block module connected after the third convolutional layer of the feature extraction backbone network (CSPDarknet53). This low-light feature enhancement convolutional block module consists of a 3×3 convolutional layer, a batch normalization (BN) layer, and a linear correction unit such as the ReLU activation function in sequence. It is used to enhance the ability to extract features such as terminal edges and contours and compensate for feature loss caused by low light.

[0068] The YOLOv5 model corresponding to a normal lighting scene has the same model parameters and structure as the normal lighting scene. The model parameters of the YOLOv5 model corresponding to a normal lighting scene can be preset values, such as anchor frame sizes being preset original sizes (10×13, 16×30, 33×23), and confidence thresholds being the original threshold of 0.5. The model structure of the YOLOv5 model corresponding to a normal lighting scene can be, for example, the model structure of existing YOLOv5 models, which will not be elaborated further.

[0069] The YOLOv5 model parameters and structure are matched to those of strong lighting scenarios. For example, the confidence threshold is relatively small, while the non-maximum suppression (NMS) threshold is relatively large. In strong lighting scenarios, metal surfaces are prone to high-gloss reflections, causing partial obscuring or weakening of the true texture and contour features of the terminals, resulting in a general decrease in the confidence score of the model output. Setting a larger NMS threshold helps avoid incomplete detection or decreased localization accuracy due to excessive suppression. For example, the confidence threshold is reduced from the original threshold of 0.5 to 0.35, and the NMS threshold is reduced from the original threshold of 0.45 to 0.3.

[0070] The YOLOv5 model parameters and structure corresponding to the low-contrast normal light scene are matched to those of the low-contrast normal light scene. For example, the anchor frame size is larger, the confidence threshold is smaller, and the NMS threshold is smaller. In the low-contrast normal light scene, the charging port terminal has little difference from the background and the edges are blurred, resulting in weak feature response and unclear spatial localization. A larger anchor frame size can better match the characteristics of the terminal's increased apparent size due to edge diffusion; a smaller confidence threshold can reduce the detection requirements and prevent real terminals from being filtered out due to weak features; a smaller NMS threshold can implement stricter inter-frame deduplication and effectively suppress multiple overlapping false detections around a single terminal due to edge blurring. Specifically, in this scene: the anchor frame size is increased by 5%-8% from the original, for example, the anchor frame size is adjusted from (10×13, 16×30, 33×23) to (11×14, 17×32, 35×25), the confidence threshold is reduced from 0.5 to 0.4, and the NMS threshold is reduced from 0.45 to 0.4.

[0071] After obtaining a sub-image that includes only the terminals, the charging device can process each sub-image individually to obtain a mask image corresponding to each sub-image. For example, the charging device can execute S103.

[0072] S103 uses a semantic segmentation model to process M sub-images to obtain corresponding M mask images.

[0073] Semantic segmentation is a deep learning model used to identify and label the category of each pixel in an image. In this embodiment, the semantic segmentation model performs precise pixel-level segmentation of the terminals in each sub-image, outputting a binary mask image. In this mask image, pixels belonging to terminals have a first value, and pixels belonging to the background have a second value. The first value can be, for example, 255, and the second value can be, for example, 0. This mask image clearly distinguishes the terminal region from the background region and provides a reliable basis for subsequent coordinate extraction.

[0074] Semantic segmentation models can be, for example, the DeeplabV3plus enhanced version. DeeplabV3plus uses ResNet-101 as the feature extraction backbone, exhibiting good segmentation performance and generalization ability. For instance, by sequentially inputting M sub-images into the semantic segmentation model, M mask images corresponding one-to-one with the M sub-images can be obtained.

[0075] In some embodiments, different semantic segmentation models correspond to different lighting scenarios. For example, low-light scenarios, normal-light scenarios, strong-light scenarios, and low-contrast normal-light scenarios each have their own semantic segmentation models. The semantic segmentation model corresponding to a lighting scenario has model parameters and / or model structure that match the corresponding lighting scenario. In other words, in this embodiment, different semantic segmentation models are trained for different lighting scenarios, achieving a synergistic improvement in accuracy, efficiency, and robustness. This not only allows each model to focus on learning features specific to a particular scenario, but also significantly improves the recognition accuracy and reliability in different lighting scenarios. The following explanation uses the DeeplabV3plus model as an example for semantic segmentation.

[0076] The model parameters and / or model structure of the DeeplabV3plus model corresponding to the low-light scene are matched with the low-light scene. For example, the weights are low-light specific weights. For example, the model is trained using charging port images collected in a low-light scene to obtain low-light specific weights. In addition, when training the model, an edge loss is introduced into the standard loss function, and its calculation formula is shown in formula (4).

[0077] , formula (4).

[0078] in, It is edge loss, M pred_edge M is the edge mask predicted by the model. gt_edge This is a ground truth edge mask labeled for the training set. By directly supervising the pixels in the edge regions, the model is forced to learn and enhance the terminal contour features, effectively compensating for the loss of segmentation accuracy caused by edge blurring in low-light environments.

[0079] In some embodiments, the dilation rate parameter of the atrous spatial pyramid pooling (ASPP) module in the DeeplabV3plus model corresponding to low-light scenes is adjusted from the original configuration (6, 12, 18) to (4, 8, 12). Appropriately reducing the dilation rate can effectively suppress noise interference while maintaining multi-scale feature extraction capabilities, allowing the model to focus on more discriminative local structural features, thereby improving segmentation robustness in low-light conditions.

[0080] The model parameters and / or model structure of the DeeplabV3plus model corresponding to the normal lighting scene are matched with those of the normal lighting scene. For example, the model parameters of the DeeplabV3plus model corresponding to the normal lighting scene are preset values, and the model structure can refer to existing technologies, which will not be elaborated further. In some embodiments, the DeeplabV3plus model corresponding to the normal lighting scene is trained based on charging port images acquired in the normal lighting scene.

[0081] The DeeplabV3plus model parameters and / or model structure corresponding to the strong light scene are matched to the strong light scene. For example, the weights are strong light-specific weights. For instance, the model is trained using charging port images collected under strong light scenes to obtain strong light-specific weights. Furthermore, when training this model, an edge loss is introduced into the standard loss function, as described above, and will not be repeated here. Strong light can easily cause the edges of terminals to become blurred due to overexposure. This edge loss term directly supervises the pixels in the edge region, forcing the model to focus on learning and recovering the contour information masked by highlights during training, thereby improving the edge segmentation integrity under strong light interference.

[0082] The model parameters and / or model structure of the DeeplabV3plus model corresponding to low-contrast normal light scenes are matched to those of the low-light scenes. For example, a texture attention module is introduced in the decoder section, embedded between the feature fusion layer and the segmentation prediction layer, to enhance the feature response to the surface texture of the terminal. The module adopts a three-way parallel convolutional architecture, using 3×3, 5×5, and 7×7 convolutional kernels to extract multi-scale texture features. The number of output channels in each branch is set to 1 / 3 of the number of input feature channels, and batch normalization and ReLU nonlinear activation are performed sequentially after the convolution operation. By performing 1×1 convolution fusion and Sigmoid activation on the three output features, an attention weight map focusing on the micro-texture region of the charging port terminal is generated. Finally, by performing element-wise multiplication of the original feature map and the attention weight map, and optimizing with the ReLU activation function, the enhanced feature is output as the final representation input to the segmentation head in the decoder stage. This design effectively improves the boundary accuracy and region consistency of terminal segmentation by strengthening the saliency expression of texture features.

[0083] For M mask images, the charging device can execute S104 to determine the local coordinates of M terminals in the M sub-images.

[0084] S104, Based on M mask images, determine the local coordinates of M terminals in the corresponding M sub-images.

[0085] Given M masked images, the charging device can determine the local coordinates of M terminals in the corresponding sub-images. The local coordinates of a terminal in a sub-image can be the coordinates of the center point of the terminal in the corresponding sub-image.

[0086] In some embodiments, a clustering algorithm can be used for each mask image to determine the coordinates of the center point of the terminal. For example, for each mask image, an edge detection algorithm is used to extract the set of contour points of the terminal. Based on the set of contour points, the integrity level of the terminal's contour is determined. Based on the integrity level of the terminal's contour, a clustering algorithm is performed to determine the coordinates of the terminal's center point in the corresponding sub-image.

[0087] Edge detection algorithms are used to accurately identify and locate the boundary between the terminal region and the background region in a masked image. An example edge detection algorithm is the Canny edge detection algorithm. For instance, inputting a masked image into an edge detection algorithm yields a set of contour points for the terminal. This set of contour points includes the set of pixels on the contour of the terminal in the target image. In other words, the set of contour points is the set of contour points of the terminal in the target image.

[0088] In some embodiments, each mask image can be pre-filtered before using the edge detection algorithm. This filtering process can, for example, involve performing a combined morphological filter on the mask image by sequentially performing "opening and closing operations on 3×3 rectangular structuring elements". This combined operation can effectively remove isolated noise points and burrs with an area smaller than 3 pixels in the mask image, and simultaneously repair tiny breaks and gaps smaller than 2 pixels on the terminal contour, thereby obtaining a mask image with smoother boundaries and better connectivity.

[0089] In some embodiments, after obtaining the set of contour points, an 8-neighborhood connected component analysis is performed on the extracted set of contour points to calculate the perimeter of each independent contour or the area of ​​the pixels it encloses, and to filter and retain only the main contour with the largest scale, thereby effectively eliminating false contour interference caused by image noise, reflection or adhesion.

[0090] The integrity level indicates the degree of completeness of the contours of terminals in the target image. Integrity levels include complete contours, normal contours, and incomplete contours.

[0091] In some embodiments, the integrity level is determined based on the notch rate and burr density of the contours of terminals in the target image. For example, the notch rate and burr density are determined based on a set of contour points. The notch rate indicates the degree of discontinuity of the contour; the burr density indicates the degree of noise interference in the contour. The integrity level of the sub-image is determined based on the notch rate and burr density.

[0092] For example, the notch rate can be calculated using formula (5) based on the set of edge contour points.

[0093] , formula (5).

[0094] Where Ngap is the total number of pixels in the contour gap, and Ntotal is the total number of pixels in the contour point set. A contour gap is defined as a gap existing when the Euclidean distance between two adjacent contour points is greater than 3 pixels; the number of pixels missing from this gap is counted in N. gap .

[0095] For example, based on the set of edge contour points, the burr density is calculated using formula (6).

[0096] , formula (6).

[0097] in, The number of pixels enclosed by the outline of the terminal. This refers to the total number of skeleton pixels that are identified as burrs. Existing technologies can be referenced, so it will not be elaborated upon here.

[0098] In some embodiments, the search radius and minimum number of points for the clustering algorithm are determined based on the integrity level of the terminal's contour; based on the search radius and minimum number of points, the clustering algorithm is executed to determine the local coordinates of M terminals in the corresponding M sub-images. The clustering algorithm can be, for example, the DBSCAN algorithm. Here, the search radius R is used to define the neighborhood range of a point; the minimum number of points MinPts is used to determine the minimum number of neighborhood points required to identify the core object.

[0099] For example, there is a mapping relationship between the integrity level and the search radius and minimum number of points. Based on this mapping relationship, the charging device determines the search radius and minimum number of points corresponding to the integrity level. This mapping relationship can be, for example, a mapping function or a mapping table. A mapping table can be shown in Table 1.

[0100] Table 1

[0101] In some embodiments, the charging device traverses the set of contour points, and for each contour point, counts its... The number of contour points within the neighborhood. If the number of points exceeds a preset threshold. If so, then that point is determined to be the core point, and it is... All contour points within the neighborhood are grouped into the same initial cluster. The charging device merges the initial clusters corresponding to spatially adjacent core points to form several candidate consensus clusters. From all candidate consensus clusters, the charging device selects those containing the initial center point. The cluster, whose coverage area is defined as the dynamic consensus region. The set of points in this region This is the effective set of points used for the final center point estimation, after filtering out noise and outliers. Then, the charging device uses weighted least squares for the final fitting. Specifically, for... Each point in Its weight Calculate using formula (7): , formula (7).

[0102] in, From the contour points to the initial center point The Euclidean distance is used. The calculated weights are inversely proportional to the distance; that is, the closer a point is to the initial center, the higher its weight and the greater its influence on the final result. Final terminal center coordinates. From formula (8): , formula (8) Where n is the set of valid points The number of contour points in the data is calculated. This weighted fitting process effectively ensures that highly reliable contour points contribute more to the center estimation results, thereby significantly improving the accuracy and robustness of center point localization.

[0103] The charging device can determine the local coordinates of M terminals in their respective sub-images, i.e., the coordinates of the center of the terminal in the corresponding sub-image, based on the method described above and M sub-images. After obtaining the coordinates of the M terminals in their respective sub-images, the charging device can determine the global coordinates of the M terminals in the target image.

[0104] S105, based on the local coordinates of the M terminals in the corresponding M sub-images, obtain the global coordinates of the M terminals in the target image.

[0105] In practice, since each sub-image is a local region cropped from the target image, it is necessary to map the local coordinates of the terminal to global coordinates in the target image. Local coordinates refer to the coordinates of the terminal in the corresponding sub-image, while global coordinates are the global coordinates of the terminal in the target image.

[0106] In some embodiments, the global coordinates of M terminals are determined based on M bounding box information and the local coordinates of M terminals. For example, based on the M bounding box information, the global coordinates of M sub-images in the target image are determined; based on the global coordinates of the M sub-images in the target image and the local coordinates of the M terminals in the corresponding M sub-images, the global coordinates of the M terminals in the target image are determined. For example, based on the M bounding box information, the global coordinates of the M sub-images in the target image can be determined. These coordinates may be, for example, the global coordinates of the center point of the sub-image in the target image. Based on the global coordinates of the M sub-images in the target image and the local coordinates of the M terminals, the global coordinates of the M terminals in the target image can be determined. For example, if a sub-image is located at the top left corner of the target image, and the coordinates of the sub-image are (x, y), and the local coordinates of the terminal in the sub-image are (u, v), then the global coordinates of the terminal in the sub-image in the target image are (x+u, y+v). In this way, the charging device can obtain the accurate coordinates of each terminal in the entire target image, laying the foundation for subsequent 3D pose estimation.

[0107] S106, based on the global coordinates of the M terminals in the target image and the three-dimensional model of the charging port, determine the pose of the charging port.

[0108] In practical implementation, the 3D model of the charging port refers to a pre-established standard charging port structure model, including the relative coordinate relationships of each terminal in 3D space. The 3D model of the charging port is typically based on a standard charging port structure design, with each terminal having its own fixed 3D coordinates. The charging device matches the center point of the terminal in the target image with the center point of the terminal in the 3D model to obtain a 2D-3D correspondence set. A preset algorithm is then used to determine the pose of the charging port. In this embodiment, the pose of the charging port can be understood as the pose of the charging device's camera relative to the charging port.

[0109] The 2D-3D point correspondence set includes M 2D-3D point correspondences. A 2D-3D point correspondence is used to indicate the correspondence between the 2D coordinates of the terminal's center in the 2D image and the 3D coordinates of the terminal's center in the 3D model. That is, it is used to indicate the correspondence between the 2D coordinates of the terminal's center point in the image and its 3D coordinates in the charging port's 3D model.

[0110] The default algorithm can be the Perspective-n-Point (PnP) algorithm. PnP is an algorithm that solves the camera pose relative to a 3D object based on the correspondence between a point in 3D space and its projection point on a 2D image plane.

[0111] For example, the three-dimensional coordinates of the center point of the three-dimensional model terminal are pre-established. (e.g.) Figure 2 As shown, the charging port includes seven terminals: terminal 1, terminal 2, terminal 3, terminal 4, terminal 5, terminal 6, and terminal 7. Terminal 4 is the center terminal, located at the center of the charging port. Using the physical end face of the charging port as the XOY plane and the center point of terminal 4 as the origin of the coordinate system, a three-dimensional coordinate system for the charging port is established based on the right-hand screw rule, determining the Z-axis direction perpendicular to the end face and inwards. Within this coordinate system, the center points of the seven terminals are selected as feature points, and their three-dimensional coordinates constitute the basic 3D point set for pose calculation. The 3D coordinates of the feature points are shown in Table 2 below, in millimeters (mm).

[0112] Table 2

[0113] For example, the average center point is calculated based on the local coordinates of the M terminals in the target image. For instance, the average center point is calculated using formula (9).

[0114] , formula (9) Among them, (x) c y c (x) represents the coordinates of the mean center point. i y i ) represents the local coordinates of the center point of the i-th terminal in the target image. M is the total number of terminals included in the charging port.

[0115] Obtain the Euclidean distance from the center point of each of the M terminals to the average center point. Select the ellipse corresponding to the nearest center point as the center terminal of the charging port and establish a correspondence between this center point and the center point of terminal 4 in the preset 3D model.

[0116] Next, sort all the ellipses according to their areas, determine the two ellipses with the smallest areas, and set the center points of these two ellipses (denoted as ). and The lines connecting these points form a reference line segment. Then, the center point of terminal 4 (denoted as...) is calculated. The foot of the perpendicular from the line containing the line segment The calculation formula is shown in formula (10): , formula (10).

[0117] Where P is a point to the straight line The dangling foot.

[0118] Will hang down The line connecting the center point of terminal 4 to the dividing line vector is used as the dividing line vector. Connecting the center points of all other points to the center point of terminal 4 forms feature vectors. The sign of the cross product between each feature vector and the dividing line vector determines whether it is located to the left or right of the dividing line, thus achieving initial orientation division. Subsequently, within each orientation region, the feature vectors are sorted in ascending or descending order based on the angle between them and the dividing line vector. This sorting order is consistent with the preset numbering order of the 3D model feature points, thereby ultimately matching all detected 2D image coordinates with the preset 3D model coordinates one by one.

[0119] In some embodiments, the pose of the charging port is determined by using the PnP algorithm, which combines the Random Sample Consensus (RANSAC) algorithm, based on the 2D-3D point correspondence set. For example, the projection process from the 3D point of the charging port to the 2D image is defined as π, and this projection process can be represented by formula (11).

[0120] π(K,R,t,P M )=R P W +t, formula (11).

[0121] Where K is the camera intrinsic parameter matrix, R is the rotation matrix, t is the translation vector, and P M Let P be the two-dimensional coordinates of the object. W These are the three-dimensional coordinates of the object.

[0122] The goal of the PnP algorithm optimization is to minimize the reprojection error, which can be expressed by formula (12).

[0123] , formula (12) in To retain only the first two dimensions of the homogeneous coordinates, These are the two-dimensional coordinates of the i-th feature point. is the three-dimensional coordinate of the i-th feature point.

[0124] The charging device can perform steps 1-2 to determine the orientation of the charging port.

[0125] Step 1: Random sampling and initial pose estimation.

[0126] From all From a set of 2D-3D point correspondences, a minimum sample set (e.g., 3 2D-3D point correspondences) is randomly selected. Using this sample set, an initial camera pose is calculated through analytical methods such as direct linear transformation (DLT) or efficient perspective-n-point (EPnP). .

[0127] Step 2, initial pose iterative optimization.

[0128] The reprojection error of all points is calculated based on the current pose, and inliers with errors below a threshold are selected. The pose is then refined through nonlinear iterative optimization using all inliers.

[0129] Repeat steps 1) and 2) above until the maximum number of RANSAC iterations is reached or the number of inliers reaches a threshold. Select the pose with the most inliers as the optimal pose estimate output.

[0130] In summary, the pose acquisition method provided in this application is suitable for home charging scenarios and has the advantages of low cost, high precision, and strong robustness, solving the problems of existing charging methods relying on high-cost hardware, being sensitive to lighting conditions, and having insufficient reliability.

[0131] The following section uses an electric vehicle charging scenario as an example to introduce a method for obtaining pose provided by this application.

[0132] With breakthroughs in artificial intelligence, 5G communication technology, and multimodal perception, global intelligent transportation is evolving towards "unmanned, all-scenario, and high-energy-efficiency." As the penetration rate of electric vehicles increases year by year, the intelligent upgrading of their supporting infrastructure, as the main carrier of autonomous driving technology, has become a core area of ​​industrial competition. Against this backdrop, the closed-loop scenario of autonomous parking and automatic charging, with its profound reconstruction of user experience, resource efficiency, and commercial value, is becoming a breakthrough point for the integration of smart cities and the new energy industry. The precise positioning of electric vehicle charging stations by charging robots is a key technology for meeting the "last mile" needs of future transportation.

[0133] Among them, charging port positioning refers to the function of accurately estimating the 6D pose of the charging port of electric vehicles in order to enable the charging robot to achieve high-precision automatic charging gun insertion.

[0134] This application provides a low-cost, high-precision, and robust method for locating electric vehicle charging ports in home charging scenarios for electric vehicle users, solving the problems of existing charging robots relying on high-cost hardware, being sensitive to lighting conditions, and lacking sufficient reliability.

[0135] Existing charging port identification and positioning solutions can be divided into two categories: active depth perception solutions and passive visual perception solutions.

[0136] Active depth sensing solutions are represented by active depth sensors such as structured light, time-of-flight (ToF) cameras, and lidar. They generate high-precision 3D point clouds by emitting coded light spots or laser pulses, and then perform post-processing using data processing algorithms such as the iterative closest point (ICP) algorithm to achieve pose calculation.

[0137] A proposed 6D pose detection method for charging ports based on a monocular camera and LiDAR has been developed. This method detects the 3D point cloud of the target object using LiDAR, performs target detection on the camera image, locates the target's position in the image, and finally uses the least squares method to locate the charging port. Such methods can achieve sub-millimeter-level positioning accuracy under ideal conditions, but they have significant limitations. For example, environmental sensitivity and accuracy limitations exist; strong light or reflections from metal surfaces can easily lead to point cloud distortion, resulting in significant multipath effects, and the inability to accurately reconstruct the internal details of the charging port, rendering rich geometric feature information unusable. Furthermore, the high cost and deployment difficulty are significant issues. High-cost sensors (such as LiDAR) are difficult to popularize in consumer scenarios, and multi-sensor calibration requires specialized equipment, resulting in complex processes and high deployment difficulty, hindering the mass deployment of charging robots. Additionally, the computational load and time-consuming nature of point cloud-based charging port pose estimation make it difficult to meet the real-time requirements of real-world scenarios, and its reliance on high-performance CPU processors also significantly increases costs. Therefore, in order for charging robots to successfully enter the consumer market and be used in home settings, it is necessary to minimize costs and improve robustness as much as possible.

[0138] Passive visual perception schemes use only ordinary cameras and rely on traditional image processing or deep learning neural networks to locate the charging port. For example, a two-stage visual detection and localization method for electric vehicle charging ports based on shape priors has been proposed. This method achieves coarse localization of the charging port by template matching of the extracted charging port contour, then initializes the center point position of the charging port in the extracted local images, and optimizes the center point position using a center point regression model to achieve fine localization. Subsequently, binocular epipolar geometry constraints are used to obtain the 3D coordinates of each center point, and finally, the ICP algorithm is used to obtain the accurate charging port pose. However, because the center point of the charging port needs to be obtained from two images, the binocular epipolar geometry for obtaining 3D coordinates introduces double errors, and the center point regression model uses a direct regression method, which is difficult to fully adapt to center point estimation under different poses.

[0139] In summary, the main challenges of charging port detection in home scenarios stem from the complex and variable lighting conditions (such as the low-light environment of underground parking garages and direct sunlight outdoors), the susceptibility of charging terminals to dust accumulation or obstruction by charging cables, and the strict cost control that limits the application of high-end equipment such as binocular vision. Existing technical solutions have not yet effectively addressed these issues.

[0140] To address these issues, this application provides a monocular vision pose estimation method for electric vehicle charging ports that is illumination-adaptive and incorporates geometric constraints. This method is characterized by low cost, high accuracy, and strong robustness, and proposes the following innovative solutions to the aforementioned problems. For example, it utilizes a passive visual perception hardware solution, employing a camera and a low-cost edge computing platform, which can be mounted on robotic arms of different configurations, without relying on common, high-cost 6-DOF collaborative robotic arms. Another example is the design of a multi-level model incorporating an illumination evaluation submodule, enabling real-time adjustment of model parameters based on scene illumination conditions, and a dynamic adjustment mechanism with illumination-adaptive functionality. Yet another example is the use of a dynamic consensus region mechanism based on contour integrity and a PnP optimization algorithm incorporating standard geometric constraints. Furthermore, a dataset construction method based on topology enhancement achieves high-precision and robust estimation of the 6D pose of charging ports in home scenarios, solving the problems of poor environmental sensitivity, robustness, and insufficient accuracy in existing solutions.

[0141] In this embodiment, the charging port has a standard structure as specified in the standard, specifically including a circular groove on the outer edge of the charging port and internal cylindrical terminals, with the charging port comprising seven cylindrical terminals. These geometric features provide a reliable basis for the algorithm's detection and positioning.

[0142] In this embodiment, a hardware platform consisting of a consumer-grade monocular camera and an edge computing device with a deep learning model hardware acceleration module is used to eliminate the dependence on depth cameras such as structured light cameras and lidar, as well as high-performance CPU main control devices, when the charging robot locates the charging port.

[0143] In some embodiments, addressing the common challenges of specific lighting environments in residential parking spaces, such as insufficient underground lighting or excessive outdoor sunlight, which lead to significant image noise, difficulty in clearly revealing key features of the charging port, and challenges in extracting effective contour features, a lighting evaluation submodule is proposed. This module constructs a multi-level deep learning model structure to achieve a dynamic adjustment mechanism for the multi-level model, adapting to lighting conditions. This allows for adaptive changes in the model's internal parameters within complex and changing lighting scenarios, enabling dynamic real-time matching of the working mode with the working environment. This avoids the poor adaptability of traditional image processing workflows while improving robustness and the accuracy of charging port detection and segmentation. The lighting evaluation module invokes two object detection models and a semantic segmentation model. One object detection model identifies the charging port image from the vehicle image, while the other crops a sub-image containing only the terminals from the charging port image. The input to this lighting evaluation module is a real-time image of the electric vehicle captured by a monocular vision camera, and the output is a high-precision segmentation mask image of the terminals inside the charging port. This allows subsequent modules to extract accurate feature information from the charging port terminals for charging port pose estimation.

[0144] For example, the core of this illumination evaluation module lies in its design and construction of a three-layer closed-loop mechanism: illumination perception, parameter mapping, and model adaptation. Illumination perception refers to extracting global brightness and brightness variance from the charging port image to construct quantitative illumination evaluation indicators, achieving accurate segmentation of the illumination scene. Parameter mapping refers to clarifying the adjustment rules for key model parameters under different illumination conditions based on the mapping relationship table between the illumination scene and the model. Model adaptation refers to loading the corresponding parameter configuration in real time according to the illumination scene label, activating the corresponding model structure, and ensuring stable detection and segmentation accuracy.

[0145] The following is combined with Figure 3 This application introduces a method for obtaining pose based on an embodiment.

[0146] S1, model preprocessing. S1 includes S11 and S12.

[0147] S11, Construct the sample dataset.

[0148] This application utilizes a monocular camera mounted on a charging robot to extensively collect images of electric vehicle charging ports under varying lighting conditions, shooting distances, and perspectives in both indoor and outdoor parking environments. Dedicated datasets are then constructed for training each model in the multi-level detection and image segmentation module. For example, after image acquisition, 5000 original images are obtained, covering all terminal layout types of AC charging ports. After enhancement, these images are expanded to 20000 sample images. The enhancement methods are described below and will not be elaborated upon further.

[0149] S12, Model Training and Deployment.

[0150] The datasets were divided into training and validation datasets in an 8:2 ratio. The OpenMMLab open-source model training framework was used to train the model for 40,000 epochs. Every 500 epochs, the model performance was evaluated on the test dataset, and the weights of the best performing model were recorded. After training was completed, the best performing model was selected for deployment.

[0151] S2, based on the 3D model of the charging port, determines the 3D coordinates of each terminal.

[0152] S3, Image Acquisition and Preprocessing. S3 includes S31 and S32.

[0153] S31 uses a monocular camera to capture images of the charging port.

[0154] S32 performs distortion correction processing on the charging port image.

[0155] Distortion correction is used to correct image distortion, ensuring image accuracy and facilitating subsequent processing.

[0156] For radial distortion, the following formula is used to correct the distortion in the first image.

[0157] .

[0158] .

[0159] Where (x, y) are the ideal coordinates of the camera's normalized plane. r is the radial distance from the pixel to the image center. For example, r = 2 + 2 . 1. 2. 3 represents the radial distortion coefficient. , () are coordinates after radial distortion correction.

[0160] To address tangential distortion, the following formula is used to perform distortion correction on the first image.

[0161]

[0162]

[0163] Where (x, y) are the ideal coordinates of the camera's normalized plane. r is the radial distance from the pixel to the image center. For example, r = 2 + 2 . , This represents the tangential distortion coefficient. , () are the coordinates after tangential distortion correction.

[0164] S4, obtain the local coordinates of the terminal. S4 includes S41-S47.

[0165] S41, Based on the charging port image, determine the lighting scene when the charging port image was acquired.

[0166] S42, based on the lighting scene when the charging port image was acquired, enhance the charging port image to obtain the target image.

[0167] S43, based on the lighting scene when the charging port image was acquired, load the corresponding target detection algorithm.

[0168] S44: Input the target image into the corresponding target detection algorithm to obtain 7 sub-images.

[0169] S45, based on the lighting scene when the charging port image was acquired, load the corresponding semantic segmentation model.

[0170] S46. Input the 7 sub-images into the corresponding semantic segmentation model to obtain the corresponding 7 mask images.

[0171] S46, for each of the 7 mask images, determine the local coordinates of the terminal in the corresponding sub-image.

[0172] Please refer to the introduction above; I will not repeat it here.

[0173] S5. Based on the local coordinates of the seven terminals in the sub-image, the global coordinates of the seven terminals in the target image are obtained.

[0174] S6, establish a 2D-3D point correspondence set.

[0175] S7 uses the PnP algorithm, which combines the RANSAC algorithm, to estimate the charging port pose based on the 2D-3D point correspondence set.

[0176] Existing methods for detecting and segmenting charging port terminals using deep learning models rely on dataset annotation methods that only provide positional and contour information of the terminals. This prevents the model from learning the structured relationship features between terminals, leading to false positives and false negatives during actual operation. To address this issue, this application proposes an integrated solution of "hierarchical annotation and topology enhancement." It designs a hierarchical annotation system based on terminal functions and a topology-preserving dataset enhancement method. The topological relationships between terminals are quantified and integrated into the dataset annotation system. Data enhancement emphasizes the invariance of topological structure. By combining training samples under occlusion scenarios, high reliability of terminal detection and sorting is achieved, thereby improving the model's performance and robustness in complex scenarios.

[0177] In some embodiments, target annotation information is added to historical charging port images. This target annotation information defines the relative positional relationships between the M terminals included in the charging port. Based on the target annotation information, the historical charging port images are enhanced. The enhanced historical charging port images are then used to train a semantic segmentation model and / or an object detection model. The target annotation information can be, for example, the topological relationship between the terminals, including Euclidean distance, relative angle, and position vector. The Euclidean distance can be the straight-line distance between the i-th terminal and the center terminal, calculated using the following formula: The center terminal can be, for example, the terminal located at the center of the charging port. For distance. , ) represents the coordinates of the i-th terminal in the historical charging port image, for example, the coordinates of the center point of the i-th terminal in the historical charging port image. , () represents the coordinates of the center terminal in the historical charging port image. The relative angle can be the angular relationship between the i-th terminal and the center terminal, calculated using the following formula: . The relative angle. The position vector can be the displacement of the i-th terminal relative to the center terminal in the X and Y directions, calculated using the following formula: , .

[0178] In some embodiments, the Euclidean distance can also be the straight-line distance between the i-th terminal and each of the (M-1) terminals. The relative angle can also be the angular relationship between the i-th terminal and each of the (M-1) terminals. The position vector can be the displacement of the i-th terminal and each of the (M-1) terminals in the X and Y directions.

[0179] The coordinates of the i-th terminal and the center terminal in the historical charging port image are determined based on manually annotated information. This manually annotated information includes the positions of the charging port detection box, the terminal detection box, and the terminal mask image. The charging port detection box is the smallest bounding rectangle encompassing the charging port image. For example, the position of the charging port detection box is (x1, y1, x2, y2), with an accuracy of ±1 pixel, covering the 0-2 pixel area of ​​the charging port's outer edge. Each terminal has an independent detection box, completely distinguishing all functional terminals of the charging port, with an accuracy of ±1 pixel, covering the 0-2 pixel area of ​​the terminal's outer edge. The mask image is a pixel-level binary mask image, with each terminal corresponding to an independent mask image. The mask shape is a standard ellipse, and the terminal edge pixel error is ≤1 pixel.

[0180] In some embodiments, scene feature annotations are added to historical charging port images. These scene feature annotations include lighting conditions, occlusion type, shooting angle, and terminal integrity. Lighting conditions can be, for example, low light, normal light, strong light, and backlight. Occlusion type can be, for example, dust occlusion, charging cable occlusion, or foreign object occlusion. Shooting angle can be, for example, frontal, oblique, secondary, or side view. Terminal integrity can be, for example, complete, slightly occluded, moderately occluded, or severely occluded.

[0181] Traditional image enhancement expands the number of training samples by scaling and cropping the original image. This application, targeting the home electric vehicle charging scenario, proposes different enhancement strategies based on target annotation information. These strategies include rigid transformation topology preservation enhancement, non-rigid transformation topology preservation enhancement, structured shearing enhancement, and scene-based occlusion simulation enhancement.

[0182] The rigid transformation topology preservation enhancement strategy includes rotation and scaling. The rotation angle θ ranges from -30° to 30°, and enhancement samples are generated in 5° increments. The coordinate synchronous transformation calculation formula is shown in formula (13).

[0183] , formula (13).

[0184] Where (x, y) are the coordinates before the transformation; These are the transformed coordinates; The rotation angle; The selected rotation center (e.g., the coordinates of the terminal center point) is used. The transformation preserves the topological relationship between terminals, with the following constraints: the angular deviation of the connection between adjacent terminals Δθij ≤ 1°; the relative distance change rate Δdij ≤ 2%.

[0185] The scaling factor is 0.8-1.2 times, with a step size of 1. The constraint is: the relative angle θ between all terminals. ij Unchanged; relative distance d ij The image is scaled synchronously using a scaling factor λ; after scaling, the entire charging port area is always contained within the image boundary.

[0186] Non-rigid transformation topology preservation enhancement includes blurring and noise reduction. For example, Gaussian blur and salt-and-pepper noise are used for enhancement. The Gaussian blur kernel size is randomly selected between 3×3 and 5×5, with a standard deviation σ of 0.5-1.5, and σ is limited to ≤1.0 in low-light scenes. Furthermore, topology protection is applied to the terminal center coordinate region, meaning that the terminal center coordinate region (10×10 pixels) is not blurred. The salt-and-pepper noise density is set to 0.01-0.05, and noise points are restricted from concentrated distribution, with ≤2 noise points in the same 3×3 region, to avoid obscuring terminal edge features.

[0187] Structured trimming enhancement can specifically be achieved by mimicking the edge importance weighting strategy in graph contrastive learning. Figure 2 For example, the relationships between charging port terminals are divided into two levels: the distance and angle relationships between the center terminal and terminals 1 and 3 are defined as primary important relationships, and the relationships between the remaining terminals are defined as secondary important relationships. This method requires that the shearing region completely avoid all terminals involved in the primary relationships, ensuring that the key topology is not destroyed; if the shearing operation affects terminals involved in the secondary relationships, the corresponding annotation data is simultaneously corrected to maintain annotation consistency, while strictly controlling the total sheared area to not exceed 30% of the original image area. This enhancement strategy effectively improves the robustness of the model to local structural changes while maintaining the core topological constraints.

[0188] The specific details of the scenario-based occlusion simulation enhancement are shown in Table 3. The scenario-based occlusion simulation enhancement is applicable to home charging scenarios.

[0189] Table 3

[0190] The method embodiments of this application have been described in detail above with reference to the accompanying drawings. The device embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be understood that the description of the device embodiments corresponds to the method embodiments; therefore, any parts not described in detail can be referred to the method embodiments above.

[0191] Figure 4 This is a schematic diagram of the structure of a charging device provided in an embodiment of this application. For example, the charging device 400 may include a first processing unit 410, a second processing unit 420, a third processing unit 430, a first determining unit 440, a second determining unit 450, and a third determining unit 460.

[0192] The first processing unit 410 is used to enhance the charging port image based on the lighting scene when the charging port image is acquired, and obtain the target image; the charging port image is the image acquired by the camera of the charging port. The second processing unit 420 is used to crop out M sub-images from the target image, each corresponding to one of the M terminals included in the charging port; where M is a positive integer; and each sub-image includes a terminal image corresponding to a terminal. The third processing unit 430 is used to process M sub-images using a semantic segmentation model to obtain corresponding M mask images. The first determining unit 440 is used to determine the local coordinates of M terminals in the corresponding M sub-images based on M mask images; The second determining unit 450 is used to obtain the global coordinates of the M terminals in the target image based on the local coordinates of the M terminals in the corresponding M sub-images; The third determining unit 460 is used to determine the pose of the charging port based on the global coordinates of the M terminals in the target image and the three-dimensional model of the charging port.

[0193] The first determining unit 440 is further configured to process the charging port image into a grayscale image; determine the global brightness and brightness variance based on the grayscale image; wherein, the global brightness is used to characterize the exposure level of the charging port image; the brightness variance is used to characterize the richness of texture in the charging port image; and determine the lighting scene when acquiring the charging port image based on the global brightness and brightness variance.

[0194] The first determining unit 440 is further configured to process the charging port image using multi-frame image fusion technology to obtain the target image when the lighting scene is a low-light scene; to process the charging port image using highlight suppression technology to obtain the target image when the lighting scene is a strong-light scene; and to process the charging port image using texture enhancement technology to obtain the target image when the lighting scene is a low-contrast normal light scene.

[0195] The second processing unit 420 is further configured to determine a target detection model corresponding to the lighting scene based on the lighting scene when the charging port image is acquired; process the target image based on the target detection model corresponding to the lighting scene to obtain M bounding box information; and crop M sub-images from the target image based on the M bounding box information; wherein the model parameters and / or model structure of the target detection model corresponding to the lighting scene are matched with the lighting scene.

[0196] The third processing unit 430 is also used to determine the semantic segmentation model corresponding to the lighting scene based on the lighting scene when the charging port image is acquired; and to process M sub-images to obtain M corresponding mask images based on the semantic segmentation model corresponding to the lighting scene; wherein the model parameters and / or model structure of the semantic segmentation model corresponding to the lighting scene are matched with the lighting scene.

[0197] The second determining unit 450 is further configured to, for each mask image, employ an edge detection algorithm to extract the set of edge contour points of the terminal; determine the integrity level of the terminal's contour based on the set of edge contour points; the integrity level is used to indicate the degree of integrity of the terminal's contour in the target image; determine the search radius and minimum number of points for the clustering algorithm based on the integrity level; and execute the clustering algorithm based on the search radius and minimum number of points to extract the coordinates of the terminal in the corresponding mask image from the set of edge contour points.

[0198] The second determining unit 450 is further configured to determine the notch rate and burr density of the terminal contour based on the set of edge contour points; wherein the notch rate is used to indicate the degree of interruption of the continuity of the terminal contour; the burr density is used to indicate the degree of noise interference of the terminal contour; and the integrity level of the sub-image is determined based on the notch rate and burr density.

[0199] The second determining unit 450 is further configured to determine the global coordinates of the M sub-images in the target image based on the M bounding box information; and to determine the global coordinates of the M terminals in the target image based on the global coordinates of the M sub-images in the target image and the local coordinates of the M terminals in the corresponding M sub-images.

[0200] The third determining unit 460 is also used to construct a 2D-3D point correspondence set based on the global coordinates of the M terminals in the target image and their predefined 3D coordinates in the 3D model of the charging port; and to determine the pose of the charging port based on the 2D-3D point correspondence set using the PnP algorithm.

[0201] The first processing unit 410 is further configured to add target annotation information to the historical charging port image, the target annotation information being used to define the relative positional relationship between the M terminals included in the charging port; and to perform enhancement processing on the historical charging port image based on the target annotation information, the enhanced historical charging port image being used to train a semantic segmentation model and / or a target detection model.

[0202] Figure 5 This is a schematic diagram of the structure of a charging device provided in an embodiment of this application. For example, the charging device 500 includes one or more processors 510, one or more memories 520, and a camera. Figure 5 The charging device 500 is used to implement the vehicle control method described in the above method embodiments. The charging device 500 may be a control unit in the vehicle, such as a vehicle controller or an engine controller. Alternatively, as a possible implementation, the control unit may be a controller independently set up to implement the above control method.

[0203] The processor 510 can support the charging device 500 in implementing the methods described in the preceding method embodiments.

[0204] The memory 520 stores a program that can be executed by the processor 510, causing the processor 510 to perform the methods described in the preceding method embodiments. The memory 520 may be independent of the processor 510 or integrated into the processor 510.

[0205] Optionally, the charging device 500 may also include a transceiver 530. The processor 510 can communicate with other devices or chips via the transceiver 530. For example, the processor 510 can send and receive data with other devices or chips via the transceiver 530.

[0206] This application provides a computer storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the methods described in any of the above embodiments.

[0207] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0208] The aforementioned processor can be at least one of the following: application-specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field-programmable gate array (FPGA), central processing unit (CPU), controller, microcontroller, and microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0209] The aforementioned computer storage media / memory can be read-only memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD ROM), etc.

[0210] This application provides a computer program including computer-readable code. When the computer-readable code runs in an electronic device, the processor in the electronic device executes some or all of the steps in the above-described method.

[0211] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0212] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0213] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0215] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0217] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0218] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an in-vehicle terminal (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0219] The above are merely embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The above embodiments are merely preferred embodiments provided to fully illustrate this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. A method for obtaining pose, characterized in that, Applied to a charging device, the charging device including a camera, the method includes: Based on the lighting scene when the charging port image is captured, the charging port image is enhanced to obtain the target image; the charging port image is the image captured by the camera of the charging port. M sub-images are cropped from the target image, each corresponding to one of the M terminals included in the charging port; where M is a positive integer; each sub-image includes the terminal image of the corresponding terminal. A semantic segmentation model is used to process the M sub-images to obtain the corresponding M mask images; Based on the M mask images, determine the local coordinates of the M terminals in the corresponding M sub-images; Based on the local coordinates of the M terminals in the corresponding M sub-images, the global coordinates of the M terminals in the target image are obtained; The pose of the charging port is determined based on the global coordinates of the M terminals in the target image and the three-dimensional model of the charging port.

2. The method according to claim 1, characterized in that, The method further includes: The charging port image is processed into a grayscale image; Based on the grayscale image, global brightness and brightness variance are determined; wherein, global brightness is used to characterize the exposure level of the charging port image; and brightness variance is used to characterize the richness of texture in the charging port image. The lighting scene when acquiring the charging port image is determined based on the global brightness and the brightness variance.

3. The method according to claim 2, characterized in that, The lighting scene is one of a low-light scene, a normal-light scene, a strong-light scene, and a low-contrast normal-light scene; the enhancement processing of the charging port image based on the lighting scene when the charging port image is acquired to obtain the target image includes: In the case of a low-light scene, the charging port image is processed using multi-frame image fusion technology to obtain the target image; in the case of a strong-light scene, the charging port image is processed using highlight suppression technology to obtain the target image; in the case of a low-contrast normal-light scene, the charging port image is processed using texture enhancement technology to obtain the target image.

4. The method according to claim 1, characterized in that, The step of cropping M sub-images from the target image, each corresponding to one of the terminals included in the charging port, includes: Based on the lighting scene when the charging port image was acquired, a target detection model corresponding to the lighting scene was determined. Based on the target detection model corresponding to the lighting scene, the target image is processed to obtain the M bounding box information; Based on the M bounding box information, the M sub-images are cropped from the target image; The model parameters and / or model structure of the target detection model corresponding to the lighting scene are matched with the lighting scene.

5. The method according to claim 1, characterized in that, The step of using a semantic segmentation model to process the M sub-images to obtain corresponding M mask images includes: Based on the lighting scene when the charging port image was acquired, a semantic segmentation model corresponding to the lighting scene was determined. Based on the semantic segmentation model corresponding to the lighting scene, the M sub-images are processed to obtain the corresponding M mask images; The model parameters and / or model structure of the semantic segmentation model corresponding to the lighting scene are matched with the lighting scene.

6. The method according to claim 1, characterized in that, Determining the local coordinates of the M terminals in the corresponding M sub-images based on the M mask images includes: For each mask image, an edge detection algorithm is used to extract the set of edge contour points of the terminal; Based on the set of edge contour points, the integrity level of the terminal's contour is determined; the integrity level is used to indicate the degree of integrity of the terminal's contour in the target image. The search radius and minimum number of points for the clustering algorithm are determined based on the integrity level. Based on the search radius and the minimum number of points, the clustering algorithm is executed to extract the coordinates of the terminal in the corresponding mask image from the set of edge contour points.

7. The method according to claim 6, characterized in that, The determination of the integrity level of the sub-image based on the set of edge contour points includes: Based on the set of edge contour points, the notch rate and burr density of the terminal contour are determined; wherein, the notch rate is used to indicate the degree of discontinuity of the terminal contour; and the burr density is used to indicate the degree of noise interference of the terminal contour. The integrity level of the sub-image is determined based on the notch rate and the burr density.

8. The method according to claim 1, characterized in that, Based on the local coordinates of the M terminals in the corresponding M sub-images, the global coordinates of the M terminals in the target image are obtained, including: Based on the M bounding box information, determine the global coordinates of the M sub-images in the target image; Based on the global coordinates of the M sub-images in the target image and the local coordinates of the M terminals in the corresponding M sub-images, the global coordinates of the M terminals in the target image are determined.

9. The method according to claim 1, characterized in that, The step of determining the pose of the charging port based on the global coordinates of the M terminals in the target image and the three-dimensional model of the charging port includes: Based on the global coordinates of the M terminals in the target image and their predefined 3D coordinates in the 3D model of the charging port, a 2D-3D point correspondence set is constructed. Based on the 2D-3D point correspondence set, the pose of the charging port is determined using the PnP algorithm.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: Add target annotation information to historical charging port images. The target annotation information is used to define the relative positional relationship between the M terminals included in the charging port. Based on the target annotation information, the historical charging port image is enhanced, and the enhanced historical charging port image is used to train the semantic segmentation model and / or the target detection model.

11. A device for acquiring pose, characterized in that, include: The first processing unit is used to enhance the charging port image based on the lighting scene when the charging port image is acquired, so as to obtain the target image. The charging port image is an image captured by the camera of the charging port; The second processing unit is configured to crop out M sub-images from the target image, each corresponding to one of the M terminals included in the charging port; where M is a positive integer; and each sub-image includes a terminal image corresponding to a terminal. The third processing unit is used to process the M sub-images using a semantic segmentation model to obtain the corresponding M mask images; The first determining unit is used to determine the local coordinates of the M terminals in the corresponding M sub-images based on the M mask images; The second determining unit is used to obtain the global coordinates of the M terminals in the target image based on the local coordinates of the M terminals in the corresponding M sub-images; The third determining unit is used to determine the pose of the charging port based on the global coordinates of the M terminals in the target image and the three-dimensional model of the charging port.

12. A charging device, characterized in that... include: A camera is used to capture images; One or more processors; A memory for storing one or more programs that, when executed by one or more processors, cause the charging device to implement the pose acquisition method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the pose acquisition method as described in any one of claims 1 to 10.

Images