Vehicle charging port image marking method and device
By employing techniques such as local bounding box selection, grayscale processing, Canny edge detection, Hough circle detection, and least squares fitting, combined with convolutional neural network semantic segmentation algorithms, the problem of low efficiency and low accuracy in vehicle charging port image annotation has been solved. This has enabled efficient and accurate annotation of charging port circular holes, meeting the needs of automatic charging robots for high-precision training datasets and promoting the industrialization of automatic charging robots.
Patent Information
- Application Number
- CN202610053894.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-15
- Publication Date
- 2026-04-28
AI Technical Summary
Existing methods for annotating vehicle charging port images are inefficient, inaccurate, and unstable, failing to meet the requirements of automated charging robots for constructing high-precision training datasets. This has become a bottleneck restricting the performance improvement of monocular perception algorithms and the industrialization of automated charging robots.
By employing techniques such as local bounding box selection, grayscale processing, Canny edge detection, Hough circle detection, and least squares fitting, combined with convolutional neural network semantic segmentation algorithms, efficient and accurate charging port circular hole annotation is achieved through human-computer collaboration. Furthermore, the annotation accuracy and stability are improved through homography matrix and iterative optimization methods.
It greatly improves annotation efficiency and accuracy, shortens the annotation time of a single frame to less than 2 minutes, ensures high accuracy and stability of annotation results, is suitable for building training datasets for automatic charging robots in complex environments, and promotes the industrialization of related technologies.
Smart Images
Figure CN121937530A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and intelligent manufacturing, specifically to a method and apparatus for annotating images of vehicle charging ports. Background Technology
[0002] With the development of the electric vehicle industry, automated charging robots have attracted widespread attention. To enable the robot to insert the charging gun, a monocular perception algorithm must utilize deep learning algorithms to accurately identify and locate several circular sockets within the charging port. The effectiveness of deep learning models is highly dependent on high-quality training data, and obtaining this training data requires a large amount of high-precision image annotation.
[0003] The existing annotation methods mainly fall into the following categories: 1. Traditional semantic segmentation annotation methods: These methods use manually drawn polygons to mark target boundaries. This approach is not only inefficient, taking approximately 5 minutes to annotate a single frame, significantly increasing both the labor and time costs of annotation, but also suffers from a natural mismatch between manually drawn boundaries and the geometric features of circular holes, making it difficult to achieve sub-pixel level annotation accuracy and severely limiting the training effectiveness of subsequent deep learning models.
[0004] 2. Traditional computer vision annotation methods: Typical techniques include Hough circle transform, ellipse fitting, and template matching. These methods require tedious parameter tuning to adapt to specific scenarios, have poor generalization ability, and are easily affected by practical factors such as changes in lighting intensity, image noise interference, and local occlusion, making it difficult to guarantee the stability and consistency of annotation results.
[0005] In summary, existing annotation methods have significant shortcomings in core dimensions such as efficiency, accuracy, and stability, and cannot meet the requirements of automatic charging robots for constructing large-scale, high-precision training datasets. This has become a key bottleneck restricting the performance improvement of monocular perception algorithms and the industrialization of automatic charging robots. Summary of the Invention
[0006] The purpose of this invention is to provide a method and apparatus for annotating vehicle charging port images, which can improve annotation efficiency, ensure annotation accuracy and stability, and be compatible with subsequent deep learning segmentation and pose estimation algorithms.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, this application discloses a method for annotating images of vehicle charging ports, which includes: Acquire an image of the vehicle charging port, and perform partial selection processing on the vehicle charging port image to obtain a partial image including at least one circular hole of the charging port. The local image is sequentially processed by grayscale conversion, contrast-limited adaptive histogram equalization, and median filtering to obtain a denoised image. The Canny edge detection algorithm is used to detect edges in the denoised image to obtain the Canny edge map; Based on the Canny edge map, the Hough circle detection algorithm is used to generate the initial position parameters of the circular hole; Alternatively, a pre-trained convolutional neural network semantic segmentation algorithm can be used to perform pixel-level prediction on the local image to obtain a mask for the charging port circular hole region, and the corresponding edge points can be extracted from the mask as the initial position parameters of the circular hole. If it is determined that the accuracy of the initial position parameter does not meet the preset annotation requirements, a point selection operation is performed on the outer contour of the circular hole on the Canny edge map, and each click position is automatically snapped to the effective Canny edge point with the nearest Euclidean distance to form a set of correction points; Based on the edge points corresponding to the correction point set and the initial position parameters, the ellipse is fitted by the least squares method to obtain the annotation result of the charging port circular hole.
[0008] Furthermore, after obtaining the annotation results of the charging port circular hole, the method also includes: calculating the homography matrix between the annotation results and the standard charging port circular hole plane; The homography matrix is used to back-project the plane of the standard charging port circular hole onto the vehicle charging port image to generate a back-projected contour. Calculate the reprojection error between the back-projection profile and the annotation result of the charging port circular hole; Based on the reprojection error, the annotation result of the charging port circular hole is iteratively corrected using the least squares method or the weighted optimization method, and the optimized annotation result of the charging port circular hole is output.
[0009] Furthermore, the reprojection error is the sum of the squares of the algebraic or geometric distances from the sampling points on the backprojection contour to the annotation results of the charging port circular hole.
[0010] Furthermore, the weighted optimization method assigns different weights to the error term based on the gradient strength of the edge points.
[0011] Furthermore, the selection operation is performed by the user, and the number of selections is 10 to 15, which are evenly distributed in different positions on the outer contour of the circular hole.
[0012] Furthermore, the convolutional neural network semantic segmentation algorithm adopts DeepLabV3, U-Net, or HRNet architecture and is trained on a dataset containing no less than 3000 frames of labeled charging port local images. The average intersection-union ratio (mIOU) of the trained convolutional neural network semantic segmentation algorithm on the validation set is no less than 0.95.
[0013] Furthermore, the marking results of the charging port circular hole are saved in mask format and / or polygon format.
[0014] Furthermore, after obtaining the initial position parameters of the circular hole, the following steps are also included: generating and displaying a visualization image that includes the Canny edge map and the initial position parameters superimposed; and presenting it to the user through a graphical user interface so that the user can judge whether the accuracy of the initial position parameters meets the preset annotation requirements.
[0015] Secondly, this application discloses a vehicle charging port image annotation device, which includes: The acquisition module is used to acquire images of the vehicle's charging port. The local selection processing module is used to perform local selection processing on the vehicle charging port image to obtain a local image including at least one charging port circular hole. The preprocessing module is used to sequentially perform grayscale processing, contrast-limited adaptive histogram equalization processing, and median filtering processing on the local image to obtain a denoised image; The edge detection module is used to detect edges in the denoised image using the Canny edge detection algorithm to obtain the Canny edge map; The generation module is used to generate the initial position parameters of the circular hole based on the Canny edge map using the Hough circle detection algorithm; or it is directly connected to the image preprocessing module and is used to perform pixel-level prediction on the local image using a pre-trained convolutional neural network semantic segmentation algorithm to obtain the mask of the charging port circular hole region, and extract the corresponding edge points from the mask as the initial position parameters of the circular hole. The interactive correction module is used to perform a point selection operation on the outer contour of the circular hole on the Canny edge map when the accuracy of the initial position parameter does not meet the preset annotation requirements. The module automatically snaps the clicked position to the effective Canny edge point with the nearest Euclidean distance to form a correction point set. The fitting module is used to fit an ellipse using the least squares method based on the edge points corresponding to the correction point set and the initial position parameters, and obtain the annotation result of the charging port circular hole.
[0016] Furthermore, it also includes an optimization module for calculating the homography matrix between the annotation result and the standard charging port circular hole plane; using the homography matrix to back-project the standard charging port circular hole plane onto the vehicle charging port image to generate a back-projection contour; calculating the reprojection error between the back-projection contour and the annotation result of the charging port circular hole; and based on the reprojection error, using the least squares method or a weighted optimization method to iteratively correct the annotation result of the charging port circular hole, and outputting the optimized annotation result of the charging port circular hole.
[0017] The present invention has the following unexpected beneficial effects: 1. The annotation method described in this invention significantly reduces manual operation time through automatic preprocessing, edge detection, and intelligent positioning. The annotation time for a single frame is reduced from 5 minutes using traditional methods to less than 2 minutes, making it particularly suitable for the rapid construction of large-scale datasets. Users only need to perform a few point selection and correction operations, and the system can automatically snap to the optimal edge points to form a correction point set, simplifying the annotation process for complex graphics. Furthermore, based on the correction point set and the edge points corresponding to the initial position parameters, an ellipse is fitted using the least squares method to obtain the annotation result for the charging port's circular hole. Compared to the traditional method of manually outlining polygons, this improves the accuracy and geometric consistency of the annotation. Thus, while ensuring high efficiency, it also ensures the precision of the annotation results, making it suitable for applications with extremely high accuracy requirements.
[0018] 2. The annotation results obtained by this invention can be used to train a convolutional neural network semantic segmentation algorithm. As the number of annotations increases, the convolutional neural network model used for generating initial position parameters can be continuously trained and optimized, thereby achieving self-improvement of the annotation tool's performance, including but not limited to increased efficiency and enhanced accuracy. Accumulated data is used to improve algorithm performance, enabling each iteration to better adapt to different imaging conditions and charging port types. Therefore, regardless of adverse conditions such as uneven illumination, noise interference, or local occlusion, this invention can provide stable and reliable annotation results, demonstrating its adaptability to various complex environments.
[0019] 3. This invention ensures annotation quality by projecting the standard charging port circular hole plane onto the image space and comparing it with the annotation results, and allows for necessary adjustments based on error feedback, thus guaranteeing the reliability of the final output. Attached Figure Description
[0020] Figure 1 A flowchart illustrating a vehicle charging port image annotation method provided in an embodiment of this application is shown.
[0021] Figure 2 A flowchart illustrating a vehicle charging port image annotation method according to another embodiment of this application is shown.
[0022] Figure 3 A schematic diagram of the physical process of a vehicle charging port image annotation method provided in another embodiment of this application is shown.
[0023] Figure 4 A schematic diagram of the structure of a vehicle charging port image annotation device provided in an embodiment of this application is shown.
[0024] Figure 5 A schematic diagram of the structure of a vehicle charging port image annotation device provided in another embodiment of this application is shown. Detailed Implementation
[0025] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.
[0026] In one embodiment, see Figure 1 and Figure 3 As shown, this application provides a method for annotating vehicle charging port images, which includes: An image of the vehicle charging port is acquired, and a local bounding box is performed on the image to obtain a partial image including at least one circular hole of the charging port. By locking the circular hole area of the charging port through local bounding box selection, the indiscriminate processing of the entire image is avoided, which greatly reduces the computational load and irrelevant interference of subsequent algorithms and lays the foundation for efficient processing.
[0027] The local image is sequentially processed by grayscale conversion, contrast-limited adaptive histogram equalization (CLAHE), and median filtering to obtain a denoised image. Grayscale conversion unifies the dimension, CLAHE addresses uneven illumination, and median filtering removes noise. These three preprocessing steps make the edge features of the charging port's circular hole clearer and more stable, providing a high-quality data foundation for subsequent localization and fitting.
[0028] The Canny edge detection algorithm is used to perform edge detection on the denoised image, resulting in a Canny edge map. The Canny edge detection algorithm can accurately extract the edges of circular holes, and can still capture effective edge features even in partially occluded scenes, overcoming the shortcomings of traditional computer vision methods that are easily affected by environmental factors.
[0029] Based on the Canny edge map, the Hough circle detection algorithm is used to generate the initial position parameters of the circular hole; alternatively, a pre-trained convolutional neural network semantic segmentation algorithm is used to perform pixel-level prediction on the local image to obtain a mask of the charging port circular hole region, and the corresponding edge points are extracted from the mask as the initial position parameters of the circular hole. When there is insufficient labeled data initially, Hough circle detection can be used to quickly generate initial localization. As the number of labels increases, the convolutional neural network model used for generating initial position parameters can be continuously trained and optimized, thereby achieving self-improvement of the labeling tool's performance. After training the convolutional neural network semantic segmentation algorithm, it can be switched to an automatic localization mode with pixel-level prediction. The localization accuracy and efficiency continuously optimize with the increase in data volume, forming a self-learning closed loop of data accumulation, model iteration, and efficiency / accuracy improvement. Accumulated data is used to improve algorithm performance, enabling each iteration to better adapt to different imaging conditions and charging port types. Therefore, regardless of unfavorable conditions such as uneven illumination, noise interference, or partial occlusion, this application can provide stable and reliable labeling results, demonstrating its adaptability in various complex environments.
[0030] If the accuracy of the initial position parameters does not meet the preset annotation requirements, a point selection operation is performed on the outer contour of the circular hole on the Canny edge map. Each click position is automatically snapped to the nearest valid Canny edge point in Euclidean distance, forming a correction point set. That is, when the initial accuracy is not up to standard, the point selection operation can automatically snap to the nearest valid Canny edge point without precise aiming at the edge. Correction can be completed with only 10 to 15 points, which reduces the difficulty of manual operation and the number of point selections. Compared with traditional manual polygon drawing (5 minutes per frame), the efficiency is improved by more than 2 times (≤2 minutes per frame), which can support the rapid construction of large-scale training datasets.
[0031] Based on the edge points corresponding to the calibration point set and initial position parameters, an ellipse is fitted using the least squares method to obtain the annotation result of the charging port circular hole. The ellipse fitting annotation result not only conforms to the physical structure of the charging port circular hole, but also provides accurate contour boundaries and position parameters. The generated mask and polygon format annotation files can be directly adapted to the training requirements of subsequent deep learning segmentation and pose estimation algorithms without additional format conversion or data correction.
[0032] In summary, this application combines the efficiency of automatic algorithms with the accuracy of manual correction through a human-machine collaborative core design. It also incorporates preprocessing optimization, geometric fitting, and self-learning iteration techniques to comprehensively address the shortcomings of traditional annotation methods in terms of efficiency, accuracy, robustness, and ease of use. This provides large-scale, high-precision training data support for monocular perception algorithms of automatic charging robots, promoting the industrialization of related technologies.
[0033] As a preferred embodiment of this application, see [link to application]. Figure 2 As shown, after obtaining the annotation result of the charging port circular hole, the process also includes: calculating the homography matrix between the annotation result and the standard charging port circular hole plane. The standard charging port circular hole plane is an ideal template based on national standard geometric features. This template contains several circular holes (such as main positioning holes, auxiliary communication holes, etc.) with known positions and dimensions. Its coordinate system origin is usually set at the center of the main charging hole, and the relative positions and diameters of each hole strictly follow the national standard definition.
[0034] The homography matrix is used to back-project the standard charging port circular hole plane onto the vehicle charging port image. All the circular hole boundaries in the standard charging port circular hole plane are back-projected onto the vehicle charging port image plane through the homography matrix to obtain a theoretically ideal set of contour points, i.e., the back-projected contour.
[0035] Calculate the reprojection error between the back-projection profile and the annotation result of the charging port circular hole.
[0036] Based on the reprojection error, the annotation results of the charging port circular hole are iteratively corrected using the least squares method or a weighted optimization method, outputting the optimized annotation results. Specifically, with minimizing the reprojection error as the objective function, a nonlinear least squares method (such as the Levenberg-Marquardt algorithm) or a weighted optimization strategy (assigning lower weights to occluded areas) is used to fine-tune the original ellipse parameters, outputting the optimized high-precision annotation results. The original ellipse parameters include the center coordinates, major axis, minor axis, and rotation angle.
[0037] While ellipse fitting in the preceding process already achieves accuracy superior to manual polygon outlining, fitting errors may still exist due to image noise and edge extraction deviations. In this preferred embodiment, the homography matrix establishes a projection mapping relationship between the annotation results and the standard charging port circular hole plane. The back-projection contour provides an objective and unified accuracy reference for the annotation results; the reprojection error quantifies the degree of deviation between the actual annotation and the ideal template, making the correction process based on evidence. On this basis, iterative correction using the least squares method or weighted optimization method can specifically offset the systematic errors in the basic fitting, making the annotation results more closely match the physical geometric features of the charging port circular hole, further improving sub-pixel accuracy, and fully meeting the stringent requirements of monocular perception algorithms for high-precision training data.
[0038] In a preferred embodiment of this application, the reprojection error is the sum of the squares of the algebraic or geometric distances from the sampling points on the backprojection contour to the annotation results of the charging port circular hole.
[0039] Specifically, for each sampling point on the back-projected contour, calculate its algebraic or geometric distance to the currently labeled ellipse. Sum the squared distances of all sampling points, and define the reprojection error as the sum of these squared distances.
[0040] The annotation results and back-projected contours of the charging port circular hole are both geometric contours. The sum of squared distances is used as an error index to directly quantify the spatial geometric deviation between the actual annotation and the ideal template. Among them, the algebraic distance can quickly reflect the overall offset trend of the contour, while the geometric distance accurately corresponds to the actual physical deviation at the pixel level, avoiding the optimization direction deviation caused by subjective evaluation or fuzzy quantization.
[0041] In a preferred embodiment of this application, the weighted optimization method assigns different weights to the error term based on the gradient strength of the edge points.
[0042] The weighted optimization method specifically includes: calculating the gradient magnitude of each pixel on the original local image to generate a gradient intensity map. This map reflects the sharpness of edges in the image: high gradient regions correspond to clear, high-contrast true edges; low gradient regions may be affected by noise, blur, or occlusion. Then, for each sampling point on the backprojection contour, its corresponding weight is defined based on its value in the gradient intensity map. The original sum of squared reprojection errors is then rewritten in a weighted form, and weighted least squares optimization is performed to iteratively update the ellipse parameters, finally outputting the optimized high-confidence annotation result.
[0043] This preferred implementation introduces a gradient-strength-based weighted optimization mechanism, which essentially emphasizes effective features and weakens interfering features during the optimization process. Specifically, by assigning high weights to edge points with strong gradients, their corresponding error terms account for a larger proportion in the optimization, guiding the annotation results towards a more realistic and clear contour; conversely, by assigning low weights to edge points with weak gradients, their interference with the overall optimization results is reduced, avoiding annotation offsets caused by unreliable points.
[0044] In a preferred embodiment of this application, the selection operation is performed by the user, and the number of selections is 10 to 15, which are evenly distributed in different positions on the outer contour of the circular hole.
[0045] This preferred embodiment allows the user to select more points, thus capturing the actual outline of the circular hole more accurately. Compared to using only a few points, such as 3 or 4, it provides a more detailed shape description. Furthermore, points evenly distributed in different orientations help the algorithm better understand and adapt to the geometric characteristics of the circular hole, maintaining high robustness even in the face of partial occlusion, image noise, or blurred edges. Although it requires the user to click more points, because these points are required to be evenly distributed on the outer contour, it effectively provides the user with clear operational guidance, reducing the difficulty of operation and minimizing errors caused by subjective judgment.
[0046] As a preferred embodiment of this application, the convolutional neural network semantic segmentation algorithm adopts DeepLabV3, U-Net or HRNet architecture, and is trained on a dataset containing no less than 3000 frames of labeled charging port local images. The average intersection-union ratio (mIOU) of the trained convolutional neural network semantic segmentation algorithm on the validation set is no less than 0.95.
[0047] DeepLabV3 effectively captures the scale changes of the charging port's circular hole at different distances and angles through dilated convolution and multi-scale feature fusion, avoiding segmentation deviations caused by differences in target size. U-Net's encoder-decoder structure and skip connection design excel at preserving target edge details, perfectly matching the sub-pixel-level edge accuracy annotation requirements of the circular hole, and is especially suitable for scenes with partial occlusion and blurred edges. HRNet, by continuously maintaining high-resolution feature maps, can accurately distinguish the boundaries between the circular hole and background texture and noise, reducing missed or false detections in segmentation. The common advantages of these three methods are high segmentation accuracy for small targets, strong anti-interference ability, and a high degree of fit with the geometric features of the charging port's circular hole and the actual annotation scene, ensuring the reliability of automatic pre-positioning from the algorithmic level.
[0048] The annotation of circular charging port holes presents diverse variables such as uneven lighting, angle variations, partial occlusion, and noise interference. Models trained on only a small amount of data are prone to overfitting, such as only being able to identify circular holes under specific lighting conditions. A local image dataset of at least 3000 frames can cover the feature distribution of the above-mentioned diverse scenarios, allowing the model to learn the common features of circular holes under different working conditions, significantly improving its generalization ability—even when faced with complex, unseen scenes, it can consistently output accurate segmentation results, avoiding the pain point of poor scene adaptability of traditional computer vision methods.
[0049] Mean Intersection over Union (mIOU) is a core quality indicator for semantic segmentation tasks, directly reflecting the degree of overlap between the model's segmentation results and the real target region. An mIOU ≥ 0.95 means that the generated circular hole mask overlaps with the real contour by over 95%, equivalent to automatic pre-positioning results approaching the level of accurate human annotation. For subsequent annotation processes, high mIOU segmentation results can be directly used as initial position parameters without manual correction, significantly reducing the frequency of manual point selection and further improving annotation efficiency. Even if minor corrections are needed, only fine-tuning of edge points is required, avoiding excessive initial positioning deviations that lead to ineffective manual re-annotation and maximizing the advantages of human-machine collaboration.
[0050] In a preferred embodiment of this application, the marking results of the charging port circular hole are saved in mask format and / or polygon format.
[0051] The mask format records the target region in a pixel-by-pixel binarization manner, with the circular hole region representing the foreground pixels and the background region representing the background pixels, perfectly matching the input requirements of semantic segmentation algorithms. Convolutional neural network semantic segmentation algorithms need to learn the distinguishing features between the target and the background through pixel-level annotation. The mask format can be directly used as label data for model training without additional format conversion or feature extraction, ensuring that the annotation accuracy is fully transferred to the algorithm training process and avoiding accuracy loss due to format conversion.
[0052] The polygon format records the boundary of the circular hole using a set of contour vertex coordinates, precisely meeting the pose estimation requirements of monocular perception algorithms in automated charging robots. Pose estimation requires calculating the spatial position and orientation using the geometric parameters of the target contour. The polygon format directly provides contour vertex data, facilitating the rapid parsing of the geometric features of the circular hole without the need to extract the contour from a mask, thus reducing the complexity of algorithm preprocessing. Simultaneously, this format can also adapt to the bounding box and contour supplementation annotation requirements in object detection algorithms, broadening the application scenarios of the data.
[0053] In a preferred embodiment of this application, after obtaining the initial position parameters of the circular hole, the following steps are further included: generating and displaying a visualization image that includes the Canny edge map and the initial position parameters superimposed; and presenting it to the user through a graphical user interface so that the user can judge whether the accuracy of the initial position parameters meets the preset annotation requirements.
[0054] This preferred embodiment overlays the Canny edge map with the initial position parameters to show the overlap between the automatic positioning result and the actual edge. This eliminates the need for users to switch between multiple images for comparison, allowing them to intuitively observe whether the initial contour fits the edge, and whether there are any offsets, missed detections, or false detections. This accurately identifies accuracy shortcomings and avoids misjudgments or overcorrections caused by fragmented information. Furthermore, since the Canny edge map has been pre-processed to enhance the edges of circular holes and filter noise, the overlay allows users to clearly distinguish between valid edges and interfering noise, avoiding judgment errors caused by uneven lighting or complex backgrounds in the original image. This makes the accuracy judgment more aligned with actual annotation needs.
[0055] In one embodiment, see Figure 4 As shown, this application discloses a vehicle charging port image annotation device. The annotation device 100 includes an acquisition module 10, a local bounding box processing module 20, a preprocessing module 30, an edge detection module 40, a generation module 50, an interactive correction module 60, and a fitting module 70.
[0056] The acquisition module 10 is used to acquire images of the vehicle's charging port.
[0057] The local selection processing module 20 is used to perform local selection processing on the vehicle charging port image to obtain a local image including at least one charging port circular hole.
[0058] The preprocessing module 30 is used to sequentially perform grayscale processing, contrast-limited adaptive histogram equalization processing, and median filtering processing on the local image to obtain a denoised image.
[0059] The edge detection module 40 is used to perform edge detection on the denoised image using the Canny edge detection algorithm to obtain a Canny edge map.
[0060] The generation module 50 is used to generate the initial position parameters of the circular hole based on the Canny edge map using the Hough circle detection algorithm; or it is directly connected to the image preprocessing module and is used to perform pixel-level prediction on the local image using a pre-trained convolutional neural network semantic segmentation algorithm to obtain the mask of the charging port circular hole region, and extract the corresponding edge points from the mask as the initial position parameters of the circular hole. The interactive correction module 60 is used to perform a point selection operation on the outer contour of the circular hole on the Canny edge map when it is determined that the accuracy of the initial position parameter does not meet the preset annotation requirements. The click position is automatically snapped to the effective Canny edge point with the nearest Euclidean distance to form a correction point set. The fitting module 70 is used to fit an ellipse using the least squares method based on the edge points corresponding to the correction point set and the initial position parameters, to obtain the annotation result of the charging port circular hole.
[0061] As a preferred embodiment of this application, see [link to application]. Figure 5 As shown, the vehicle charging port image annotation device 100 provided in this application embodiment further includes an optimization module 80, which is used to calculate the homography matrix between the annotation result and the standard charging port circular hole plane; use the homography matrix to back-project the standard charging port circular hole plane onto the vehicle charging port image to generate a back-projection contour; calculate the reprojection error between the back-projection contour and the annotation result of the charging port circular hole; based on the reprojection error, use the least squares method or weighted optimization method to iteratively correct the annotation result of the charging port circular hole, and output the optimized annotation result of the charging port circular hole.
[0062] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention. Furthermore, the embodiments of the present invention and the features thereof can be combined with each other unless otherwise specified.
Claims
1. A method for annotating vehicle charging port images, characterized in that, include: Acquire an image of the vehicle charging port, and perform partial selection processing on the vehicle charging port image to obtain a partial image including at least one circular hole of the charging port. The local image is sequentially processed by grayscale conversion, contrast-limited adaptive histogram equalization, and median filtering to obtain a denoised image. The Canny edge detection algorithm is used to detect edges in the denoised image to obtain the Canny edge map; Based on the Canny edge map, the Hough circle detection algorithm is used to generate the initial position parameters of the circular hole; or, a pre-trained convolutional neural network semantic segmentation algorithm is used to perform pixel-level prediction on the local image to obtain the mask of the charging port circular hole region, and the corresponding edge points are extracted from the mask as the initial position parameters of the circular hole. If it is determined that the accuracy of the initial position parameter does not meet the preset annotation requirements, a point selection operation is performed on the outer contour of the circular hole on the Canny edge map, and each click position is automatically snapped to the effective Canny edge point with the nearest Euclidean distance to form a set of correction points; Based on the edge points corresponding to the correction point set and the initial position parameters, the ellipse is fitted by the least squares method to obtain the annotation result of the charging port circular hole.
2. The vehicle charging port image annotation method according to claim 1, characterized in that: After obtaining the annotation results of the charging port circular hole, the process also includes: calculating the homography matrix between the annotation results and the standard charging port circular hole plane; The homography matrix is used to back-project the plane of the standard charging port circular hole onto the vehicle charging port image to generate a back-projected contour. Calculate the reprojection error between the back-projection profile and the annotation result of the charging port circular hole; Based on the reprojection error, the annotation result of the charging port circular hole is iteratively corrected using the least squares method or the weighted optimization method, and the optimized annotation result of the charging port circular hole is output.
3. The vehicle charging port image annotation method according to claim 2, characterized in that: The reprojection error is the sum of the squares of the algebraic or geometric distances from the sampling points on the backprojection contour to the annotation results of the charging port circular hole.
4. The vehicle charging port image annotation method according to claim 2, characterized in that: The weighted optimization method assigns different weights to the error term based on the gradient strength of the edge points.
5. The vehicle charging port image annotation method according to claim 1, characterized in that: The selection operation is performed by the user, and the number of selections is 10 to 15, which are evenly distributed in different positions on the outer contour of the circular hole.
6. The vehicle charging port image annotation method according to claim 1, characterized in that: The convolutional neural network semantic segmentation algorithm adopts DeepLabV3, U-Net or HRNet architecture and is trained on a dataset containing no less than 3000 frames of labeled charging port local images. The average intersection-union ratio (mIOU) of the trained convolutional neural network semantic segmentation algorithm on the validation set is no less than 0.
95.
7. The vehicle charging port image annotation method according to claim 1, characterized in that: The marking results of the charging port circular hole are saved in mask format and / or polygon format.
8. The vehicle charging port image annotation method according to claim 1, characterized in that: After obtaining the initial position parameters of the circular hole, the following steps are also included: generating and displaying a visualization image that includes the Canny edge map and the initial position parameters superimposed; and presenting it to the user through a graphical user interface so that the user can judge whether the accuracy of the initial position parameters meets the preset annotation requirements.
9. A vehicle charging port image annotation device, characterized in that, include: The acquisition module is used to acquire images of the vehicle's charging port. The local selection processing module is used to perform local selection processing on the vehicle charging port image to obtain a local image including at least one charging port circular hole. The preprocessing module is used to sequentially perform grayscale processing, contrast-limited adaptive histogram equalization processing, and median filtering processing on the local image to obtain a denoised image. The edge detection module is used to detect edges in the denoised image using the Canny edge detection algorithm to obtain the Canny edge map; The generation module is used to generate the initial position parameters of the circular hole based on the Canny edge map using the Hough circle detection algorithm; or it is directly connected to the image preprocessing module and is used to perform pixel-level prediction on the local image using a pre-trained convolutional neural network semantic segmentation algorithm to obtain the mask of the charging port circular hole region, and extract the corresponding edge points from the mask as the initial position parameters of the circular hole. The interactive correction module is used to perform a point selection operation on the outer contour of the circular hole on the Canny edge map when the accuracy of the initial position parameter does not meet the preset annotation requirements. The click position is automatically snapped to the effective Canny edge point with the nearest Euclidean distance to form a correction point set. The fitting module is used to fit an ellipse using the least squares method based on the edge points corresponding to the correction point set and the initial position parameters, and obtain the annotation result of the charging port circular hole.
10. The vehicle charging port image annotation device according to claim 9, characterized in that: It also includes an optimization module for calculating the homography matrix between the annotation result and the standard charging port circular hole plane; using the homography matrix to back-project the standard charging port circular hole plane onto the vehicle charging port image to generate a back-projection contour; calculating the reprojection error between the back-projection contour and the annotation result of the charging port circular hole; and based on the reprojection error, using the least squares method or a weighted optimization method to iteratively correct the annotation result of the charging port circular hole, and outputting the optimized annotation result of the charging port circular hole.