Vehicle charging cover automatic positioning and charging method and system based on cascade tracking

Abstract

The application relates to the technical field of automatic control of vehicle charging, and discloses a vehicle charging cover automatic positioning and charging method and system based on cascaded tracking, which can reduce the search range of the charging cover, avoid large-scale scanning of a mechanical arm, and improve detection efficiency by fusing the poses of multiple cameras, tracking the states, dynamically generating a charging cover constraint ROI in combination with vehicle structure prior, and further performing coarse detection on the charging cover in the ROI and starting re-detection according to the vehicle motion state when the charging cover is not detected, so that the continuity and stability of charging cover positioning can be improved; after the coarse positioning result is obtained, visual closed-loop fine positioning is performed through a visual sensor at the end of the mechanical arm, so that the cumulative error of the mechanical arm and the slight deviation of the vehicle can be corrected, and the accuracy and reliability of the cover opening, gun inserting and charging operations can be improved.

Description

Technical Field

[0001] This invention relates to the technical field of automatic vehicle charging control, specifically to a method and system for automatic positioning and charging of vehicle charging covers based on cascade tracking. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of autonomous driving technology and the new energy vehicle industry, the demand for new energy vehicles with autonomous driving capabilities in scenarios such as public charging, logistics fleet refueling, and unmanned shuttle operations is constantly increasing. The level of automation and intelligence in the charging process directly affects vehicle refueling efficiency, operating costs, and charging safety. The charging cover, as the external protective structure of the charging interface, is usually closed before charging and is a key component that needs to be identified and located first in the automatic charging process. However, the charging cover is usually embedded in the vehicle body surface, is small in size, has inconspicuous appearance features, and its color and texture are similar to the vehicle body surface; the placement, shape, opening direction, and relative relationship between the charging cover and vehicle body structural components also vary among different vehicle models. Therefore, compared to larger targets such as the vehicle body and wheels, the charging cover is more susceptible to factors such as changes in vehicle posture, differences in camera angles, partial occlusion, changes in lighting, and vehicle body reflections, resulting in higher difficulty in detection and location.

[0004] Current automatic charging solutions typically rely on a camera at the end of a charging robotic arm to progressively scan the vehicle's exterior surface and identify and locate the charging cover based on a single visual image. This search over a large area results in a long scanning path for the robotic arm, low detection efficiency, and is prone to missed detections, false detections, or location jumps when the charging cover is small, image features are unstable, or detection results are lost briefly. Furthermore, during automatic charging, the vehicle may still be moving or slightly offset before charging, and the robotic arm may accumulate errors during movement and docking. Relying solely on a single detection result or a single-end visual positioning result can easily lead to deviations in the opening and insertion positions of the charging cover, thus affecting the accuracy and safety of the automatic charging operation. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes an automatic positioning and charging method and system for vehicle charging covers based on cascade tracking, thereby improving the stability of charging cover detection and positioning, the accuracy of robotic arm docking, and the safety of the automatic charging process.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: One or more embodiments provide an automatic positioning and charging method for vehicle charging covers based on cascade tracking, including the following steps: Multi-camera pose fusion and multi-target tracking are performed on the target vehicle to obtain the target vehicle's global pose information and tracking status information; Based on the vehicle pose of the target vehicle and the pre-established vehicle structure priors, the charging cover constraint region of interest is generated and dynamically updated. A coarse detection of the charging cover is performed within the region of interest; when the charging cover is detected, the location of the charging cover is obtained; if the charging cover is not detected, a re-detection is initiated based on the vehicle's motion state until the charging cover is detected and the location of the charging cover is obtained. Based on the coarse detection of the charging cover's location, the robotic arm is driven to move to the charging cover area. The end vision sensor of the robotic arm is used to perform visual closed-loop fine positioning of the charging cover. After the charging cover's detection confidence and positioning accuracy meet the preset conditions, the robotic arm is controlled to perform the charging process.

[0007] A further technical solution involves detecting and tracking the target vehicle to obtain its pose and motion state, including the following steps: Image data collected from the target vehicle is acquired. The original RGB image is enhanced using a joint enhancement network to obtain an enhanced RGB image. The original depth image is then processed by median filtering and morphological closing operations to obtain a D-RGB image that is pixel-aligned with the RGB image. The enhanced RGB image is input into the pre-trained YOLOv26_EVCar vehicle and charging cover detection model to obtain the detected vehicle target; based on the D-RGB image, the 6D pose of the detected vehicle target is obtained through the pose regression network, and the 6D pose matrix of the vehicle is output to obtain the target vehicle detection data. Multi-target tracking is performed based on the target vehicle detection data to obtain the vehicle's global pose information and tracking status information.

[0008] Further technical solutions, including methods for multi-target tracking based on target vehicle detection data, include: The 6D vehicle pose matrix detected by the fixed-position camera is converted into a pose matrix in the world coordinate system. , Based on the translation vector of the pose matrix, it is determined whether it is the same vehicle pose. The poses of the same vehicle are weighted and fused to obtain global pose information. Based on the target vehicle confidence score and feature similarity score detected by the fixed camera, camera attention weights are calculated, and the visual features of the same target vehicle are weighted and fused to obtain the target feature vector. It can assign a unique tracking ID to the same target vehicle under global image information; Based on the fused target feature vector, global pose information, and two-dimensional detection box, a global tracking state dictionary is established to obtain and maintain the tracking state information of the target vehicle.

[0009] A further technical solution, based on the vehicle pose of the target vehicle and pre-established vehicle structure priors, generates and dynamically updates the charging cover constraint region of interest, including the following steps: Based on the global pose information of the target vehicle, the prior key points of the vehicle structure are transformed into the world coordinate system; Project the structural key points in the world coordinate system onto the image pixel coordinate system; The charging cover constrains the region of interest based on the projected structural key points.

[0010] A further technical solution involves coarsely detecting the charging cover within the constrained region of interest. This coarse detection is performed using a trained YOLOv26_EVCar detection model. If the target vehicle has come to a complete stop, the target vehicle features are matched with the vehicle models in a pre-built vehicle structure prior library. If the match is successful, the position of the charging cover is predicted by combining historical valid frames with the fixed transformation matrix of the charging cover relative to the vehicle body. If the match fails, the system enters a standby state and requests manual intervention.

[0011] A further technical solution, after the coarse detection of the charging cover's location, also includes: performing depth edge fitting and pixel-level charging cover positioning based on the coarsely detected charging cover location, and combining temporal tracking smoothing and Kalman filtering to achieve continuous positioning and tracking of the charging cover.

[0012] A further technical solution utilizes a visual sensor at the end of the robotic arm to perform visual closed-loop precise positioning of the charging cover. Specifically, the depth camera at the end of the robotic arm scans the area of ​​the charging cover, and a trained YOLOv26_EVCar vehicle and charging cover detection model combined with an edge fitting positioning algorithm is used to detect and accurately locate the charging cover. The deviation between the target position of the charging cover and the current end pose of the robotic arm is compared in real time, and the cumulative motion error of the robotic arm and the micro-movement deviation of the vehicle are corrected through visual servo control, thereby adjusting the end pose of the robotic arm in real time.

[0013] One or more embodiments provide an automatic positioning and charging system for vehicle charging covers based on cascade tracking, including: The multi-target tracking module is configured to perform multi-camera pose fusion and multi-target tracking on the target vehicle to obtain the target vehicle's global pose information and tracking status information. The region of interest identification module is configured to generate and dynamically update the charging cover constrained region of interest based on the vehicle pose of the target vehicle and the pre-established vehicle structure prior. The coarse detection module is configured to perform coarse detection of the charging cover within the region of interest; when the charging cover is detected, the charging cover positioning result is obtained; when the charging cover is not detected, re-detection is initiated according to the vehicle's motion state until the charging cover is detected and the charging cover positioning position is obtained. The precision positioning charging module is configured to locate the charging cover based on coarse detection, drive the robotic arm to move to the charging cover area, and use the vision sensor at the end of the robotic arm to perform visual closed-loop precision positioning of the charging cover. After the detection confidence and positioning accuracy of the charging cover meet the preset conditions, the robotic arm is controlled to perform the charging process.

[0014] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, complete the steps in the above-described method for automatic positioning and charging of a vehicle charging cover based on cascading tracking.

[0015] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the above-described method for automatic positioning and charging of a vehicle charging cover based on cascading tracking.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: The automatic positioning and charging method for charging covers of this invention achieves continuous positioning of the charging cover from coarse to fine through a cascaded processing flow of vehicle detection and tracking—generation of vehicle structure prior constraint ROI—detection / re-detection of charging covers within the ROI—end-of-arm visual closed-loop fine positioning. By obtaining the vehicle's global pose and tracking state through multi-camera pose fusion and multi-target tracking, and combining this with the dynamic generation of charging cover constraint ROIs based on vehicle structure priors, the search range for the charging cover can be reduced, avoiding large-scale scanning by the robotic arm and improving detection efficiency. Furthermore, performing coarse detection of the charging cover within the ROI and initiating re-detection based on the vehicle's motion state when no charging cover is detected improves the continuity and stability of the charging cover positioning. After obtaining the coarse positioning result, visual closed-loop fine positioning is performed using a visual sensor at the end of the robotic arm, which can correct for cumulative errors of the robotic arm and minor vehicle offsets, improving the accuracy and reliability of opening the cover, inserting the charging gun, and charging operations.

[0017] The advantages of the present invention, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0019] Fig. 1This is a flowchart of the automatic positioning and charging method for the vehicle charging cover according to Embodiment 1 of the present invention; Fig. 2 This is a flowchart illustrating an implementation example of the automatic positioning and charging method for a vehicle charging cover according to Embodiment 1 of the present invention. Fig. 3 This is a schematic diagram of the PredLink-MOT multi-target tracking framework of Embodiment 1 of the present invention; Detailed Implementation The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] It should be noted that the terminology used herein is for describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0022] Example 1 In one or more of the technical solutions disclosed in the embodiments, such as Figs. 1 to 3 As shown, an automatic positioning and charging method for vehicle charging covers based on cascade tracking includes the following steps: Step 1: Perform multi-camera pose fusion and multi-target tracking on the target vehicle to obtain the global pose information and tracking status information of the target vehicle; Step 2: Based on the vehicle pose of the target vehicle and the pre-established vehicle structure priors, generate and dynamically update the charging cover constrained region of interest (ROI). Step 3: Perform coarse detection of the charging cover within the region of interest; when the charging cover is detected, obtain the charging cover positioning result; if the charging cover is not detected, start re-detection according to the vehicle's movement status until the charging cover is detected and the charging cover positioning position is obtained. Step 4: Based on the coarse detection of the charging cover's positioning position, drive the robotic arm to move to the charging cover area, and use the end vision sensor of the robotic arm to perform visual closed-loop fine positioning of the charging cover. After the charging cover's detection confidence and positioning accuracy meet the preset conditions, control the robotic arm to perform the charging process. The process of controlling the robotic arm to perform the charging process can include opening the cover, inserting the charging gun and charging. During the charging process, the vehicle's position and posture are continuously monitored. When the vehicle deviates beyond the safety threshold, the charging robotic arm is controlled to stop charging and perform the gun removal operation. This embodiment of the automatic vehicle charging cover localization and charging method based on cascaded tracking achieves continuous localization of the charging cover from coarse to fine through a cascaded processing flow of vehicle detection and tracking—generation of vehicle structure prior constraint ROI—detection / re-detection of charging cover within the ROI—closed-loop fine localization by the end-effector vision. Specifically, firstly, the global pose information and tracking status information of the target vehicle are obtained through multi-camera pose fusion and multi-target tracking, enabling subsequent charging cover localization to be based on the actual position and attitude of the target vehicle. Then, the charging cover constraint region of interest (ROI) is generated and dynamically updated using vehicle structure priors, limiting the charging cover detection range to candidate areas matching the vehicle structure. This avoids large-scale stepwise scanning of the vehicle's outer surface by the robotic arm's end-effector camera, thereby reducing the search range and improving detection efficiency. Furthermore, coarse detection of the charging cover is performed within the charging cover constraint ROI, and re-detection is initiated based on the vehicle's motion state when no charging cover is detected. This allows for continuous acquisition of the charging cover's location even when the vehicle's attitude changes, detection results are temporarily lost, or the charging cover target is not obvious, improving the continuity and stability of the charging cover detection process. After obtaining the rough detection location of the charging cover, the robotic arm is driven to move to the charging cover area, and visual closed-loop fine positioning is performed using the vision sensor at the end of the robotic arm. This can further correct the cumulative error during the movement of the robotic arm and the slight deviation that the vehicle may produce, so that the confidence level of the charging cover detection and the positioning accuracy meet the requirements of the subsequent robotic arm actions, thereby improving the accuracy and reliability of opening the cover, inserting the charging gun and charging operations.

[0023] Step 1 involves detecting and tracking the target vehicle to obtain its pose and motion state, including the following steps: Step 11: Acquire image data collected for the target vehicle; perform environmental robustness optimization on the original RGB image using a joint enhancement network to obtain the enhanced RGB image; perform median filtering and morphological closing operation on the original depth image to obtain a D-RGB image aligned with the pixels of the RGB image. Step 12, Vehicle Detection: Input the enhanced RGB image into the pre-trained YOLOv26_EVCar vehicle and charging cover detection model to obtain the detected vehicle target; Based on the D-RGB image, obtain the 6D pose of the detected vehicle target through the pose regression network, output the vehicle 6D pose matrix, and obtain the target vehicle detection data. Step 13: Perform multi-target tracking based on the target vehicle detection data to obtain the vehicle's global pose information and tracking status information; The implementation methods for multi-target tracking based on target vehicle detection data include: S131: Convert the 6D vehicle pose matrix output from the fixed-position camera into a pose matrix in the world coordinate system. , Based on the translation vector of the pose matrix, it is determined whether it is the same vehicle pose. The poses of the same vehicle are weighted and fused to obtain global pose information. Step 132: Based on the target vehicle confidence score and feature similarity output from the fixed camera, calculate the camera attention weights and perform weighted fusion of the visual features of the same target vehicle to obtain the target feature vector. To assign a unique tracking ID to the same target vehicle under the global image information; S133: Based on the fused target feature vector, global pose information and two-dimensional detection box, establish a global tracking state dictionary to obtain and maintain the tracking state information of the target vehicle; This embodiment employs multi-camera pose fusion and multi-target tracking for the target vehicle. On one hand, the 6D pose matrices of the vehicle output from different fixed cameras are uniformly transformed to the world coordinate system. Based on positional distance, it is determined whether the detection results from different cameras correspond to the same target vehicle. If the same target vehicle is identified, its pose is weighted and fused, avoiding pose deviations caused by relying solely on a single camera viewpoint and improving the accuracy and robustness of the target vehicle's global pose estimation. On the other hand, camera attention weights are determined based on detection confidence and feature similarity, and the visual semantic features and depth geometric features of the target vehicle are fused. This enhances the target feature representation capability and reduces the impact of occlusion, viewpoint differences, and false detections on target matching. Furthermore, by establishing a global tracking state dictionary, the unique tracking ID, fused feature vector, global pose matrix, 2D detection box, and number of live frames for the target vehicle are uniformly maintained. This enables continuous tracking and state updates of the vehicle target, reducing target identity switching and tracking loss, thereby improving the stability of vehicle detection, pose estimation, and multi-target tracking in complex scenes.

[0024] Automated charging systems typically rely on a single camera or a robotic arm's end-effector to acquire local images of the vehicle, making it difficult to continuously obtain a stable pose of the vehicle in global space. Due to potential approach movements, angular deflections, or short-term occlusions, a single viewpoint can easily lead to unstable vehicle pose estimation, thus affecting subsequent charging cover positioning. This embodiment utilizes multi-camera pose fusion and multi-target tracking to obtain the target vehicle's global pose and tracking status information. This allows subsequent charging cover positioning to be based on the target vehicle's actual spatial position and attitude, improving the stability of vehicle positioning and providing a reliable foundation for charging cover area constraints and subsequent robotic arm movements.

[0025] In step 2, based on the target vehicle's pose and pre-established vehicle structure priors, the charging cover constrained region of interest (ROI) is generated and dynamically updated, including the following steps: Step 21: Based on the global pose information of the target vehicle, transform the prior key points of the vehicle structure to the world coordinate system; Step 22: Project the structural key points in the world coordinate system to the image pixel coordinate system; Step 23: Generate the charging cover constrained region of interest (ROI) based on the projected structural key points.

[0026] Existing methods for detecting charging covers typically require searching within a full vehicle image or a large area of ​​the image. This results in a large detection range and susceptibility to interference from vehicle textures, reflections, logos, and gaps. Because charging covers are small and their appearance features are not readily apparent, full-image detection is prone to missed or false detections. This embodiment generates a Region of Interest (ROI) constrained by the charging cover based on the target vehicle's pose and pre-established vehicle structural priors. This ROI is dynamically updated as the vehicle's pose changes, limiting the charging cover detection range to candidate regions that match the vehicle's structure. This reduces invalid search areas, minimizes background interference, and improves the efficiency and accuracy of charging cover detection.

[0027] In step 3, coarse detection of the charging cover is performed within the region of interest (ROI) constrained by the charging cover. The trained YOLOv26_EVCar detection model is used to perform coarse detection of the charging cover. If the target vehicle has completely stopped, the features of the target vehicle are matched with the vehicle models in the pre-built vehicle structure prior library. If the match is successful, the position of the charging cover is predicted by combining the historical valid frames and the fixed transformation matrix of the charging cover relative to the vehicle body. If the match fails, the system enters a standby state and requests manual intervention.

[0028] Traditional charging cover detection often relies on a robotic arm's end-effector camera for full-vehicle scanning. This requires the robotic arm to move repeatedly to locate the charging cover, resulting in long scanning paths, high detection time, and poor real-time performance. This embodiment performs coarse detection of the charging cover within a dynamically updated ROI (Region of Interest) constraint. This allows for initial positioning of the charging cover before the robotic arm makes large-scale movements, reducing unnecessary scanning and search actions, improving initial positioning efficiency, and providing a foundation for subsequent fine positioning.

[0029] In step 3, re-detection is initiated based on the vehicle's motion state, corresponding to steps S41-S43 in the subsequent steps; specifically: when the target vehicle is in motion, detection continues within the dynamically updated charging cover constraint region of interest; when the target vehicle is stationary, the target vehicle is matched with the vehicle model in the vehicle structure prior library, and after successful matching, the position of the charging cover is predicted based on historical valid frames and the fixed transformation relationship between the charging cover and the vehicle body, and re-detection is performed. In existing charging control schemes, when the charging cover detection fails or the positioning result is lost, the existing system often lacks an effective re-detection mechanism, easily resulting in direct error reporting or manual intervention, leading to insufficient automated continuous operation capability. Especially when the vehicle is not completely stationary, the charging cover position changes with the vehicle's movement, and the single detection result is prone to instability. This embodiment continuously judges the vehicle's movement state based on the global tracking state dictionary when the charging cover is not detected, and initiates re-detection within the ROI; when the vehicle is completely stationary, it further combines the vehicle structure prior library to predict the charging cover position. This allows for continued acquisition of the charging cover positioning position even when detection fails briefly, the vehicle's posture changes, or the target features are not obvious, improving the fault tolerance and continuity of the charging cover detection process.

[0030] Furthermore, after the coarse detection of the charging cover location in step 3, the method further includes: performing depth edge fitting and pixel-level charging cover positioning based on the coarsely detected charging cover location, and combining temporal tracking smoothing and Kalman filtering to achieve continuous positioning and tracking of the charging cover. Coarse detection typically only outputs the detection frame of the charging cover, and its positioning accuracy is significantly affected by regression errors, model errors, and image noise, making it difficult to directly meet the requirements of precise operations such as opening the cover and inserting the charging gun by a robotic arm. This embodiment, after obtaining the charging cover's positioning position through coarse detection, further combines RGB edge information and depth transition edge information to fit the charging cover's edges, obtaining pixel-level positioning results. Furthermore, temporal tracking smoothing and Kalman filtering are used to suppress positioning jitter. This refines the coarse detection results into a stable charging cover positioning position, improving the charging cover's positioning accuracy and continuous tracking stability.

[0031] In step 4, based on the coarse detection of the charging cover's location, the robotic arm is driven to move to the charging cover area, including the following steps: Step 41: Perform a coarse 3D coordinate calculation on the final charging cover position, transform it to the world coordinate system, and transmit the 3D coordinates to the robotic arm that performs the charging. Step 42: Move the end effector of the robotic arm to the vicinity of the charging cover, align the charging gun port and the depth camera at the end effector of the robotic arm with the center point of the charging cover, and complete the coarse positioning of the charging robotic arm.

[0032] In step 4, the charging cover is precisely located using a visual sensor at the end of the robotic arm. Specifically, the charging cover area is scanned by a depth camera at the end of the robotic arm. The charging cover is detected and accurately located using a trained YOLOv26_EVCar vehicle and charging cover detection model combined with an edge fitting localization algorithm. The deviation between the target position of the charging cover and the current end pose of the robotic arm is compared in real time. The cumulative motion error of the robotic arm and the slight deviation of the vehicle are corrected by visual servo control, and the end pose of the robotic arm is adjusted in real time. The robotic arm moves directly to the target position based on the front-end visual detection results. However, cumulative errors may occur during the robotic arm's movement, and the vehicle may also experience slight deviations, resulting in a discrepancy between the actual position reached by the robotic arm's end effector and the charging cover's position. In this embodiment, after the robotic arm moves to the charging cover area, a visual closed-loop precision positioning of the charging cover is performed using a visual sensor at the robotic arm's end effector, and the detection confidence and positioning accuracy of the charging cover are verified to meet preset conditions. This allows for further correction of the robotic arm's cumulative errors and the vehicle's slight deviations after the robotic arm approaches the charging cover, improving the positioning accuracy before opening the cover and inserting the charging gun.

[0033] Once the confidence level and positioning accuracy meet the standards, the robotic arm is controlled to perform the charging task and continuously monitor charging safety. Specifically, after the accuracy verification meets the standards, the precise 3D coordinates of the charging cover are converted into coordinates in the robotic arm's base coordinate system. The upper computer plans and drives the robotic arm to perform the opening and insertion actions at the set speed, completing the precise docking of the charging gun with the vehicle's charging interface. After docking is completed, the charging gun is started to output charging current and perform the charging task. Furthermore, during the charging process, the vehicle's position is continuously tracked by a fixed-position camera. If a safety threshold is set for the vehicle's position offset, the charging output is immediately interrupted, and the charging gun is pulled out to avoid damage to the equipment. After the charging task is completed, the robotic arm smoothly pulls out the charging gun at a set speed, returns to the initial standby position, and returns to step S1 to continuously listen for new charging task instructions to be executed.

[0034] The following is a specific implementation process: An automatic positioning and charging method for charging covers of new energy vehicles based on cascade tracking includes the following steps: S1: The charging robotic arm completes the startup initialization and enters the standby stage. It continuously checks whether there is a charging task to be performed. If there is no task, it remains in standby. If there is a task, it starts the subsequent process and executes step S2. S2: Acquire multi-source sensor data and complete preprocessing. Perform vehicle detection based on the preprocessed data to obtain target vehicle detection data. If no vehicle is detected, continue with the vehicle detection operation. If a vehicle is detected, proceed to step S3. Step S1 determines the target vehicle for the charging task; step S2 corresponds to the vehicle detection in step 1 above. S3: Perform multi-target tracking based on the target vehicle detection data to obtain the vehicle's global pose information and tracking status information. Combined with the vehicle's structural priors, generate the charging cover constrained region of interest (ROI). S4: Perform target tracking on the region of interest (ROI) constrained by the charging cover, and complete the coarse detection of the charging cover by the multi-source sensor within the region. If the charging cover is detected, obtain the location of the charging cover and execute step S5; if the charging cover is not detected, start the re-detection process; this step corresponds to the aforementioned steps 2 and 3. S5: Based on the coarse detection of the charging cover's location, perform depth edge fitting and pixel-level fine positioning, and combine temporal tracking smoothing and Kalman filtering to achieve continuous positioning of the charging cover. The next step is executed after the vehicle is completely stationary. S6: Based on the camera intrinsic parameters, complete the coarse calculation of the 3D coordinates of the charging cover, drive the end of the charging robotic arm to move to the charging cover area, and complete the coarse positioning of the charging robotic arm; S7: Perform visual closed-loop fine positioning using the depth camera at the end of the charging robotic arm to verify the confidence level and positioning accuracy of the charging cover detection. If the confidence level is not met, return to step S4 to perform the coarse detection process. If the positioning accuracy is not met, repeat the fine positioning process in step S7. If any verification reaches the error threshold, report the error and request manual intervention. S8: After the confidence level and positioning accuracy meet the standards, control the robotic arm to perform the charging task and continuously monitor the charging safety. After the charging task is completed, return to the standby stage and re-determine whether there is a charging task to be performed.

[0035] Steps S5 to S8 correspond to step 4 above. The charging cover is precisely positioned through a cyclic process to improve the success rate of the robotic arm's charging task.

[0036] In step S1, system initialization is first performed. The charging robotic arm powers on the hardware system, performs the full initialization process, and starts standby listening. The initialization steps specifically include: S11: Control the charging robotic arm hardware system to power on and initialize, start the camera and configure parameters, and set the image output resolution and frame rate; Specifically, in this embodiment, two fixed-position depth cameras are set up, and a depth camera is also set up at the end of the robotic arm. First, the two fixed-position depth cameras and the depth camera at the end of the robotic arm are started, and the three cameras are configured to output RGB images and D-RGB images synchronously. The image output resolution is set to 640×480, the output frame rate is 30FPS, and the frame timestamp synchronization error of the three cameras is ensured to be ≤30ms. S12: Complete the power-on reset of the charging robotic arm and preset the relevant thresholds for the automatic charging process, which may include: detection confidence threshold, consecutive detection failure number threshold, positioning accuracy threshold, and vehicle deviation safety threshold. The relevant thresholds can be set as needed. In this embodiment, the detection confidence threshold is set to 0.85, the number of consecutive detection failures threshold is set to 30, the positioning accuracy threshold is set to 0.5mm, and the vehicle deviation safety threshold is set to 5mm. S13: Load the pre-calibration parameters of the robotic arm system, including the intrinsic parameter matrix of the camera, the extrinsic parameter matrix of the fixed-position camera, and the hand-eye homogeneous transformation matrix; In this embodiment, the loading system calibration parameters include three pre-calibrated camera intrinsic parameter matrices. extrinsic matrix of fixed-position dual cameras Hand-eye homogeneous transformation matrix The intrinsic parameter matrix of the i-th camera is defined in standard form as follows: ; In the formula, , Let x be the equivalent focal length of the i-th camera along the x and y axes. , Let be the coordinates of the principal point of the image from the i-th camera.

[0037] Transformation matrix from coordinate system B to coordinate system A in space Defined as: ; In the formula, For rotation matrix, It is a translation vector.

[0038] S14: Load the pre-trained YOLOv26_EVCar vehicle and charging cover detection model and the new energy vehicle parameterized structure prior library; The YOLOv26_EVCar vehicle and charging cover detection model can be obtained by transfer training based on YOLOv26 and the constructed new energy vehicle dataset. The prior library of parametric structures for new energy vehicles includes recorded exterior information of vehicle models and fixed three-dimensional transformation matrices of the charging cover relative to key points of the vehicle body. The vehicle charging cover refers to an openable and closable structural component that covers the outside of the vehicle's charging port.

[0039] In step S1, the charging robotic arm completes the startup initialization and enters the standby stage, continuously judging whether there is a charging task to be executed. Specifically, the charging robotic arm continuously listens for charging task instructions to be executed through the vehicle network communication interface. When there is no task, it maintains a low-power standby state. After receiving a valid charging task instruction, it wakes up the entire system and starts the automatic charging process, entering the multi-source sensor data acquisition and preprocessing and vehicle detection stage.

[0040] In this embodiment, the multi-source sensor is a camera used to acquire images. In step S2, the acquired multi-source sensor data are the original RGB image, the original depth image and the corresponding frame timestamp synchronously output by the fixed-position depth camera. The RGB image is used for semantic feature extraction and target detection, and the original depth image is used for geometric feature extraction, pose calculation and depth edge localization. The preprocessed depth image is a D-RGB image aligned with the pixels of the RGB image.

[0041] In step S2, the specific steps for acquiring and preprocessing data from multiple sensors, and then performing vehicle detection based on the preprocessed data to obtain vehicle target detection data include: S21. Preprocessing: The original RGB image in the acquired multi-source sensor data is subjected to environmental robustness optimization using a joint enhancement network to obtain an enhanced RGB image; the original depth image is subjected to median filtering and morphological closing operation to obtain a D-RGB image aligned with the pixels of the RGB image. Specifically, the lightweight joint enhancement network MIRNetV2 is used to achieve end-to-end image data enhancement. Selective kernel convolution establishes local-global feature dependencies, suppressing environmental interferences in outdoor scenes such as overexposure in strong light, underexposure in weak light, reflections on vehicle paint, and blurring due to rain and fog. The enhanced RGB image is output using the following formula: ; In the formula, The original RGB image, For network pre-training weights, For feature extraction and reconstruction mapping; Furthermore, a 3×3 kernel-sized median filter is applied to the original depth image to remove depth noise, followed by morphological closing operations to fill depth holes. The final output is the enhanced RGB image. D-RGB image with perfectly aligned pixels Complete data preprocessing; S22. Vehicle Detection: Based on the preprocessed multi-source sensor data described in S21 and the pre-trained YOLOv26_EVCar vehicle and charging cover detection model described in S12, vehicle targets are detected from the enhanced global RGB image. This may include outputting the vehicle's 2D detection box and confidence score, and filtering false detection results with confidence scores less than a threshold. Based on the vehicle's D-RGB image, the 6D pose of the detected vehicle target is obtained through a pose regression network, and the vehicle's 6D pose matrix is ​​output to obtain the target vehicle detection data. Specifically, the pose regression network is designed based on the PVN3D network, does not rely on RGB textures, and adopts a point cloud voting mechanism based on D-RGB depth priority to register the point clouds of the two fixed-position cameras into the same coordinate system to generate a global point cloud. The point cloud is then input into the PVN3D network to return the vehicle's 6D pose.

[0042] In step S3, multi-target tracking is performed based on the target vehicle detection data. The specific steps include: S31: Convert the 6D vehicle pose matrix output from the fixed-position camera into a pose matrix in the world coordinate system. , Based on the translation vector of the pose matrix, it is determined whether it is the same vehicle pose. The poses of the same vehicle are weighted and fused to obtain global pose information. In this embodiment, the 6D vehicle pose matrix detected and output by two fixed-position depth cameras is converted into a pose matrix in the world coordinate system using coordinate transformation formulas. , ; One feasible implementation method is to determine whether the poses of two vehicles are the same based on the translation vectors of the pose matrix. The correlation between the translation vectors of the two poses can be calculated, such as the Euclidean distance or the Pearson correlation coefficient. Specifically, when using Euclidean distance d to determine correlation, if the Euclidean distance d is less than a preset position threshold, it is determined to be the same target vehicle. The two poses of the same vehicle are then weighted and fused to obtain the globally optimal pose matrix. That is, global pose information; Specifically, the formula for calculating the Euclidean distance between two pose translation vectors is: ; In the formula, , These are the pose matrices of the same vehicle detected by camera A and camera B in the world coordinate system. , The corresponding translation vector; , They are respectively , The corresponding rotation matrix, This is a spherical linear interpolation algorithm used for the smooth fusion of rotation matrices. Let A be the pose confidence score of camera A. , The confidence levels of the vehicle outputs are measured by cameras A and B, respectively.

[0043] It is configurable. In this embodiment, the Euclidean distance position threshold is 100mm. When the Euclidean distance is less than this threshold, it is determined to be the same target vehicle. Step 32: Based on the target vehicle confidence score and feature similarity output from the fixed camera, calculate the camera attention weights, and perform weighted fusion of the visual features of the same target vehicle to obtain the target feature vector. To assign a unique tracking ID to the same target vehicle under the global image information; In this embodiment, a unique tracking ID is assigned to the same target vehicle detected under global image information. Based on the PredLink-MOT multi-target tracking framework, the target vehicle information detected by two fixed cameras is correlated and analyzed. The attention weights corresponding to different cameras are determined by combining the confidence of the target vehicle, the similarity between the detected features and the known template features.

[0044] Fig. 3 This is a schematic diagram of the PredLink-MOT multi-target tracking framework in this embodiment, clearly showing the process of achieving end-to-end multi-target tracking based on dual fixed-position cameras; Based on the PredLink-MOT multi-target tracking framework, this method performs cross-camera association, feature fusion, and cross-frame ID allocation for vehicle targets detected by two fixed-position cameras. Specifically, as shown in Figure 3, the detection results from camera A and camera B are first input into the Fea-Dec Fusion dual-camera multi-source detection module. Through cross-modal candidate box matching and encoding, the target candidate boxes, target features, and pose information detected by the two cameras are encoded into a sparse input tensor T. Subsequently, sparse feature extraction, QKV splitting, attention mechanism, and feedforward network are used to enhance the features and match the associated candidate targets, resulting in the target detection results and the association relationships between the candidate targets.

[0045] Furthermore, the PredLink-MOT multi-target tracking framework inputs the target detection results and their historical multi-target tracking IDs from historical frames, along with newly detected and continuously tracked targets in the current frame, into a Transformer-based encoder. By performing temporal modeling of historical target features, current target features, and their correlations, it predicts the multi-target tracking IDs corresponding to each vehicle target in the current frame. For existing vehicle targets, the framework maintains their original tracking IDs; for newly detected vehicle targets, the framework assigns new learnable IDs, thus obtaining the multi-target tracking results for the current frame.

[0046] In one feasible implementation, this embodiment uses two fixed cameras to acquire the detection confidence scores of camera A and camera B for the same target vehicle. , And the cosine similarity between the detected vehicle features and the corresponding template features. , Calculate the attention weights for camera A and camera B. , The calculation formula is: ; The vehicle visual semantic features extracted from the two cameras are weighted and fused with the depth map geometric features to obtain the target feature vector: Visual features of the same target vehicle detected by two fixed-position cameras were extracted. , The visual features include visual semantic features of the RGB image and geometric features of the depth map. The final target feature vector is obtained by weighted fusion of features from the two cameras. : ; By using the above method, the confidence level and feature matching similarity of different camera detection results can be used to adaptively adjust the multi-camera feature fusion weights, thereby suppressing the interference of low-confidence detection results or feature matching errors on target association and improving the accuracy and continuity of tracking ID allocation for the same target vehicle.

[0047] S33: Based on the fused target feature vector, global pose information and two-dimensional detection box, establish a global tracking state dictionary to obtain and maintain the tracking state information of the target vehicle; In one feasible implementation, the global optimal pose matrix obtained in step S31 is used as the basis. and the target feature vector obtained in step S32 Each target vehicle is bound to its corresponding global tracking ID, and a global tracking status dictionary is constructed, represented as follows:

[0048] in, Let k be the target feature vector of the k-th target vehicle. Let be the 6D pose matrix of the k-th target vehicle in the world coordinate system. Let be the 2D detection bounding box for the k-th target vehicle. To track the number of frames the target survives, continuous multi-target tracking of all detected vehicle targets is achieved.

[0049] Specifically, for target vehicles that have been matched in the current frame, their fused target feature vectors are... 6D pose matrix in world coordinate system and two-dimensional detection frame Write the corresponding global tracking state dictionary and update the survival frame count of the target vehicle. For new detected targets that fail to match existing tracked targets, a new unique tracking ID is assigned to it, and it is added as a new target status item to the global tracking status dictionary.

[0050] Furthermore, for target vehicles that have not been detected again or have failed to complete a valid match for several consecutive frames, the global tracking status dictionary is updated based on the number of frames that survive or the number of frames that are lost. If the number of frames exceeds the preset lifecycle threshold, the status item corresponding to the target vehicle is removed from the global tracking status dictionary to avoid invalid targets occupying tracking resources for a long time.

[0051] In this way, the appearance features, spatial pose, image detection position and temporal state of the target vehicle can be uniformly recorded in the global tracking state dictionary, so that the system can maintain stable target identity information between consecutive image frames, thereby realizing continuous multi-target tracking of all detected vehicle targets.

[0052] This embodiment uses the attention mechanism of the PredLink-MOT multi-target tracking framework and the target feature vector obtained by the above weighted fusion. It suppresses false detection interference through accurate feature matching and confidence screening, realizes end-to-end ID allocation and prediction, and thus achieves multi-target tracking of all detected vehicle targets.

[0053] A further technical solution involves prior knowledge of the vehicle structure, which consists of geometric constraints determined in advance based on the target vehicle's body structure. This includes the positional relationships and dimensional ranges of key body parts of the target vehicle within its own coordinate system, as well as the relative positional relationships between the potential location of the charging cover and the body structure components. Specifically, the prior knowledge of the vehicle structure may include a set of prior key points for the front fender, rear fender, front bumper, rear bumper, and charging cover mounting area of ​​the target vehicle within its own coordinate system.

[0054] In other words, the vehicle structure prior is not a result obtained directly from the current image, but rather a structural template pre-established based on the body structure of the new energy vehicle. This structural template is used to characterize the geometric patterns of the charging cover, which is typically located in a specific area on the rear side of the vehicle, near the fender, or close to the area between the front and rear bumpers, thus providing positional constraints for the detection of the charging cover.

[0055] Generating and dynamically updating the charging cover constrained region of interest (ROI) includes the following steps: First, based on the global pose information of the target vehicle, the prior key points of the vehicle structure are transformed into the world coordinate system; In one feasible implementation, the target vehicle's 6D pose matrix in the world coordinate system obtained in step S31 is used as the basis. This refers to global pose information; transforming the prior structural key points in the vehicle's own coordinate system to the world coordinate system, we obtain the world coordinate points corresponding to the candidate areas of the front and rear fenders, front and rear bumpers, and charging cover of the target vehicle. : ; in, These are the structural prior key points in the vehicle's own coordinate system. To transform the structural key points to the world coordinate system, The target vehicle's 6D pose matrix in the world coordinate system.

[0056] Furthermore, combining the intrinsic parameter matrix of the i-th camera... And the transformation matrix from the world coordinate system to the coordinate system of the i-th camera. By projecting the structural key points in the world coordinate system onto the image pixel coordinate system, the corresponding two-dimensional pixel points are obtained. Its projection formula is: ; In the formula, This represents the depth value of a 3D point in the camera coordinate system. Let be the transformation matrix from the world coordinate system to the i-th camera coordinate system; After projection, based on the pixel positions of the front fender, rear fender, front bumper, and rear bumper obtained from the projection, the possible image areas where the charging cover may appear are determined, and a constrained region of interest (ROI) for the charging cover is generated. The constrained ROI for the charging cover is used to limit the search range for subsequent charging cover detection or recognition.

[0057] Specifically, one or more candidate ROI regions can be generated in the image based on the relative positional relationship between the charging port cover and the fender and bumper in the prior knowledge of the vehicle structure. For example, for models where the charging port is located on the side and rear of the vehicle, the side and rear ROI can be generated based on the projected positions of the rear fender and the rear bumper; for models where the charging port is located on the front of the vehicle, the front ROI can be generated based on the projected positions of the front fender and the front bumper.

[0058] Furthermore, since the generation of ROI depends on the 6D pose information of the target vehicle, when the target vehicle changes position or pose in consecutive frames, the ROI constrained by the charging cover can be updated synchronously with the pose change of the target vehicle, thereby realizing cascaded tracking of the candidate regions of the charging cover without the need to set up a separate tracker for the charging cover.

[0059] This embodiment introduces vehicle structure priors, which utilize the fixed relative positional relationship between the charging cover and the fender and bumper within the vehicle's own structure to spatially constrain the detection range of the charging cover, avoiding indiscriminate searching across the entire image and thus reducing background interference and false detection probability. Simultaneously, the charging cover constrained ROI can be dynamically updated with the 6D pose change of the target vehicle, enabling the candidate region of the charging cover to maintain consistent movement with the target vehicle, thereby improving the accuracy and stability of subsequent charging cover detection, localization, and tracking.

[0060] Furthermore, coarse detection of the charging cover is performed within the constrained region of interest (ROI). The trained YOLOv26_EVCar detection model is used to perform coarse detection of the charging cover. If the target vehicle has completely stopped, the target vehicle features are matched with the vehicle models in the parameterized structure prior library of new energy vehicles. If the match is successful, the position of the charging cover is predicted by combining the historical valid frames and the fixed transformation matrix of the charging cover relative to the vehicle body. If the match fails, the system enters a standby state and requests manual intervention.

[0061] Specifically, within the region of interest (ROI) constrained by the charging cover of two fixed-position cameras, the YOLOv26_EVCar vehicle and charging cover detection model is used to detect the presence of a charging cover. If a charging cover is detected, it is projected onto the world coordinate system, ending the tracking of the charging cover constrained ROI based on the vehicle tracking state dictionary, and starting the tracking of the charging cover based on the vehicle tracking state dictionary, continuously determining whether the vehicle is stationary; if it is not detected, a re-detection process is initiated, continuously determining whether the vehicle has stopped based on the vehicle tracking state. If the target vehicle is still moving, multi-source sensor coarse detection of the charging cover continues within the ROI. In step S4, a re-detection is initiated based on the vehicle's motion state until the charging cover is detected, thus obtaining the charging cover's location. The specific steps of the re-detection process include: S41: Continuously determine whether the target vehicle has completely stopped based on the global tracking state dictionary. If the target vehicle is still moving, continue to perform coarse detection on the charging cover within the region of interest (ROI) constrained by the charging cover. If the target vehicle has completely stopped, proceed to step S42. Among them, coarse detection can use a pre-trained YOLOv26_EVCar detection model; Specifically, if the vehicle pose translation change is ≤10mm and the rotation angle change is ≤1° for 150 consecutive frames, it is determined that the vehicle is completely stationary, and vehicle model matching and charging cover position prediction are performed. S42: The detected target vehicle features are matched with the vehicle models in the pre-built vehicle structure prior library. If the match is successful, the position of the charging cover is predicted by combining the historical valid frame data and the fixed transformation matrix of the charging cover relative to the vehicle body. If the match fails, step S43 is executed. Optionally, the detected target vehicle features can be matched with the vehicle models in the pre-built vehicle structure prior library. Cosine similarity matching can be used. If the highest similarity is ≥0.8, the match is considered successful; if the highest similarity is <0.8, the match is considered unsuccessful. S43: The robotic arm reports an error and requests manual intervention, and enters standby mode. After manual intervention assists the robotic arm in recognizing and locating the charging cover, the location of the charging cover is obtained. Further, the vehicle model data and the charging cover's relative vehicle position data are saved into the vehicle structure prior library, and the prior library is iteratively updated. Then, step S6 is executed.

[0062] In step S43, manual intervention specifically refers to manually inputting the location coordinates of the charging cover in the fixed-position camera image through the host computer's interactive interface, and the host computer simultaneously extracting and storing the vehicle model features and the charging cover's pose data relative to the vehicle body.

[0063] In step S5, after the coarse detection of the charging cover's location, the following steps are also included: based on the coarsely detected charging cover location, depth edge fitting and pixel-level charging cover localization are performed, and continuous localization and tracking of the charging cover are achieved by combining temporal tracking smoothing and Kalman filtering. Specific implementation steps include: S51: Based on the 2D detection box of the charging cover obtained by coarse detection, extract the local image of the charging cover and generate a sub-pixel level edge point set; Specifically, based on the 2D detection box of the charging cover obtained from coarse detection, the image is cropped to obtain a local image containing only the charging cover. Edge detection is performed on the local RGB image, and depth transition edge extraction is performed on the depth image of the corresponding region. The results are then fused to obtain a sub-pixel level edge point set of the charging cover. ; in, For the i-th edge point, ( Let be the coordinates of the i-th edge point in the image pixel coordinate system; Number of edge points S52: Based on the sub-pixel edge point set, a robust edge fitting algorithm with prior constraints is used to obtain the center coordinates of the charging cover. ; The prior constraints may include shape constraints, size constraints, aspect ratio constraints, and center position constraints of the charging cover, used to eliminate interference from local noise edges or background edges. The minimization objective function of the fitting process can be expressed as: ; In the formula, is the squared error term between the sub-pixel edge point set and the model; FIT is the fitted charging cover edge model; S53: Based on the fitted center coordinates of continuous frames, perform time-series smoothing processing on the center coordinates of the charging cover; Specifically, the sliding window length is set to... The time-series smoothing queue uses a weighted moving average method to smooth the center coordinates, suppressing coordinate abrupt changes caused by coarse detection errors; The smoothing formula is: ; In the formula, the weights It is positively correlated with the detection confidence, and the newer the frame, the greater the weight. The coordinates of the center of the charging cover after smoothing in the kth frame; The center coordinates of the charging cover are obtained by fitting the i-th frame; S54: Establish a Kalman filter model for the charging cover, with the state vector being the pixel coordinates and movement speed of the center of the charging cover. Output the optimal pixel coordinate positioning result of the center of the charging cover to achieve continuous and stable tracking of the charging cover.

[0064] The above method enables fine positioning of the charging cover by utilizing RGB edge and depth edge information on the basis of coarse detection results. Furthermore, temporal smoothing and Kalman filtering reduce detection jitter, improve the continuity and stability of the charging cover positioning results, and thus achieve continuous positioning and tracking of the charging cover.

[0065] Furthermore, in step S6, based on the camera intrinsic parameters, a coarse calculation of the 3D coordinates of the charging cover is completed, and the end effector of the charging robotic arm is driven to move to the vicinity of the charging cover to complete the coarse positioning of the charging robotic arm. The specific implementation steps include: The final charging cover position is coarsely calculated in 3D coordinates, transformed into the world coordinate system, and then the 3D coordinates are transmitted to the charging robotic arm.

[0066] The end effector of the charging robotic arm moves to the vicinity of the charging cover, and the charging gun port and the depth camera at the end effector of the robotic arm are aligned with the center point of the charging cover, thus completing the coarse positioning of the charging robotic arm.

[0067] Specifically, based on the principle of binocular stereo vision in-camera input and combined with the extrinsic matrix of two fixed-position cameras, the optimal pixel coordinates of the charging cover center are calculated to obtain the 3D coordinates of the charging cover center in the world coordinate system. The 3D coordinates are sent to the robotic arm motion control system. The robotic arm host computer plans the obstacle avoidance motion trajectory and drives the end of the charging robotic arm to move to 200mm in front of the charging cover, so that the charging gun socket is aligned with the center of the charging cover, thus completing the coarse positioning of the charging robotic arm.

[0068] Furthermore, in step S7, visual closed-loop precision positioning is performed using a depth camera at the end of the charging robotic arm to verify the confidence level and positioning accuracy of the charging cover detection. Specific implementation steps include: Step S71: Use the depth camera at the end of the robotic arm to scan the charging cover. Use the trained YOLOv26_EVCar vehicle and charging cover detection model combined with the fitting localization algorithm to detect and locate the charging cover. Verify whether the detection confidence of the charging cover meets the standard. If it does not meet the standard, repeat the coarse localization process. If it meets the threshold, perform visual closed-loop fine localization using the depth camera at the end of the robotic arm. Specifically, if the confidence level of the detected charging cover is <0.85, return to the coarse detection process in step S4 and re-execute the global detection and ROI generation; if the confidence level is ≥0.85, proceed to the positioning accuracy verification step, i.e., step S72. Step S72: Based on the visual closed loop of the end-effector camera, correct the cumulative error of the robotic arm and the micro-movement offset of the vehicle, adjust the end-effector pose in real time, and verify whether the positioning accuracy meets the standard. If it does not meet the standard, repeat the positioning process.

[0069] Specifically, the deviation between the target position of the charging cover and the current end-effector pose of the robotic arm is compared in real time. In this embodiment, the initial deviation detected is 0.7mm in the X direction, 0.3mm in the Y direction, and 0.4mm in the Z direction. If the initial deviation in the X direction exceeds the set threshold of 0.5mm, the cumulative motion error of the robotic arm is corrected by image-based visual servo control, and the end-effector pose of the robotic arm is adjusted in real time. After the adjustment is completed, the positioning accuracy is verified. If the positioning accuracy is still >0.5mm, the fine positioning process is repeated. If the number of consecutive detection failures reaches the threshold, the charging robotic arm will report an error, enter standby mode, and request manual intervention.

[0070] Optionally, if the number of consecutive detection failures reaches the error threshold of 30, the charging robotic arm immediately reports the error and requests manual intervention; in this embodiment, the adjusted positioning accuracy meets the accuracy requirements, the verification is passed, and step S8 is executed.

[0071] S8: After the confidence level and positioning accuracy meet the standards, control the robotic arm to perform the charging task and continuously monitor the charging safety. Specifically, after the accuracy verification is passed, the precise 3D coordinates of the charging cover are converted into coordinates in the base coordinate system of the robotic arm. The host computer plans and drives the robotic arm to perform the opening and insertion actions at a stable speed of 10mm / s, completing the precise docking of the charging gun with the vehicle's charging interface. After docking is completed, the charging gun is started to output charging current and perform the charging task. Furthermore, during the charging process, the vehicle's position is continuously tracked by a fixed-position camera. If the vehicle's position deviation exceeds the safety threshold of 5mm, the charging output is immediately interrupted and the charging gun is pulled out to avoid damage to the equipment. After the charging task is completed, the robotic arm smoothly pulls out the charging gun at a speed of 20mm / s, returns to the initial standby position, and returns to step S1 to continuously listen for new charging task instructions to be executed.

[0072] Example 2 Based on Embodiment 1, this embodiment provides an automatic positioning and charging system for vehicle charging covers based on cascade tracking, including: The multi-target tracking module is configured to perform multi-camera pose fusion and multi-target tracking on the target vehicle to obtain the target vehicle's global pose information and tracking status information. The region of interest identification module is configured to generate and dynamically update the charging cover constrained region of interest based on the vehicle pose of the target vehicle and the pre-established vehicle structure prior. The coarse detection module is configured to perform coarse detection of the charging cover within the region of interest; when the charging cover is detected, the charging cover positioning result is obtained; when the charging cover is not detected, re-detection is initiated according to the vehicle's motion state until the charging cover is detected and the charging cover positioning position is obtained. The precision positioning charging module is configured to locate the charging cover based on coarse detection, drive the robotic arm to move to the charging cover area, and use the vision sensor at the end of the robotic arm to perform visual closed-loop precision positioning of the charging cover. After the detection confidence and positioning accuracy of the charging cover meet the preset conditions, the robotic arm is controlled to perform the charging process.

[0073] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0074] Example 3 Based on Embodiment 1, this embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps in the automatic positioning and charging method for vehicle charging covers based on cascade tracking described in Embodiment 1.

[0075] Example 4 Based on Embodiment 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they complete the steps in the automatic positioning and charging method for vehicle charging covers based on cascade tracking described in Embodiment 1.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0077] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for automatic positioning and charging of vehicle charging covers based on cascade tracking, characterized in that, Includes the following steps: Multi-camera pose fusion and multi-target tracking are performed on the target vehicle to obtain the target vehicle's global pose information and tracking status information; Based on the vehicle pose of the target vehicle and the pre-established vehicle structure priors, the charging cover constraint region of interest is generated and dynamically updated. Perform coarse detection of the charging cover within the region of interest; when the charging cover is detected, obtain the charging cover location result; If the charging cover is not detected, a re-detection is initiated based on the vehicle's movement status until the charging cover is detected, and the location of the charging cover is obtained. Based on the coarse detection of the charging cover's location, the robotic arm is driven to move to the charging cover area. The end vision sensor of the robotic arm is used to perform visual closed-loop fine positioning of the charging cover. After the charging cover's detection confidence and positioning accuracy meet the preset conditions, the robotic arm is controlled to perform the charging process.

2. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, The target vehicle is detected and tracked to obtain its pose and motion state, including the following steps: Image data collected from the target vehicle is acquired. The original RGB image is enhanced using a joint enhancement network to obtain an enhanced RGB image. The original depth image is then processed by median filtering and morphological closing operations to obtain a D-RGB image that is pixel-aligned with the RGB image. The enhanced RGB image is input into the pre-trained YOLOv26_EVCar vehicle and charging cover detection model to obtain the detected vehicle target; based on the D-RGB image, the 6D pose of the detected vehicle target is obtained through the pose regression network, and the 6D pose matrix of the vehicle is output to obtain the target vehicle detection data. Multi-target tracking is performed based on the target vehicle detection data to obtain the vehicle's global pose information and tracking status information.

3. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, The implementation methods for multi-target tracking based on target vehicle detection data include: The 6D vehicle pose matrix detected by the fixed-position camera is converted into a pose matrix in the world coordinate system. , Based on the translation vector of the pose matrix, it is determined whether it is the same vehicle pose. The poses of the same vehicle are weighted and fused to obtain global pose information. Based on the target vehicle confidence score and feature similarity score detected by the fixed camera, camera attention weights are calculated, and the visual features of the same target vehicle are weighted and fused to obtain the target feature vector. It can assign a unique tracking ID to the same target vehicle under global image information; Based on the fused target feature vector, global pose information, and two-dimensional detection box, a global tracking state dictionary is established to obtain and maintain the tracking state information of the target vehicle.

4. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, Based on the target vehicle's pose and pre-established vehicle structure priors, the charging cover constraint region of interest is generated and dynamically updated, including the following steps: Based on the global pose information of the target vehicle, the prior key points of the vehicle structure are transformed into the world coordinate system; Project the structural key points in the world coordinate system onto the image pixel coordinate system; The charging cover constrains the region of interest based on the projected structural key points.

5. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, Within the region of interest constrained by the charging cover, a coarse detection of the charging cover is performed using a trained YOLOv26_EVCar detection model. If the target vehicle has come to a complete stop, the target vehicle features are matched with the vehicle models in the pre-built vehicle structure prior library. If the match is successful, the position of the charging cover is predicted by combining historical valid frames and the fixed transformation matrix of the charging cover relative to the vehicle body. If the match fails, the system enters a standby state and requests manual intervention.

6. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, After the coarse detection of the charging cover's location, the process also includes: performing depth edge fitting and pixel-level charging cover positioning based on the coarsely detected charging cover location, and combining temporal tracking smoothing and Kalman filtering to achieve continuous positioning and tracking of the charging cover.

7. The automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in claim 1, characterized in that, The charging cover is precisely located visually using a vision sensor at the end of the robotic arm. Specifically, the charging cover area is scanned by a depth camera at the end of the robotic arm, and the charging cover is detected and accurately located by using a trained YOLOv26_EVCar vehicle and charging cover detection model combined with an edge fitting and localization algorithm. The system compares the deviation between the target position of the charging cover and the current end-effector pose of the robotic arm in real time, and corrects the cumulative motion error of the robotic arm and the micro-movement deviation of the vehicle through visual servo control, thereby adjusting the end-effector pose of the robotic arm in real time.

8. A vehicle charging cover automatic positioning and charging system based on cascade tracking, characterized in that, include: The multi-target tracking module is configured to perform multi-camera pose fusion and multi-target tracking on the target vehicle to obtain the target vehicle's global pose information and tracking status information. The region of interest identification module is configured to generate and dynamically update the charging cover constrained region of interest based on the vehicle pose of the target vehicle and the pre-established vehicle structure prior. The coarse detection module is configured to perform coarse detection of the charging cover within the region of interest; when the charging cover is detected, the charging cover positioning result is obtained; when the charging cover is not detected, re-detection is initiated according to the vehicle's motion state until the charging cover is detected and the charging cover positioning position is obtained. The precision positioning charging module is configured to locate the charging cover based on coarse detection, drive the robotic arm to move to the charging cover area, and use the vision sensor at the end of the robotic arm to perform visual closed-loop precision positioning of the charging cover. After the detection confidence and positioning accuracy of the charging cover meet the preset conditions, the robotic arm is controlled to perform the charging process.

9. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the steps in the automatic positioning and charging method for a vehicle charging cover based on cascade tracking as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the steps in the automatic positioning and charging method for vehicle charging covers based on cascade tracking as described in any one of claims 1-7.

Images