Mowing robot and method for recharging positioning of a mowing robot

Abstract

This application relates to the field of lawnmower robot positioning and navigation technology, and discloses a method for recharging and positioning of a lawnmower robot and the lawnmower robot itself. The method includes: obtaining a first set of key points for the charging dock through a global positioning model, and extracting multiple local image regions with different views from an image based on this, then processing them separately using a local positioning model to obtain multiple sets of second key point groups. The first set of key points is fused with the multiple sets of second key point groups, and the fusion result is optimized for key points. After obtaining the fused key point group, a point-by-point consistency check is performed based on the first set of key points, correcting any key points that fail the check, and obtaining the final positioning result. This application integrates global structural priors and multi-scale local details, possessing the ability to verify and correct the reliability of key point positioning results, effectively improving the robustness and accuracy of recharging and positioning in complex outdoor environments.

Description

Technical Field

[0001] This application relates to the field of lawn mowing robot positioning and navigation technology, and in particular to a recharging positioning method for a lawn mowing robot and a lawn mowing robot. Background Technology

[0002] When lawnmower robots perform recharging tasks in outdoor environments such as yards, they rely on visual information to accurately obtain the position and orientation of the charging dock. Existing visual positioning solutions mostly use single-pass keypoint regression to directly predict the coordinates of each corner of the charging dock. However, in practical applications, factors such as strong light, shadows, reflections, occlusion, and robot tilt often cause significant drift or jitter in the positions of some key points. Due to the lack of a mechanism for verifying and correcting the reliability of the positioning results, deviations at individual points can easily affect the overall positioning accuracy and even cause structural distortion of the charging dock, reducing the success rate of recharging and docking. Furthermore, a single observation scale cannot adequately consider both global context and local details, resulting in insufficient adaptability in complex scenarios. Therefore, improving the robustness and accuracy of charging dock positioning remains a pressing issue. Summary of the Invention

[0003] Based on this, it is necessary to address the technical problem of inaccurate positioning of the charging dock by existing lawnmower robots and propose a recharging positioning method for lawnmower robots.

[0004] Firstly, a recharging and positioning method for a lawnmower robot is provided, including: Acquire an image including the charging dock; The image is globally located using a global localization model to obtain the first set of key points for the charging dock; Based on the first key point group, multiple local image regions with different field of view are extracted from the image; The local image regions are locally located using a local localization model to obtain the corresponding second keypoint group; The first key point group is fused with multiple groups of the second key point group, and the key points of the fusion result are optimized to obtain the fused key point group. Based on the first key point group, a point-by-point consistency check is performed on the fused key point group, and key points that fail the check are corrected to obtain a corrected key point set.

[0005] Secondly, a recharging and positioning device for a lawnmower robot is provided, the device comprising: The acquisition module is used to acquire images including the charging dock; A global positioning module is used to perform global positioning on the image using a global positioning model to obtain the first key point group of the charging dock. An extraction module is used to extract multiple local image regions with different field of view from the image based on the first key point group; The local positioning module is used to perform local positioning on each of the local image regions using a local positioning model to obtain the corresponding second key point group; The fusion module is used to fuse the first key point group with multiple groups of the second key point group, and to optimize the key points of the fusion result to obtain a fused key point group. The correction module is used to perform point-by-point consistency verification on the fused key point group based on the first key point group, and to correct the key points that fail the verification, so as to obtain the corrected key point set.

[0006] Thirdly, a lawnmower robot is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned lawnmower robot recharging and positioning method.

[0007] Fourthly, a computer storage medium is provided, which stores a computer program that, when executed by a processor, implements the above-mentioned recharging and positioning method for the lawnmower robot.

[0008] Beneficial effects: This application effectively solves the problem that a single observation scale cannot simultaneously capture both global context and local details by employing a two-stage processing architecture that combines global localization and multi-view local localization. First, a global localization model is used to acquire a first set of key points across the entire image, establishing an initial understanding of the charging dock's overall position and orientation. Then, based on this first set of key points, multiple local image regions with different view ranges are extracted, and each is precisely located by a local localization model to obtain multiple sets of second key points. This multi-scale collaborative localization approach ensures that the charging dock's key points maintain structural integrity from a global perspective while capturing detailed differences near corner points through local observations from different view ranges. This significantly reduces the impact of individual point drift caused by factors such as lighting changes, occlusion, or viewing angle tilt on the overall localization results.

[0009] This application fuses a first set of keypoints with multiple sets of second keypoints and applies keypoint optimization to the fusion result to obtain a fused keypoint set. This process effectively integrates the stable priors of the global model with the detailed observations of the multi-path local model, and further constrains the geometric relationships between keypoints through optimization methods. This ensures that the output result conforms to the true features of the image while also conforming to the inherent structural shape of the charging dock, overcoming the frame distortion problem that easily occurs in single regression.

[0010] This application, by comparing the consistency of each point in the fused keypoint group with that in the first keypoint group, can automatically identify and locate unreliable keypoints with significant discrepancies, and then correct them accordingly. This mechanism provides a post-implementation reliability verification and error correction method for the fused keypoints, ensuring that individual erroneous points severely deviated due to partial occlusion or strong reflection do not propagate to the entire group's positioning failure. It also helps improve the robustness of recharge positioning in complex outdoor scenarios and the success rate of recharge docking. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] in: Figure 1 This is a schematic diagram of a lawnmower robot in one embodiment; Figure 2 This is a flowchart illustrating the recharging and positioning method for a lawnmower robot in one embodiment. Figure 3A Another flowchart illustrating the recharging and positioning method for the lawnmower robot provided in this application embodiment; Figure 3B A schematic diagram of the error index card provided in the embodiments of this application; Figure 3C The final fine positioning result diagram provided for the embodiments of this application; Figure 3D This is a schematic diagram of post-processing and key point correction provided for an embodiment of this application; Figure 3E This is a comparison chart of GT / Stage 1 / Final for the difficult samples provided in the embodiments of this application; Figure 4 This is a structural block diagram of the recharging and positioning device for a lawnmower robot in one embodiment; Figure 5 This is a structural block diagram of a lawnmower robot in one embodiment. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1 As shown, this application scenario can be applied to a courtyard or semi-open outdoor environment, involving a lawnmower robot 1 and a charging base station 2. The lawnmower robot 1 is equipped with a LiDAR 11 and a camera 12, which is used to collect environmental images in the direction the lawnmower robot 1 is traveling.

[0014] During the recharging task, the lawnmower robot 1 approaches the area where the charging base station 2 is located via its navigation module. Once the charging base station 2 enters the field of view of the camera 12, the processor of the lawnmower robot 1 acquires the image containing the charging base station 2 captured by the camera 12 and triggers the execution of the recharging fine positioning method provided in this embodiment. The processor utilizes its internal global and local positioning models to perform multi-stage localization and fusion optimization of the inner and outer frame corners of the charging base station 2 in the image, ultimately outputting the precise key point coordinates of the charging base station 2. Based on these key point coordinates, the lawnmower robot 1 calculates the relative position and attitude angle of the charging base station 2, and then adjusts its own trajectory to complete the precise docking and recharging with the charging base station 2.

[0015] In this application scenario, the courtyard environment may present complex conditions such as direct sunlight, tree shadows, surface reflections from the charging base station, and tilting of the robot due to uneven ground. The method provided in this application, through multi-view collaborative positioning, key point optimization, and point-by-point consistency verification and correction, can maintain the stability and accuracy of key point positioning of the charging base station under the above complex conditions, ensuring the successful recharging of the lawnmower robot 1.

[0016] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the recharging and positioning method based on a lawnmower robot provided in this application embodiment includes the following steps: S1. Acquire an image including the charging dock.

[0017] The lawnmower robot uses its onboard camera to capture real-time environmental images, which are RGB color images including the charging dock. After acquiring this color image, a copy of the original image is saved for subsequent localization processing, and a grayscale image is generated based on the color image. This grayscale image is stored for later keypoint positioning fine-tuning. Throughout the entire process, the coordinates of all keypoints are output based on the coordinate system established from the original image.

[0018] In one example, when the lawnmower is positioned on the lawn and facing the charging dock, its front-facing camera continuously captures RGB images at resolutions of 640*480 or 1280*720. Upon receiving a frame containing the complete appearance of the charging dock, the original RGB image data is stored in memory, and an image conversion function is called to generate the corresponding grayscale image. All subsequent calculated corner coordinates of the charging dock, such as a corner with an x-coordinate of 320 and a y-coordinate of 240 in the image, refer to its specific location within the pixel coordinate system of that 640*480 image frame.

[0019] S2. Use the global localization model to perform global localization on the image to obtain the first key point group of the charging dock.

[0020] The acquired complete RGB image is input into a pre-trained and deployed global localization model. This global localization model is a lightweight convolutional neural network that performs both object detection and keypoint coordinate regression by inferring at the scale of the entire image. The model first outputs a bounding box representing the approximate location and extent of the charging dock, defined by four parameters: center point coordinates, width, and height. Simultaneously, the model directly regresses the coordinates of eight initial keypoints for the charging dock's inner and outer frames. These eight points correspond to the top-left, top-right, bottom-right, and bottom-left corners of the inner frame, and the top-left, top-right, bottom-right, and bottom-left corners of the outer frame, respectively. These eight coordinates form the first keypoint group, which provides the global location and rough shape information of the charging dock within the entire image.

[0021] In one example, the global localization model is configured to process the entire input image. Its encoder consists of a lightweight backbone network, which can employ a CSPDarkNet architecture containing multiple cascaded, cross-stage local network layers to extract multi-scale visual features layer by layer from the input image. In the decoder, the model branches into two parallel output branches. The first branch is the object detection branch, which receives the high-level semantic feature map output from the backbone network. A detection head network predicts the bounding box offset, size, and object confidence at each grid cell of the feature map, and finally, after non-maximum suppression, outputs the charging cradle bounding box with the highest confidence. The second branch is the keypoint regression branch, which also receives the feature map from the backbone network, but outputs a set of continuous values ​​directly after a fully connected layer or a feature map-based global pooling layer through a separate regression head network. These values ​​represent the eight pairs of coordinates for the first set of keypoints.

[0022] In one example, the training process of the global localization model is as follows. First, a dataset containing a large number of charging dock images is constructed. Each image is manually and precisely labeled with the ground truth bounding box and the ground truth coordinates of 8 keypoints. During training, the images are input into the model, and the model predicts both the bounding box and the first set of keypoints simultaneously. When calculating the total loss function, the CIoU loss function is used for the object detection branch to calculate the deviation between the predicted box and the ground truth box. The SmoothL1 loss function is used for the keypoint regression branch to calculate the absolute deviation between the coordinates of all 8 predicted points and the coordinates of the 8 ground truth points, and a weight coefficient λ is assigned to this loss term. pts The coefficient is set to 5.0 to enhance the model's learning of keypoint location accuracy. Furthermore, to further constrain the geometric rationality of the shape formed by the keypoints, a shape regularization penalty term is introduced into the total loss function. This term calculates the sum of the angular deviations between the four interior angles formed by the four points of the predicted outer frame and the 90-degree right angle. The model employs stochastic gradient descent or its variant optimizer, using backpropagation to update the network parameters until the total loss function converges.

[0023] S3. Based on the first key point group, extract multiple local image regions with different field of view from the image.

[0024] Using the first set of keypoints and the bounding boxes that the global localization model might output, multiple local image regions of varying sizes are planned and cropped from the original image. The generation rules for these local regions are as follows: One method is to use the bounding box obtained in step S2 as a reference, keeping its center point unchanged, and expanding the width and height of the bounding box outwards by different scaling factors, thus obtaining at least two rectangular regions with different field-of-view ranges. Another method is to first calculate the smallest rectangle that can enclose all 8 points in the first set of keypoints, and then, using the center of this smallest rectangle as the center, expand its width and height outwards by a scaling factor to obtain another rectangular region with a different field of view. All cropping operations must ensure that the coordinates of the cropped region do not exceed the boundaries of the original image; if they do, they are filled with black pixels. In this way, multiple local image regions covering the charging dock and its surrounding background of different ranges are obtained, each with a different field of view and contextual information relative to the charging dock.

[0025] In one example, assuming the bounding box has a width w of 160 and a height h of 200, a first local region with a width of 192 and a height of 240 will be generated, corresponding to an outward expansion of 0.2 times; a second local region with a width of 200 and a height of 250 will also be generated, corresponding to an outward expansion of 0.25 times. Simultaneously, the smallest rectangle enclosing the eight first keypoint groups will be found, assuming its width w... kp The value is 150, and the height is h. kpIf the value is 220, a third local region with a width of 195 and a height of 286 is generated, which corresponds to an outward expansion of 0.3 times. In this way, three local images with different viewpoints are obtained.

[0026] S4. Use the local localization model to perform local localization on each local image region to obtain the corresponding second key point group.

[0027] In this process, each local image region generated in step S3 is scaled to a fixed input size required by the local localization model, such as 256*256 pixels. These scaled local images are then sequentially fed into a pre-trained local localization model. This model is also a lightweight convolutional neural network, but it is specifically trained for high-precision localization of key points within local regions. The model infers from each local image and outputs the coordinates of eight key points within that local image's field of view. Since these coordinates are relative to the local image's coordinate system, they need to be mapped back to the original image's coordinate system based on the cropping position and scaling ratio of the local image on the original image. Thus, each input local image region produces a set of eight key points in the original image's coordinate system, forming a second set of key points. Ultimately, multiple sets of second key point sets are obtained, each representing a precise observation of the charging dock's key points from a specific field of view.

[0028] In one example, the local localization model has a more streamlined and focused structure. Its encoder can use the same backbone network as the global localization model, but its input size is fixed to a uniform scaling of the cropped region, such as 256 by 256 pixels. Since the model's task no longer involves object detection, its decoder retains only a keypoint regression branch. This regression branch has fewer network layers and fewer convolutional kernels, making it a lighter network than the global localization model. This structural design allows the model to focus on high-precision coordinate regression of fine features such as charging dock corners within local image regions.

[0029] The local localization model is trained using a strategy based on imperfect candidate regions to simulate real-world input during inference. First, the trained global localization model is used to infer the location of all sample images in the training set, generating bounding boxes and a first set of keypoints for each image. Then, according to the rules in step S3 of this scheme, several local image regions with random offsets and scale jitter are automatically generated. These regions do not perfectly align with the charging dock, but rather simulate the uncertainty of the global model's output. These local regions are then cropped from the original image and scaled to a fixed size, serving as the training input for the local localization model. The corresponding real keypoint coordinates are mapped based on the cropping and scaling transformations, serving as the training target. The model uses only the SmoothL1 loss function to calculate the coordinate deviation between the predicted keypoints and the mapped real keypoints. Through this training method, the local localization model learns how to stably regress accurate keypoint positions based on local texture and structural information, even when the input region is not perfectly aligned with the center of the charging dock.

[0030] S5. Merge the first key point group with multiple second key point groups, and optimize the key points of the fusion result to obtain the fused key point group.

[0031] First, preset weight values ​​are assigned to the first keypoint group and each of the second keypoint groups. Next, for each keypoint index, such as the top-left corner of the inner frame, the coordinates of that point in each weighted group are extracted, and a weighted average is calculated to obtain a new set of coordinates that integrates global and multi-path local information. After completing the weighted average of all eight points, an initial fused keypoint group is generated. To further improve the geometric rationality and positional accuracy of this keypoint group, keypoint optimization operations are performed on this initial fused keypoint group.

[0032] Keypoint optimization refers to the post-processing of the coordinates of multiple keypoints contained in the fusion result after obtaining the fusion result of the first keypoint group and multiple second keypoint groups, in order to improve the rationality of its geometric structure and the accuracy of the point positions. This optimization does not refer to changing the fusion algorithm or reallocating weights, but rather applying two independent constraints and fine-tuning methods to the already generated initial fusion point group. This ensures that the output result, while maintaining consistency with the original positioning information, better conforms to the inherent physical shape characteristics of the charging dock and more accurately matches the actual corner positions in the image. The optimized point group is the final fusion keypoint group output in step S5.

[0033] In one possible implementation, the specific implementation process of key point optimization includes two independent processing methods, which can be executed only one or both.

[0034] I. Shape Constraints Based on the Known Geometry of the Charging Stand. An internally pre-stored set of standard keypoint templates, obtained through high-precision calibration, defines the ideal geometric relationship between the four corner points of the inner frame and the four corner points of the outer frame of the charging stand from a standard frontal view, including the parallelism and right-angle characteristics of the rectangular border. The processor first uses the Umeyama algorithm to calculate the similarity transformation matrix that achieves optimal alignment between the standard template and the current fusion point group to be optimized. This matrix contains four parameters: rotation angle, uniform scaling ratio, and two-dimensional translation. Since similarity transformation only allows rotation, translation, and uniform scaling, and does not allow shearing or non-uniform deformation, the point group obtained after transforming the standard template using this matrix perfectly maintains the standard rectangular structure of the charging stand, while its overall pose remains consistent with the point group to be optimized. Subsequently, the transformed template point group and the point group to be optimized are weighted and averaged according to preset weights, typically 0.5 to 0.5, thus obtaining a set of structural constraint points that closely resemble the original positioning result while eliminating geometric distortion.

[0035] 2. Fine-tuning of keypoint positions using local image features. A grayscale image pre-generated during image acquisition is used. For each keypoint in the group to be optimized, a square local image patch with a side length W equal to 15 pixels is cropped from the grayscale image, centered on the current coordinates of that point. Within this image patch, the processor uses the Shi-Tomasi corner detection algorithm to extract corner features and selects the candidate corner point as the one with the closest Euclidean distance to the current keypoint. Subsequently, the cornerSubPix iterative algorithm is used to refine the candidate corner point's coordinates at the subpixel level. The iteration terminates when the maximum number of iterations is 10 or the step size is less than 0.03 pixels. After refinement, the processor calculates the Euclidean distance between the subpixel coordinates and the current keypoint. If the distance is less than the preset maximum allowable deviation of 3.0 pixels, the original keypoint coordinates are replaced with the subpixel coordinates to accurately attach the point to the true corner features in the image; otherwise, the original coordinates are retained to avoid incorrectly pulling the point to noise or non-target corners.

[0036] In one example, the weight of the first keypoint group is set to 0.15, and the weights of the three second keypoint groups are 0.15, 0.35, and 0.35, respectively. For the point at the top left corner of the inner frame, the x-coordinates of the four groups are 270, 272, 268, and 271, respectively. Therefore, its weighted average x-coordinate is 270 multiplied by 0.15, 272 multiplied by 0.15, 268 multiplied by 0.35, and 271 multiplied by 0.35, resulting in 269.8. After performing this operation on all eight points, a shape regularization based on a standard template and a fine-tuning based on image corners are performed on this new set of coordinates. Points that do not conform to the rectangular structure constraints are pulled back to reasonable positions, and points close to obvious image corners are snapped to them. Finally, a set of eight optimized coordinates is obtained, which is the fused keypoint group.

[0037] S6. Based on the first key point group, perform point-by-point consistency verification on the fusion key point group, and correct the key points that fail the verification to obtain the corrected key point set.

[0038] The credibility of each point in the fused keypoint group is verified. For a keypoint of a specific index, all coordinate values ​​of that point in the first keypoint group obtained in step S2 and all coordinate values ​​in the second keypoint group obtained in step S4 are collected. Then, the dispersion between these coordinate values ​​is calculated, for example, the standard deviation in the horizontal axis and the standard deviation in the vertical axis are calculated, and the square root of the sum of the squares of the two standard deviations is taken to obtain a value representing the degree of divergence. If the degree of divergence of a point exceeds a preset threshold, it indicates that there is a large dispute between the positioning results of each path, and the point is judged as an untrusted point. For untrusted points, the corresponding point in the first keypoint group is used for replacement and correction first, because the first keypoint group is the result obtained based on the whole map context and has global stability. However, if the point in the first keypoint group itself is also seriously deviated from the median of all points, it is abandoned and the median coordinates are used for correction instead. For points with a degree of divergence below the threshold, their coordinates in the fused keypoint group are kept unchanged. Finally, after point-by-point verification and selective correction, the coordinates of the eight keypoints obtained constitute the corrected keypoint set.

[0039] In one example, for the bottom right corner of the outer frame, its x-coordinates in the four sets of points are 370, 372, 380, and 420, and its y-coordinates are 290, 292, 295, and 330. The calculated standard deviation of its x-coordinate is approximately 20.3, and its standard deviation of its y-coordinate is approximately 16.4, resulting in a discrepancy of approximately 26.1 pixels. This exceeds the preset threshold of 3.0 pixels, therefore it is judged as an untrusted point. At this point, the coordinates of this point in the first keypoint group are checked, for example, x-coordinate 370 and y-coordinate 290. It is found that the deviation from the median x-coordinate of 376 and the median y-coordinate of 293.5 of the four sets of points is within an acceptable range. Therefore, the x-coordinate of 370 and the y-coordinate of 290 in the first keypoint group are used to replace the point in the merged keypoint group. If the value of the first keypoint group itself deviates too much from the median, for example, more than 4.5 pixels, then the median coordinates are used directly for correction.

[0040] This embodiment employs a two-stage global and local localization strategy, combining the advantages of overall image context understanding with multi-scale local detail observation to overcome the limitations of single-shot localization in terms of field of view and accuracy. By introducing key point optimization techniques such as geometric constraints and image feature fine-tuning, it ensures that the shape of the output point group conforms to the inherent structure of the charging dock and matches the real features of the image. Finally, consistency verification using multi-path localization results can intelligently identify and correct drift and errors of individual points, preventing overall localization failure. The entire method requires no additional hardware, which helps improve the stability and accuracy of recharge positioning in challenging scenarios such as strong light, shadow, and tilted viewing angles.

[0041] In one possible implementation, before fusing the first keypoint group with multiple second keypoint groups, the method further includes: S7. Perform keypoint optimization on the first keypoint group and / or at least one second keypoint group to obtain the optimized first keypoint group and / or optimized second keypoint group for fusion.

[0042] Before performing the fusion operation in step S5, at least one set of keypoints to be fused is optimized to improve the quality of each set of keypoints. The optimization can target only the first set of keypoints, only a few sets of second keypoints, or both the first and some second keypoints. By performing keypoint optimization, efforts are made to eliminate any unreasonable structural distortions within each set of keypoints and point offsets caused by image noise, resulting in better reliability and geometric consistency for each set of points fed into the fusion step.

[0043] The keypoint optimization process involves two independent processing methods, which can be performed individually or both. The first method utilizes the known geometry of the charging dock for shape constraints. A set of standard keypoint templates, obtained through high-precision calibration, is pre-stored. This template defines the ideal geometric relationship between eight keypoints of the charging dock in a standard frontal view. By calculating the similarity transformation relationship between the keypoint group to be optimized and this standard template, a transformation matrix is ​​obtained, including rotation angle, uniform scaling ratio, and two-dimensional translation. Next, this similarity transformation matrix is ​​applied to the standard template to generate a set of transformed keypoints that perfectly conform to the standard inner and outer frame geometry and whose overall posture is consistent with the keypoint group to be optimized. Finally, this set of transformed keypoints with standard shapes is mixed with the original keypoints to be optimized using a weighted average method, typically with a weight ratio of 0.5 to 0.5, resulting in a set of structural constraint points that closely resemble the original positioning result while eliminating geometric distortion.

[0044] The second processing method is to fine-tune the key point positions using local image features. The grayscale image pre-generated and saved in step S1 is used. For each key point in the group of points to be optimized, a local image patch of size W*W pixels is cropped from the grayscale image, centered on that point, where W is typically 15. Within this local image patch, a corner detection algorithm is first used to extract corner features, and the corner closest to the current key point is selected as a candidate. Next, sub-pixel level coordinate refinement is performed on this candidate corner point, using an iterative algorithm to improve its coordinate precision from integer pixel level to floating-point sub-pixel level. The iteration termination condition is typically set to a maximum of 10 iterations or a coordinate movement of less than 0.03 pixels. If the Euclidean distance between the refined subpixel coordinates and the current key point is less than the preset maximum allowable deviation, such as 3.0 pixels, then the original key point coordinates are replaced with the subpixel coordinates to make the point accurately snap to the real corner features in the image; otherwise, the original coordinates are retained to prevent the point from being mistakenly pulled to noise or non-target corners.

[0045] The third approach is to use cross-validation information between multiple local localization results for key point fine-tuning. When optimizing at least one set of second keypoint groups, since step S3 generates multiple local image regions with different views, theoretically, the localization of the same keypoint should be highly consistent among the various second keypoint groups. This characteristic can be utilized to perform cross-validation fine-tuning on each keypoint in the current second keypoint group to be optimized. Specifically, for a keypoint index, firstly, the coordinate values ​​of the other second keypoint groups at that index are collected, and the statistical center value of these coordinates, such as the median coordinate, is calculated. Then, the Euclidean distance between the current point to be optimized and this statistical center value is calculated. If the distance is less than a preset cross-validation threshold, such as 2.0 pixels, it indicates that the point is consistent with the localization results of other groups and has high confidence. The point can be fine-tuned towards the statistical center value, for example, by taking the weighted average of the point and the median coordinate. If the distance exceeds the threshold, it indicates that the point differs from the majority opinion. The point can be marked as a low-confidence point, and its weight can be reduced in subsequent fusion.

[0046] For example, we are optimizing the second keypoint group from the first local image region. For the keypoint at the top right corner of the inner frame, the current coordinates are x-coordinate 380, y-coordinate 152. Looking at the corresponding point coordinates from the second and third local image regions, they are x-coordinate 382, ​​y-coordinate 150 and x-coordinate 379, y-coordinate 151 respectively. The median coordinates of these three are x-coordinate 380, y-coordinate 151. The distance between the current point and the median coordinate is only 1.0 pixel, less than the 2.0 pixel threshold, indicating good consistency. Taking the weighted average of the current point and the median coordinate, with each weight set to 0.5, we obtain the optimized x-coordinate 380, y-coordinate 151.5. Conversely, if the distance between a point and the median coordinate reaches 5.0 pixels, far exceeding the threshold, then this point is marked as low confidence, and its preset weight will be halved in the weighted fusion step S5.

[0047] The fourth approach is to use the linear constraints of the charging dock's frame edge line for fine-tuning of key points.

[0048] The visual characteristic that the charging dock's border region appears as a continuous straight line segment in the image can be utilized to fine-tune the position of key points located on the border. Specifically, for the four corner points of the outer frame or the four corner points of the inner frame, each pair of adjacent corner points should correspond to an edge line of the charging dock's border in the image. First, on the grayscale image, along the direction of the line connecting two adjacent corner points, a certain width is taken on both sides of the line, for example, 3 pixels on each side, forming a strip-shaped region of interest. Within this strip-shaped region, an edge detection algorithm is used to extract the image's edge pixels, and a straight line fitting algorithm, such as the least squares method, is used to fit these edge pixels into a straight line segment. Next, the vertical distance from the current two key points to this fitted straight line segment is calculated. If the vertical distance from a key point to the fitted straight line segment is less than a preset distance threshold, for example, 2.0 pixels, then the key point is projected onto the fitted straight line segment along a direction perpendicular to the straight line segment, so that it falls precisely on the physical edge line of the charging dock's border. If the vertical distance exceeds the threshold, the original coordinates are retained.

[0049] In one example, optimization is being performed on the top-left and top-right corner points of the outer frame in the first keypoint group. The current coordinates of these two points are 250 x 130 and 390 x 128, respectively. A 6-pixel wide strip is extracted from the grayscale image along the line connecting these two points, and a straight line segment is fitted using edge detection and least squares. The equation is y = -0.01 * x + 132.5. The vertical distance from the top-left corner point to this straight line segment is calculated to be 1.8 pixels, less than the 2.0 pixel threshold. Therefore, this point is projected onto the line, and its optimized coordinates are 250 x 130.0. The vertical distance from the top-right corner point is 0.3 pixels, also less than the threshold, and its coordinates are updated to 390 x 128.6 after projection. This optimization precisely constrains the two corner points of the top edge of the outer frame to the same physical edge line, eliminating the vertical misalignment caused by network regression errors.

[0050] After optimization using at least one of the above processing methods, an optimized first keypoint group or an optimized second keypoint group is obtained. In the subsequent step S5, these optimized keypoint groups, instead of the original localization model's direct output values, will be used to perform weighted fusion.

[0051] For example, suppose we decide to optimize both the first keypoint group and the second keypoint group from the first local image region simultaneously. For the first keypoint group, geometric constraints are first applied. A pre-stored standard template is aligned to the points in this group using a calculated similarity transformation of 2 degrees rotation and 1.02x scaling. The transformed standard points are then blended with the original points with a weight of 0.5, correcting a slight imbalance issue at the four points of the inner frame. Next, local feature fine-tuning is performed. For the top-left corner of the inner frame, a corner point is detected within a 15x15 pixel grayscale neighborhood, and its sub-pixel coordinates are refined to 270.35 x 150.68 y. This sub-pixel coordinate is only 1.2 pixels away from the origin, less than the 3.0 pixel threshold, so the top-left corner of the inner frame is updated to 270.35 x 150.68 y. For the point at the bottom right corner of the outer frame, the sub-pixel coordinates of the nearest detected corner point in its neighborhood are 5.1 pixels away from the origin, exceeding the threshold. This indicates that there may be no clear corner point near this point due to reflection, so the original coordinates of this point are retained and not updated. After this round of optimization, the point position errors and structural deviations in the first keypoint group and the selected second keypoint group are effectively eliminated. Subsequently, these two optimized point sets, together with other unoptimized second keypoint groups, will enter the weighted fusion process in step S5.

[0052] By introducing a keypoint optimization step before fusion, this method significantly improves the basic quality of each keypoint group participating in the fusion. Geometric constraints ensure that each keypoint group better conforms to the inherent rectangular frame structure of the charging dock, effectively suppressing point distortion caused by occlusion or viewing angle tilt. Local feature fine-tuning enables keypoints to accurately capture and lock the true corner positions in the image, eliminating slight drift caused by lighting changes or texture blurring. Sending this pre-processed high-quality point group into the fusion stage can prevent erroneous information from being amplified during fusion, ensuring the accuracy and stability of the final localization result from the source.

[0053] In one possible implementation, when performing key point optimization, if the known geometry of the charging dock is used for shape constraint, the specific implementation process includes two sequential processing stages.

[0054] The first stage is similarity transformation alignment. A set of standard keypoint templates is pre-stored. These templates are obtained by high-precision calibration of the charging dock under a standard frontal view, using the coordinates of eight key points. The arrangement of these eight points is completely consistent with the order of the inner and outer frame corner points in the actual positioning process: upper left of the inner frame, upper right of the inner frame, lower right of the inner frame, lower left of the inner frame, upper left of the outer frame, upper right of the outer frame, lower right of the outer frame, and lower left of the outer frame. This set of template points defines the ideal geometric ratio relationship and rectangular structural constraints between the inner and outer frames of the charging dock. After obtaining a set of keypoints to be optimized, a point correspondence is established between this set of points and the standard keypoint templates. That is, the point of index 1 corresponds to the template point of index 1, the point of index 2 corresponds to the template point of index 2, and so on. Using these eight pairs of matching points, an optimal similarity transformation matrix is ​​calculated using the Umeyama algorithm. This matrix contains four parameters: rotation angle θ, uniform scaling ratio s, lateral translation tx, and longitudinal translation ty. The optimization objective of the Umeyama algorithm is to minimize the weighted mean square error between the standard template points and their corresponding points in the target point group after the similarity transformation. Since the similarity transformation only allows rotation, translation, and uniform scaling, and disallows shearing, non-uniform scaling, or perspective distortion, the transformed template points perfectly maintain the standard geometry of the charging dock's inner and outer frames, including the parallelism and right angles of the rectangles, while their overall position, orientation, and scale remain consistent with the target point group. The calculated similarity transformation matrix is ​​applied to each point of the standard keypoint template to generate a transformed keypoint group.

[0055] The second stage is weighted fusion. After obtaining the transformed keypoint group, it is not directly used to replace the point group to be optimized. Instead, a weighted average method is used to fuse the information of the two. For each of the eight keypoints, the weighted average of the coordinates of the point in the transformed keypoint group and the coordinates in the point group to be optimized is calculated. The weight coefficient α is used to balance the influence of the two parts, and the typical value of α is 0.5. The x-coordinate of the fused point is α multiplied by the x-coordinate of the transformed point plus 1 minus α multiplied by the x-coordinate of the point to be optimized, and the y-coordinate is α multiplied by the y-coordinate of the transformed point plus 1 minus α multiplied by the y-coordinate of the point to be optimized. When α is 0.5, the weights of the transformed point and the point to be optimized are each 50%, and the fusion result retains the perfect geometric structure constraints from the standard template while closely approximating the actual localization response of the network model to the image content. If more emphasis is needed on structural constraints in specific application scenarios, α can be adjusted towards 1.0; if more trust is needed in the network localization results, α can be adjusted towards 0.0. After completing point-by-point weighted fusion for all eight points, a set of keypoints with shape constraint optimization is obtained.

[0056] This technical solution effectively utilizes the standard geometry of the charging dock in the key point optimization stage. The Umeyama similarity transformation ensures that the transformation template can adaptively match the overall pose of the point group to be optimized while maintaining the rectangular parallelism and right angle constraints. The weighted fusion strategy achieves a balance between structural correctness and positioning sensitivity, so that the optimization result not only follows the inherent physical shape prior of the charging dock, but also retains the actual position information extracted by the network model from the image, thereby significantly improving the geometric stability and structural consistency of the key point group when facing tilted viewpoints or partial occlusion.

[0057] In one alternative implementation, when performing keypoint optimization, if the local features of the image are used to fine-tune the keypoint positions, the specific implementation process includes three stages: extracting local image patches, detecting and refining candidate corner points, and updating the point positions based on distance conditions.

[0058] The first stage is the extraction of local image patches. First, a grayscale image pre-generated and stored during the image acquisition stage is accessed. This grayscale image maintains the exact same pixel coordinate system as the original color image, so keypoint coordinates do not need to be converted between the two. For each keypoint in the current keypoint group to be optimized, a square local image patch is extracted from the grayscale image, centered on the current coordinates of that point. The side length of this image patch is denoted as W, typically 15 pixels. If the center point is too close to the image edge, causing the local image patch to extend beyond the grayscale image boundary, the excess portion is filled with black pixels to ensure a constant image patch size.

[0059] The second stage involves detecting and refining candidate corner points. A corner detection algorithm, specifically Shi-Tomasi corner detection, is executed within the local image patch. This algorithm calculates the gradient covariance matrix at each pixel of the image patch and uses the smaller eigenvalue of this matrix as the corner response value; a larger response value indicates a higher likelihood of a corner. A quality level threshold is set, typically 0.01, meaning candidate points with corner response values ​​below 1% of the maximum response value are directly eliminated. A minimum corner spacing parameter is also set to suppress the detection of overly dense clusters of corner points in the neighborhood. If one or more corner points are detected within the local image patch, the corner point with the closest Euclidean distance to the current keypoint to be refined is selected as a candidate corner point. If no corner point meeting the criteria is detected, the refinement process for that keypoint terminates, preserving its original coordinates.

[0060] For the selected candidate corner points, sub-pixel level coordinate refinement is further performed. The refinement algorithm employs an iterative corner refinement algorithm, specifically the cornerSubPix algorithm. This algorithm uses the integer pixel-level coordinates of the candidate corner point as initial values, sets a search window around it, and gradually converges the corner point coordinates to sub-pixel accuracy by iteratively calculating the image gradient and solving a system of equations with a zero dot product within the window. The termination condition for the iteration process is typically set to a maximum of 10 iterations or an iteration step size of less than 0.03 pixels. After refinement, the sub-pixel-level coordinates of the candidate corner point are obtained.

[0061] The third stage is point update based on distance conditions. The Euclidean distance between the sub-pixel coordinates and the current keypoint to be fine-tuned is calculated. The Euclidean distance is obtained by taking the square root of the sum of the squares of the differences in the x-coordinates and y-coordinates. This distance is compared with a preset maximum allowable deviation threshold, typically 3.0 pixels. If the distance is less than or equal to 3.0 pixels, it means the detected real corner point is close enough to the current keypoint, and the original keypoint coordinates can be replaced with sub-pixel coordinates to improve accuracy. If the distance is greater than 3.0 pixels, it means the candidate corner point deviates too much from the current keypoint, and the candidate corner point may belong to the texture of another object rather than the real corner point of the charging dock. In this case, the original keypoint coordinates are retained to avoid interference from erroneous corner points that could reduce positioning accuracy. After performing the above process on all 8 points in the keypoint group to be optimized, the keypoint fine-tuning based on local image features is completed.

[0062] This technical solution achieves high-precision fine-tuning of keypoint positions by combining Shi-Tomasi corner detection within local image blocks with cornerSubPix subpixel refinement. This processing method allows keypoints to automatically snap from the coarse positions obtained from network regression to actual corner features in the image, effectively eliminating point position drift caused by lighting changes, reflections, or texture blurring. Simultaneously, by setting a maximum allowable deviation threshold, it prevents keypoints from being incorrectly pulled towards noisy or non-target corners, ensuring that the optimization process only enhances reliable points without introducing additional errors, thereby improving the overall positioning accuracy and robustness of the final keypoint group.

[0063] In one possible embodiment, after obtaining the first set of keypoints and the bounding box output by the global localization model, multiple local image regions with different field of view ranges need to be extracted from the original image for subsequent processing by the local localization model. The generation rules for these local image regions include two independent optional methods; either method can be used alone, or both methods can be used simultaneously to generate at least two local image regions.

[0064] The first method involves expanding the bounding box as a reference. In step S2, the global positioning model outputs a bounding box representing the approximate location and range of the charging dock, in addition to outputting the first set of key points. This bounding box is defined by four parameters: the x-coordinate of the center point (cx), the y-coordinate of the center point (cy), the width (w), and the height (h). Using this bounding box as a reference, while keeping the center point coordinates unchanged, the width and height of the bounding box are expanded according to different dimensional expansion parameters. Specifically, at least two different expansion ratios are preset, for example, the first expansion ratio r1 is 0.20, and the second expansion ratio r2 is 0.25. For the first expansion ratio, the new width w1 is calculated as w multiplied by 1 plus 2 multiplied by r1, and the new height h1 is h multiplied by 1 plus 2 multiplied by r1. Since the center point remains unchanged, the x-coordinate of the top-left corner of this local image region is cx minus w1 divided by 2, the y-coordinate of the top-left corner is cy minus h1 divided by 2, the x-coordinate of the bottom-right corner is cx plus w1 divided by 2, and the y-coordinate of the bottom-right corner is cy plus h1 divided by 2. Similarly, for the second expansion scaling factor, w2 is calculated as w multiplied by 1 plus 2 multiplied by r2, and h2 is calculated as h multiplied by 1 plus 2 multiplied by r2, and the range of this local image region is determined in the same way. Due to the different expansion scaling factors, the two generated local image regions have different field of view ranges. The larger the expansion scaling factor, the more background context information around the charging dock is contained in the local image.

[0065] The second method expands the bounding box defined by the first set of keypoints as a reference. First, iterate through the coordinates of all eight keypoints in the first set, finding the minimum x-coordinate among all the possible x-coordinates. min and maximum value xmax and the minimum value y among all ordinates. min and maximum value y max By x min y min x max and y max The rectangular area defined by the four values ​​is the smallest axis-aligned bounding box that can enclose all eight points in the first keypoint group. The width of this bounding box is w. kp equals x max Subtract x min Height h kp equals y max Subtract y min The x-coordinate of the center point is cx kp equals x min Add x max Divide by 2, the ordinate of the center point is cy kp equals y min Add y max Divide by 2. Using this bounding box as a reference, keeping its center point unchanged, expand outward according to a preset expansion ratio r3, where r3 typically takes the value of 0.30. The width w3 of the expanded local image region is equal to w kp Multiply by 1, add 2, multiply by r3, height h3 equals h kp Multiply by 1, add 2, multiply by r3. The x-coordinate of the top-left corner of this local image region is cx. kp Subtract w3 and divide by 2, the top left ordinate is cy kp Subtract h3 and divide by 2.

[0066] After determining the rectangular range of each local image region using any of the methods described above, boundary constraint processing is required for the rectangular range. If any side of the rectangular range exceeds the boundary of the original image, for example, if the x-coordinate of the upper left corner is less than 0 or the y-coordinate of the lower right corner is greater than the image height, then the rectangular range is cropped to within the image boundary. The portion exceeding the image boundary is filled with black pixels in subsequent processing to ensure that the size of the local image region remains the fixed size obtained from the outward expansion calculation. Cropping is performed on the original image according to each rectangular range to generate corresponding local image regions. These local image regions have different field of view ranges due to different expansion parameters, providing multi-scale observation windows for subsequent local localization models.

[0067] In an optional implementation, when performing the fusion operation, it is first necessary to determine the sets of key points participating in the fusion and the preset weights corresponding to each set. The sets of key points participating in the fusion include the first key point group and all the second key point groups obtained in step S4, with a total number of N. The first key point group is denoted as P0, and the second key point groups are denoted as P1, P2, and so on, respectively. N-1Each group contains 8 key points, numbered from 1 to 8, corresponding to the 8 corner points of the inner and outer frames of the charging dock, respectively. There is a one-to-one correspondence between key points of the same index in each group.

[0068] Each keypoint set is assigned a preset weight coefficient, which is a real number greater than 0 and less than 1, and the sum of all N weight coefficients is 1. The specific values ​​of the weight coefficients are predetermined and stored in the system based on the field of view and localization reliability of each local image region through offline verification experience. Generally, the second keypoint group corresponding to the local image region obtained by expanding the keypoint bounding box, and the second keypoint group corresponding to the local image region obtained by expanding the bounding box by a larger expansion ratio, have higher localization stability because they contain more contextual information, and therefore are assigned higher weights; while the second keypoint group and the first keypoint group corresponding to the local image region obtained by expanding the bounding box by a smaller expansion ratio are assigned relatively lower weights. A typical weight allocation scheme is as follows: when there are three groups of second keypoints, the weight w0 of the first keypoint group is 0.15, the weight w1 of the second keypoint group obtained by expanding the bounding box by 0.20 is 0.15, the weight w2 of the second keypoint group obtained by expanding the bounding box by 0.25 is 0.35, and the weight w3 of the second keypoint group obtained by expanding the keypoint bounding box by 0.30 is 0.35.

[0069] After assigning weights, a point-by-point weighted average is performed. For each keypoint index i, i takes values ​​from 1 to 8, from P0, P1, P2 to P... N-1 Extract the two-dimensional coordinates corresponding to each index, and denote them as ( , ), ( , )until( , The coordinates of the point after fusion ( ). , The weighted average is calculated using the following formula: =(w0* +w1* +...+w N-1 * ) / (w0+w1+...+w N-1 ).

[0070] =(w0* +w1* +...+w N-1 * ) / (w0+w1+...+w N-1 ).

[0071] Since the sum of all weights is 1, the denominator of the above formula is equal to 1, which can be simplified to direct summation. After performing this operation on all 8 indices, a set of 8 weighted average keypoint coordinates is obtained. This set of points is the initial result after fusion. Then, the keypoint optimization step S5 will be entered to generate the final fused keypoint set.

[0072] In one possible embodiment, after obtaining the fused keypoint group, the credibility of each keypoint in the group needs to be independently evaluated, and points that fail the evaluation are selectively corrected. The entire process consists of three sequential stages: calculating dispersion, determining unreliable points, and replacement correction.

[0073] The first stage is calculating the dispersion. For each index i in the eight keypoint indices, where i ranges from 1 to 8, the coordinate values ​​of that index in all candidate sources are collected. Candidate sources include the first keypoint group and all second keypoint groups obtained in step S4, with a total of K candidate sources. The coordinates of each group at that index are represented by a series of two-dimensional points (x1, y1), (x2, y2), up to (x... K y K Calculate the mean of each of the K horizontal axes. x and standard deviation σ x and the mean of the K ordinates. y and standard deviation σ y The standard deviation is calculated by taking the square root of the sum of the squares of the differences between each item and its mean, divided by K. Based on this, the dispersion of the index keypoints is calculated. Dispersion is defined as the square root of the sum of the squares of the standard deviations of the x-axis and y-axis, i.e., spread equals σ. x squared plus σ y The square root of the square of the result. The larger the dispersion value, the more serious the discrepancy between the positioning results of different paths to the location of the point.

[0074] The second stage is to determine untrustworthy points. This involves comparing the spread of each keypoint with a preset spread threshold T. spread For comparison, T spread The typical value is 3.0 pixels. If the dispersion of a key point is greater than 3.0 pixels, it means that there is a large dispute among the candidate points in the group regarding the location of the point, and the point is marked as an untrusted point; if the dispersion is less than or equal to 3.0 pixels, it means that the opinions of the groups are relatively consistent, and the point is regarded as a trustworthy point. Its original coordinates in the fused key point group are retained unchanged and it does not enter the third stage of the correction process.

[0075] The third stage involves replacing and correcting untrusted points. For each key point deemed untrusted, the statistical center value at that index is first calculated from all K candidate points. The statistical center value is obtained by taking the median of the K x-coordinates, which is m. x The median of the K ordinates is m. y This forms the median coordinate point (m) x m y The median is naturally robust to extreme deviations and reflects the central tendency of most opinions. Meanwhile, the coordinates corresponding to this index in the first keypoint group are extracted and denoted as ( , ). Calculate the Euclidean distance between the point in the first keypoint group and the statistical center value, i.e. minus m x square plus minus m y The square root of the square is then taken. This distance is compared with a preset deviation threshold, which is typically 1.5 times the discrete threshold, i.e., 1.5 multiplied by 3.0 equals 4.5 pixels.

[0076] If the distance between the corresponding point in the first keypoint group and the statistical center value is less than or equal to 4.5 pixels, it indicates that the location of that point in the first keypoint group is basically consistent with the majority opinion and is qualified as a stable prior for backtracking. In this case, the coordinates of that point in the first keypoint group are used to replace the current coordinates of the untrusted point in the fused keypoint group.

[0077] If the distance between the point corresponding to the first keypoint group and the statistical center value is greater than 4.5 pixels, it indicates that the positioning of the point by the first keypoint group itself deviates significantly from the majority opinion. Therefore, the first keypoint group lacks credibility at that point and cannot be used as a baseline for backtracking. In this case, backtracking to the first keypoint group should be abandoned, and the statistical center value, i.e., the median coordinates (m), should be used instead. x m y ), to replace the coordinates of the untrusted point in the fusion keypoint group.

[0078] After completing the above three stages for all eight key points, we obtain the corrected set of key points after point-by-point consistency verification and selective correction.

[0079] Furthermore, when performing replacement corrections on untrusted points, the corresponding points in the first keypoint group are not directly used. Instead, the credibility of the corresponding points in the first keypoint group is first evaluated, and then the final coordinate source to be used for correction is determined based on the evaluation results. This process includes three stages: calculating the deviation, comparing thresholds, and selecting the correction source.

[0080] The first stage is to calculate the bias. First, determine the statistical center value at index i of the untrusted point. The statistical center value is obtained by taking the median of the components of the coordinates of all candidate sources at this index. Candidate sources include the first keypoint group and all second keypoint groups. Let the total number of candidate sources be K, and the x-coordinates of these K points at this index be x1 to x2. K The vertical coordinates are y1 to y2 respectively. K After sorting these x-coordinates, take the median m. x After sorting the ordinates, take the median m. y Therefore, the coordinates of the statistical center point M are m. x and m y Next, extract the point corresponding to index i from the first keypoint group, and denote it as... Its coordinates are and Calculation points The Euclidean distance between the point and the statistical center M is used as the deviation value. The formula for calculating the deviation D is: D = ... minus m x squared plus minus m y Take the square root of the sum of the squares of the two numbers.

[0081] The second stage is the threshold comparison. The calculated deviation D is compared with a preset deviation threshold T. dev The comparison is performed. The typical value of T_dev is 4.5 pixels, which is about 1.5 times the preset discrete threshold. The basis for this setting is that if the deviation between the positioning result of the first keypoint group at this point and the median opinion of each candidate point is within this range, then the positioning of the first keypoint group can be considered to still have reference value; if it exceeds this range, it means that the first keypoint group itself is also an outlier at this point and should not be used as a basis for correction.

[0082] The third stage is selecting the correction source. This stage is handled in two ways based on the deviation comparison results.

[0083] Scenario 1: If the deviation D is greater than 4.5 pixels. In this case, the corresponding points in the first keypoint group... The location deviates too much from the consensus center formed by the majority of candidate points, indicating that the positioning results of the first keypoint group at that location are unreliable. If this is forcibly used... Performing a rollback correction might introduce even larger errors. Therefore, the strategy of rolling back to the first keypoint group was abandoned. Instead, the statistical center point M, composed of the medians of all candidate points, was used as the correction source. The coordinates of M were used to replace the current coordinates of the untrusted point in the fused keypoint group.

[0084] Scenario 2: If the deviation D is less than or equal to 4.5 pixels. In this case, the corresponding points in the first keypoint group... The location of the keypoints largely matches the majority opinion of the candidate points, with deviations within an acceptable range. The first keypoint group, obtained from the global localization model within the entire map context, exhibits good global stability. Therefore, the localization results of the first keypoint group are trusted and used. The coordinates of the untrusted point in the fusion keypoint group are replaced to achieve rollback correction.

[0085] Each critical point deemed untrustworthy is processed independently through the three stages described above, with each point's correction decision remaining independent, ultimately resulting in a complete set of corrected critical points.

[0086] In one possible implementation, after completing the point-by-point consistency verification and correction in step S6, a set of corrected key points is obtained. However, in some extremely difficult scenarios, such as when there is strong reflection in the inner frame area of ​​the charging dock, when it is partially obscured by weeds, or when it is at a severely tilted viewing angle, relying solely on the aforementioned multi-path positioning fusion and consistency verification methods may still not be able to completely restore the precise position of the inner frame corner points. To address this, an inner contour rescue function is also provided. This function utilizes the known geometric relationship between the outer and inner frames of the charging dock to independently extract and replace the inner frame corner points in the fused key point set when preset trigger conditions are met. The specific implementation of this function includes four stages: trigger condition judgment, perspective correction, inner frame corner point extraction, and back projection replacement.

[0087] The first stage is trigger condition judgment. Before performing inner contour rescue, it is first checked whether the current sample meets the preset rescue trigger conditions. The rescue trigger conditions include two independent judgment criteria, and the rescue process is triggered if either one is met. The first judgment criterion is to count the number of key points that actually triggered replacement correction in step S6, that is, the number of points that were judged as unreliable points and whose coordinates have been replaced. If this number reaches or exceeds the preset number threshold, such as 2, it indicates that there is a large range of multi-path positioning divergence in the current sample, and the overall reliability of the fused key point group is questionable, so inner contour rescue needs to be initiated. The second judgment criterion is to calculate the angle deviation between the quadrilateral formed by the four corner points of the outer frame in the fused key point group and the standard rectangle. Calculate the angle between each of the four sides of the outer frame and the horizontal or vertical direction, and compare it with the 0-degree or 90-degree direction of the standard rectangle. If the average angle deviation of the four sides exceeds the preset angle threshold, such as 15 degrees, it indicates that there is a significant systematic distortion in the current sample, so inner contour rescue also needs to be initiated. If neither of the two criteria is met, skip the inner contour rescue and proceed directly to the next step.

[0088] The second stage is perspective correction. When the rescue trigger condition is met, perspective correction is performed on the original grayscale image using the corner points of the outer frame in the fused keypoint group. The corner points are the four points corresponding to indices 5 to 8 of the eight keypoints, namely the upper left, upper right, lower right, and lower left of the outer frame. Using the current coordinates of these four points as the source points and a preset standard rectangle as the target points, a 3*3 perspective transformation matrix M is calculated. The coordinates of the four corner points of the standard rectangle are: upper left corner x-coordinate 0, y-coordinate 0, upper right corner x-coordinate W. ref The vertical coordinate is 0, and the horizontal coordinate of the lower right corner is W. ref , ordinate H ref The lower left corner has an x-coordinate of 0 and a y-coordinate of H. ref W ref and H ref For example, W is a constant preset according to the typical proportions of the charging dock frame. ref Set H_ref to 180 pixels. The perspective transformation matrix M can map the outer frame area in the original image that may have perspective distortion into a rectangle from a standard front view perspective. Using the calculated matrix M, perform a perspective transformation on the original grayscale image to generate a standard front view based on the charging dock's outer frame.

[0089] The third stage is the extraction of inner frame corner points. In the generated front view, the inner frame area of ​​the charging dock exhibits different grayscale characteristics from the outer frame and background, typically appearing as a distinct dark area. This visual characteristic is utilized for inner frame region detection. First, image segmentation techniques, such as threshold-based binarization, are applied to the grayscale image of the front view to extract dark connected components with grayscale values ​​lower than the surrounding areas. These connected components are then filtered by area, with the area selection range preset based on the standard ratio between the inner and outer frames; for example, the lower limit is set to 0.3 multiplied by W. ref Multiplied by H ref The maximum area is 0.7 multiplied by W. ref Multiplied by H ref For connected components that pass the area screening, further shape screening is performed to determine whether their outer contour can be well approximated by a quadrilateral. The contour of the connected component is extracted and approximated by a polygon. If the approximation result is a quadrilateral, then this quadrilateral is identified as the inner frame region. The four vertices of this quadrilateral are the inner frame corner points in the standard front view coordinate system, and these four points are denoted as Q. front The numbers correspond to the top left, top right, bottom right, and bottom left of the inner frame, respectively.

[0090] The fourth stage is back projection replacement. Using the perspective transformation matrix M obtained in the second stage, the inverse matrix of M is calculated. Using the inverse matrix, the four inner frame corner points Q in the standard front view coordinate system are... frontEach point in the image is mapped back to the coordinate system of the original image to obtain the reconstructed inner frame corner coordinates. This mapping process is achieved by multiplying the point coordinates in the front view by the inverse matrix. These four back-projected inner frame corner coordinates are then used to directly replace the original inner frame corner coordinates corresponding to indices 1 to 4 in the fusion keypoint group, while the outer frame corner coordinates in the fusion keypoint group remain unchanged. After the replacement, the updated fusion keypoint group with the reconstructed inner contour is obtained.

[0091] This technical solution introduces an inner contour rescue mechanism based on outer frame perspective correction, providing a geometric verification and repair path independent of the neural network for locating the corner points of the charging dock's inner frame. When there are serious discrepancies in multiple positioning results or significant perspective distortion in the samples, the image is geometrically normalized using the stable structural information provided by the outer frame. Then, the inner frame contour is directly extracted based on the image's grayscale features from a frontal viewpoint, effectively restoring the positions of inner frame corner points that are occluded, reflected, or blurred. This rescue mechanism is triggered only when necessary and will not intervene when the normal positioning results are reliable, balancing overall positioning efficiency with positioning robustness in extreme scenarios, significantly improving the system's recharging success rate in complex courtyard environments.

[0092] Furthermore, when determining whether to initiate the inner contour rescue process, two independent triggering conditions are used. The rescue process is triggered as soon as either condition is met. These two conditions assess the positioning quality of the current sample from two dimensions: the degree of point divergence and the degree of overall geometric distortion.

[0093] The first trigger condition is based on the number of keypoints corrected during point-to-point consistency verification. During the point-to-point consistency verification process in step S6, the dispersion of each keypoint in the fused keypoint group is calculated, and keypoints with dispersion exceeding a preset dispersion threshold are identified as untrusted points. Replacement correction is then performed on these untrusted points. After all eight keypoints have been verified and corrected, the number of keypoints that actually triggered the replacement correction operation is counted; that is, the number of keypoints identified as untrusted due to dispersion exceeding the threshold and replaced with other coordinate sources. This number is denoted as N. back N back The number is compared with a preset quantity threshold, which typically has a value of 2. If N back A value greater than or equal to 2 indicates a significant discrepancy in the multipath localization results of at least two keypoints in the current sample. This suggests a substantial decrease in the overall reliability of the fused keypoint group, potentially indicating systemic observation difficulties such as local occlusion, strong reflections, or excessively tilted viewing angles. At this point, the first trigger condition is met, initiating the inner contour rescue process. If N backA value less than 2 indicates that the opinions on the positioning of most key points are relatively consistent, the overall credibility of the fused key point group is high, and the first triggering condition is not met.

[0094] The second trigger condition is based on the angular deviation between the shape formed by the corner points of the outer frame in the fusion keypoint group and the standard rectangle. The outer frame of the charging dock is a standard rectangular structure in the physical world. Therefore, in a normally captured image, the four corner points of the outer frame should form a quadrilateral that conforms to the rules of perspective projection. However, when the shooting angle is too tilted or the positioning result is severely distorted, the directions of each side of this quadrilateral will have a significant angular deviation from the standard rectangle. The four outer frame corner points corresponding to indices 5 to 8 in the fusion keypoint group are extracted: upper left P5, upper right P6, lower right P7, and lower left P8. The direction vectors of the four sides of the outer frame are calculated sequentially: the vector from P5 to P6 is the top, the vector from P7 to P8 is the bottom, the vector from P5 to P8 is the left, and the vector from P6 to P7 is the right. For the top and bottom edges, calculate their angles with the horizontal direction; theoretically, the two horizontal edges should have an angle of 0 degrees with the horizontal direction. For the left and right edges, calculate their angles with the vertical direction; theoretically, the two vertical edges should have an angle of 0 degrees with the vertical direction. The angle of each edge is calculated using the arctangent function of the direction vector, with a value ranging from 0 to 90 degrees. The average of the four calculated angles is taken to obtain the average angular deviation θ. avg If θ avg If the angle exceeds a preset threshold, such as 15 degrees, it indicates that the quadrilateral formed by the current outer frame has significantly deviated from the reasonable perspective distortion range, resulting in structural distortion. The second trigger condition is met, initiating the inner contour rescue process. If θ avg If the angle is less than or equal to 15 degrees, it means that the shape of the outer frame basically meets the expectations, and the second triggering condition is not met.

[0095] The two triggering conditions above are logically ORed. The two conditions are judged in turn. Once either condition is met, the judgment is terminated and the rescue process is started. Only when neither condition is met will the inner contour rescue be skipped and the inner frame corner points in the fusion key point group remain unchanged.

[0096] In one possible embodiment, after obtaining the corrected set of key points or completing the inner contour rescue, a special correction function for skewed samples is also included. Under certain specific viewing conditions, such as when the lawnmower robot is tilted relative to the charging dock with the left side higher than the right or vice versa, the positions of the inner frame corner points relative to the outer frame corner points will exhibit a systematic offset pattern. This offset may still persist even after the aforementioned geometric constraints and consistency checks. The skewed sample correction function utilizes the known standard geometric relationship between the inner and outer frames of the charging dock to directionally correct key points that conform to a preset abnormal pattern. The implementation of this function includes three stages: abnormal pattern detection, ideal key point calculation and replacement.

[0097] The first stage is anomaly pattern detection. A set of anomaly pattern definitions obtained from offline calibration is pre-stored. Each anomaly pattern describes the geometric anomaly characteristics of the inner frame corner point relative to the outer frame corner point under a specific tilted viewpoint. These anomaly pattern definitions are based on the offset vector from the inner frame corner point to the corresponding outer frame corner point. Each inner frame corner point typically corresponds to an outer frame corner point on the same side; for example, the upper left corner of the inner frame corresponds to the upper left corner of the outer frame, the upper right corner of the inner frame corresponds to the upper right corner of the outer frame, and so on. For each pair of corresponding points, the offset vector from the inner frame corner point to the outer frame corner point is calculated. This vector contains two attributes: direction angle and magnitude. The direction angle is the angle between the vector and the horizontal or vertical direction, and the magnitude is the Euclidean distance from the inner frame corner point to the outer frame corner point. The definition of anomaly patterns includes two constraints: the direction angle deviation exceeds a preset angle threshold, for example, the direction angle deviation from the vector in the standard template exceeds 20 degrees; and the magnitude is lower than a preset proportion of the standard magnitude, for example, lower than 0.8 times the standard magnitude. The offset vector direction and magnitude in the standard template are obtained by calibrating the standard orthographic sample and are pre-stored in it.

[0098] For each of the four pairs of inner and outer frame points in the current keypoint group, the offset vector is calculated, and the calculation results for each group are matched with all pre-stored anomaly patterns. If the offset vector of a certain pair of corresponding points satisfies all the constraints of a certain anomaly pattern, then the corner point of the inner frame is determined to have experienced this type of anomaly, and the point is marked as a point to be corrected. Different anomaly patterns correspond to different tilt directions. For example, the "left-top-low" pattern corresponds to the situation where the magnitude of the offset vector from the upper left corner of the inner frame to the upper left corner of the outer frame is too short and the direction is abnormal under a left-high-right-low view. The "right-bottom-low" pattern corresponds to a similar anomaly from the lower right corner of the inner frame to the lower right corner of the outer frame under a right-high-left-low view. A single detection may match zero, one, or more anomaly patterns, and the corresponding locations are marked one by one.

[0099] The second stage is the calculation of ideal key points. For each inner frame corner point marked as needing correction, its coordinates are not directly modified. Instead, the ideal position of the point is recalculated based on the standard geometry of the charging dock. The calculation process relies on reference key points that have not experienced any anomalies and pre-stored standard geometric information. The reference key points are usually selected from the four corner points of the outer frame in the current key point group, because the outer frame is generally more stable and reliable in the image than the inner frame, and skewing mainly affects the internal offset relationship of the inner frame relative to the outer frame. Based on the quadrilateral formed by the four corner points of the outer frame and the proportional relationship between the inner and outer frames of the standard charging dock, the coordinates of the ideal point of the inner frame are calculated.

[0100] The specific calculation method involves using the corner points of the outer frame to calculate the boundary range of the outer frame, including its width and height. Standard geometry defines the normalized position of the inner frame within the outer frame's range; for example, the ratio of the distance from the left boundary of the inner frame to the left boundary of the outer frame to the width of the outer frame is r. left The ratio of the distance between the upper boundary of the inner frame and the upper boundary of the outer frame to the height of the outer frame is r. top The ratio of the inner frame width to the outer frame width is r. width The ratio of the inner frame height to the outer frame height is r. height These proportional parameters are pre-stored after precise measurements of a standard charging dock. For each inner frame corner point to be corrected, its ideal coordinates in the outer frame coordinate system are calculated based on its specific index, using the outer frame boundary and the corresponding standard proportional parameters. For example, for the upper left corner point of the inner frame, its ideal x-coordinate is the left boundary of the outer frame plus r. left *Outer frame width, ideally the ordinate is the top boundary of the outer frame plus r. top *Outer frame height.

[0101] The third stage is replacement. Using the calculated ideal keypoint coordinates, the coordinates of the inner frame corner points to be corrected at the corresponding indices in the fused keypoint group are replaced point by point. Replacement only applies to points marked as conforming to the abnormal pattern; other unmarked inner frame corner points and all outer frame corner points remain unchanged. At this point, the correction of the skewed sample is complete.

[0102] This technical solution accurately identifies systematic skew patterns caused by specific tilt angles by detecting anomalies in the direction and magnitude of the offset vector between the inner and outer frame corners. After matching the abnormal pattern, the outer frame corners without anomalies are used as stable references, and the ideal inner frame point position is calculated based on the standard geometric proportions of the charging base's inner and outer frames, thus achieving directional correction of skew deviations. This correction mechanism, together with the aforementioned geometric constraints and consistency checks, forms a complementary three-layer positioning guarantee system. Geometric constraints handle general distortions, consistency checks handle local random drifts, and skew correction handles systematic offsets under specific angles. This three-layer progressive guarantee ensures that the final output keypoint set maintains high structural accuracy and positioning precision under various complex shooting conditions.

[0103] See Figure 3A The diagram shown is another flowchart illustrating the recharging and positioning method for a lawnmower robot provided in this application embodiment, which specifically includes the following steps: Step 1: Image Acquisition and Input Preprocessing.

[0104] The lawnmower's onboard camera captures environmental RGB color images, including the charging dock. The original image coordinate system is retained as the final keypoint output coordinate system. Simultaneously, the input image is converted to grayscale (I_gray) for sub-pixel corner extraction in the subsequent local corner fine-tuning stage.

[0105] Step 2: First stage - Full-image outer contour localization.

[0106] The entire image is input into the first-stage full-image object detection and keypoint regression network (Stage-1 network). The network outputs: the bounding box of the charging dock target, B0=(cx, cy, w, h). The first set of eight initial keypoint values, P0_raw (corresponding to the coordinates of the eight corner points of the inner and outer bounding boxes), is then applied. Subsequently, the same geometric regularization and local corner fine-tuning process as in subsequent steps is performed on P0_raw to obtain the first-stage stable prior point P0.

[0107] Step 3: Generation of multiple candidate ROIs.

[0108] Based on the target bounding box B0 and keypoint bounding boxes output from the first stage, three sets of candidate cropping regions (ROIs) are generated for the second stage of fine localization: Candidate ROI-1: Expanded by 20% based on the target bounding box B0.

[0109] Candidate ROI-2: Expanded by 25% based on the target bounding box B0.

[0110] Candidate ROI-3: Minimum bounding box with 8 key points expanded by 30%.

[0111] Ensure that each ROI does not extend beyond the image boundaries.

[0112] Step 4: Second Stage - Precise ROI Key Point Identification.

[0113] The three candidate ROI images are scaled to a fixed size and fed into the Stage-2 keypoint regression network for inference. Each network outputs eight keypoints relative to its ROI coordinate system. These coordinates are then mapped back to the original image coordinate system, forming three sets of candidate keypoints: P_c1, P_c2, and P_c3.

[0114] Step 5: Post-processing of candidate keypoints.

[0115] For the first-stage stable prior point P0 and the three second-stage candidate points P_c1, P_c2, and P_c3, the same post-processing is performed, including: Geometric structure regularization based on standard eight-point template: The optimal similarity transformation between candidate points and standard template is calculated using the Umeyama algorithm. The transformed template points are then weighted and mixed with the original candidate points (α=0.5) to obtain structural constraint points.

[0116] Corner fine-tuning based on local image patches: On the grayscale image I_gray, a local image patch is cropped with each keypoint as the center. The Shi-Tomasi algorithm is used to detect corners and perform sub-pixel refinement (cornerSubPix). If the distance between the refined corner and the original point is within the allowable range (e.g., 3 pixels), the coordinates are updated.

[0117] The post-processed point sets are denoted as P0_post, P_c1_post, P_c2_post, and P_c3_post, respectively.

[0118] Step 6: Weighted fusion of multiple candidates.

[0119] The candidate keypoints after post-processing from the four paths are then fused using a weighted average. The weight allocation is based on offline experience: P_c2_post and P_c3_post have the highest weights (0.35 each), while P_c1_post and P0_post have lower weights (0.15 each). After fusion, P_fused is obtained, and geometric regularization and local corner point fine-tuning are performed again to obtain P_fused_final.

[0120] Step 7: Point-level rollback and outlier rescue in candidate divergence perception.

[0121] Calculate the degree of dispersion (spread_i) of the four candidate point sets (P0_post, P_c1_post, P_c2_post, P_c3_post) at each key point.

[0122] Point-level rollback: If the spread_i of a point is greater than a threshold (e.g., 3.0 pixels), then the candidate for that point is considered to have large divergence and is unreliable. In this case, rollback is only performed on that point, and the current fused point p_fused_final_i is replaced with the first-stage stable prior point p0_i.

[0123] Outlier rescue: Before backtracking, determine whether the first-stage point p0_i itself is an outlier (i.e., whether its distance from the four candidate medians is too large). If so, abandon the backtracking to the first-stage point and instead use the four candidate medians m_i as the final value of that point.

[0124] Step 8: Inner contour rescue and correction of skewed samples.

[0125] This step is initiated when the number of trigger point-level backoff points is large (e.g., N_back≥2), or when the shape of the merged outer frame deviates too much from the standard rectangle.

[0126] Perspective straightening: Calculate the perspective transformation matrix using the four points of the merged outer frame to transform the grayscale image I_gray into a standard front view I_front.

[0127] Inner contour extraction: In the front view I_front, extract the contour of the dark connected region, filter and approximate to obtain the vertex Q_front of the inner frame quadrilateral.

[0128] Back projection: Using inverse perspective transformation, Q_front is mapped back to the original coordinate system to obtain the four points Q_rescue of the rescued inner frame, and the corresponding inner frame points in the original fusion point are replaced.

[0129] Skewed sample correction: If a predefined abnormal pattern is matched (such as the upper left being too low), the current point is directly replaced with the calculated ideal inner frame point according to the offline calibration correction mapping table.

[0130] Step 9: Output the final fine-tuning results.

[0131] The output is the final, stable coordinates of the eight key points after all the above steps. This result can be directly used to calculate the center position of the charging dock, the attitude angle of the outer frame, etc., for use by the downstream recharge control logic.

[0132] Step 10: Model Training.

[0133] To obtain the Stage-1 and Stage-2 networks, model training is required: Dataset: Images of charging docks in various environments were collected, and the target bounding boxes and coordinates of 8 key points were manually annotated.

[0134] Loss function: The Stage-1 network uses a weighted sum of the bounding box IoU loss and the keypoint SmoothL1 loss; the Stage-2 network uses only the keypoint SmoothL1 loss. A shape regularization loss is also introduced to strengthen the rectangular constraints.

[0135] Training strategy: First, train the Stage-1 network independently; then use Stage-1 to infer jittery ROIs from the training samples to train the Stage-2 network, so that it can adapt to the situation of imperfect ROI alignment.

[0136] Step 11: Verification and Evaluation (Results Output).

[0137] The entire process is evaluated on the validation set, for example, by outputting an error index card (including average error, median error, P95 error, maximum error, etc.) to quantitatively demonstrate the accuracy and robustness of the method, and by showing a comparison of the localization results for typical and difficult samples.

[0138] See Figure 3B The image shown is a schematic diagram of the error index card provided in an embodiment of this application.

[0139] The error index card is a set of technical charts quantifying the positioning performance accuracy of this application, obtained through testing on a validation set of 89 images (containing 712 key points). The charts, presented in the form of bar charts, box plots, or clear numerical lists, display the following four key pixel error statistics: Average error: approximately 0.763 pixels, which is the average of the prediction deviations of all keypoints and reflects the overall accuracy level.

[0140] Median error: approximately 0.677 pixels, which is the median of half the predicted error values. This value is not sensitive to extreme errors and shows the typical accuracy of the method in most cases.

[0141] P95 error: approximately 1.605 pixels, indicating that 95% of keypoint prediction errors are below this value, which intuitively demonstrates the algorithm's high stability and consistent good performance in the vast majority of scenarios.

[0142] Maximum error: Approximately 4.762 pixels, representing the largest single-point error occurring in the validation set, used to demonstrate the performance boundary of the method under the most unfavorable conditions.

[0143] These data collectively demonstrate that this application achieves sub-pixel level positioning accuracy, possesses extremely high engineering practical value, and solves the problems of key point jitter, offset, and distortion mentioned in the background technology.

[0144] join Figure 3C As shown, Figure 3CThis is a diagram showing the final detailed positioning result provided in the embodiments of this application.

[0145] Figure 3C It is a direct visualization of the positioning effect of this application, which usually includes one or more images of the lawnmower recharging scene taken under different common lighting and background conditions (such as normal lighting), and is presented on the image together with the final eight key point output after being processed by the complete technical solution of this application.

[0146] Figure 3C Different markers (such as colored solid dots or sequentially numbered markers) clearly indicate the precise positions of the four corner points of the inner frame and the four corner points of the outer frame of the charging dock on the image. These markers are precisely overlaid on the physical corner points of the inner and outer frames of the charging dock, intuitively demonstrating that this technical solution can achieve accurate and stable key point detection and positioning in normal scenarios, meeting the basic technical goal of fine positioning.

[0147] See Figure 3D The diagram shown illustrates the post-processing and key point correction procedures provided in an embodiment of this application.

[0148] Figure 3D This aims to explain the operational details and effects of post-processing of candidate key points and rescue and skew correction of inner contours, and is a concrete example of the technical solution.

[0149] Figure 3D The text uses a local image of a charging dock with initial prediction points as an example, and compares it using different graphic symbols and connection relationships, for example: Yellow rings: may represent the raw, unprocessed prediction points (P0_raw or Pk_raw) obtained from the initial direct regression of the network.

[0150] A cyan cross may indicate a pixel-level refined point obtained after "corner point fine-tuning based on local image patches" and correction using sub-pixel refinement techniques (such as the Shi-Tomasi algorithm and the cornerSubPix function).

[0151] Colored solid points: These may represent a set of points that have been adjusted by "geometric regularization based on a standard eight-point template" and have a more regular structure (e.g., the inner and outer frames are closer to the ideal rectangle). They are also likely the final points resulting from the fusion of fine-tuning and regularization.

[0152] This intuitive diagram illustrates how the post-processing module iteratively corrects an initial set of measurement points that may have geometric distortions or pixel-level offsets into a final stable, accurate, and regularly shaped set of key points by combining mathematical geometric constraints (similar transformation to standard shape) and low-level image feature analysis (extracting precise corner points from the grayscale image).

[0153] See Figure 3E The image shown is a comparison chart of GT / Stage 1 / Final for the difficult sample provided in this application embodiment.

[0154] Figure 3E This provides strong evidence of the application's validity and resistance to interference. It presents three key point localization results side-by-side on the same challenging charging scenario image (such as images with strong light and shadow contrasts, severe partial obstruction, and significant tilt of the charging dock), providing a direct comparison: GT: "Ground Truth" refers to the precise, human-defined true coordinates of ideal key points. It is usually represented by markers marked with an "×" or a specific color, serving as the baseline truth for technical effects.

[0155] Stage 1: Only the keypoint results predicted by the first-stage network are used. The results may exhibit a variety of undesirable states: such as a significant shift in the overall value relative to the ground truth (inaccurate global detection box), distortion of the overall structure of the eight keypoints (tilt or deformation), or severe drift of one or two keypoints.

[0156] Final: The final key point result output after applying the complete technical solution of this application (including second-stage fine localization, multi-path fusion, post-processing and possible rescue mechanisms). This result will be very close to or completely coincide with the GT point position, and the eight points will maintain a regular geometric relationship.

[0157] The purpose of this figure is to demonstrate, through comparison under difficult scenarios, that the traditional single-stage method (Stage 1) is overwhelmed under complex conditions, and its output may be unreliable for high-precision refilling; while the two-stage fusion and multi-level post-processing / rescue framework of this application can effectively correct these biases and errors, and the final output is very close to the accurate result of GT, thus proving the effectiveness of this scheme in surpassing the baseline scheme in complex environments.

[0158] Please see Figure 4 As shown, in one embodiment, a recharging and positioning device for a lawnmower robot is provided, the device comprising: Acquisition module 401 is used to acquire an image including the charging dock; Global positioning module 402 is used to perform global positioning on the image using a global positioning model to obtain the first key point group of the charging dock; Extraction module 403 is used to extract multiple local image regions with different fields of view from the image based on the first key point group; The local positioning module 404 is used to perform local positioning on each of the local image regions using a local positioning model to obtain the corresponding second key point group. The fusion module 405 is used to fuse the first key point group with multiple groups of the second key point group, and to optimize the key points of the fusion result to obtain a fused key point group. The correction module 406 is used to perform point-by-point consistency verification on the fused key point group based on the first key point group, and to correct the key points that fail the verification, so as to obtain a corrected key point set.

[0159] For further details regarding the implementation of the above-mentioned technical solutions by each module in the recharging and positioning device of the lawnmower robot, please refer to the description of the recharging and positioning method of the lawnmower robot provided in the above-mentioned embodiments of the invention, which will not be repeated here.

[0160] In one embodiment, a lawnmower robot is provided, the internal structure of which can be shown in the following diagram: Figure 5 As shown, the lawnmower robot includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the lawnmower robot is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the methods described in any of the foregoing embodiments of this application.

[0161] This application also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the methods described in any of the foregoing embodiments of this application.

[0162] This application also provides a chip for executing instructions, which is used to perform the methods described in any of the preceding embodiments as performed by the lawnmower robot in any of the preceding embodiments of this application.

[0163] This application also provides a computer program product, which includes a computer program that, when executed by a processor, can implement the methods described in any of the foregoing embodiments performed by the lawnmower robot in any of the foregoing embodiments of this application.

[0164] It should be noted that the functions or steps that the computer-readable storage medium or lawnmower robot can achieve are described in the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0166] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0167] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A recharging and positioning method for a lawnmower robot, characterized in that, include: Acquire an image including the charging dock; The image is globally located using a global localization model to obtain the first set of key points for the charging dock; Based on the first key point group, multiple local image regions with different field of view are extracted from the image; The local image regions are locally located using a local localization model to obtain the corresponding second keypoint group; The first key point group is fused with multiple groups of the second key point group, and the key points of the fusion result are optimized to obtain the fused key point group. Based on the first key point group, a point-by-point consistency check is performed on the fused key point group, and key points that fail the check are corrected to obtain a corrected key point set.

2. The method according to claim 1, characterized in that, Before fusing the first keypoint group with multiple sets of the second keypoint group, the method further includes: Keypoint optimization is performed on the first keypoint group and / or at least one group of the second keypoint group to obtain the optimized first keypoint group and / or optimized second keypoint group for use in the fusion.

3. The method according to claim 1 or 2, characterized in that, The key point optimizations include: Shape constraints are applied using the known geometry of the charging dock, and / or, the positions of key points are fine-tuned using local features of the image.

4. The method according to claim 3, characterized in that, The shape constraint using the known geometry of the charging dock includes: Using a preset standard key point template, perform similar transformation and alignment with the key point group to be optimized to obtain a transformed key point group that conforms to the standard geometric shape; The transformation key point group and the key point group to be optimized are weighted and fused.

5. The method according to claim 3, characterized in that, The step of fine-tuning the key point positions using local features of the image includes: On the grayscale image of the image, local image blocks are extracted centered on the key points to be fine-tuned; Detect corner points within the local image block and determine the sub-pixel level coordinates of candidate corner points; When the distance between the subpixel coordinates and the key point to be fine-tuned meets the preset conditions, the key point is updated with the subpixel coordinates.

6. The method according to claim 1, characterized in that, The step of extracting multiple local image regions with different field of view from the image based on the first key point group includes: Based on the bounding box output by the global localization model and / or the bounding box determined by the first keypoint group, the bounding boxes are expanded outward according to different size expansion parameters to generate at least two local image regions.

7. The method according to claim 1, characterized in that, The step of fusing the first keypoint group with multiple sets of the second keypoint group includes: Preset weights are assigned to the first key point group and each group of the second key point group, and a point-by-point weighted average is performed.

8. The method according to claim 1, characterized in that, Based on the first key point group, a point-by-point consistency check is performed on the fused key point group, and key points that fail the check are corrected to obtain a corrected key point set, including: For each key point, calculate its dispersion between the corresponding point positions in the first key point group and each of the second key point groups; Key points whose dispersion exceeds a preset dispersion threshold are identified as untrustworthy points; For the unreliable points, the corresponding points in the first key point group or the statistical center values ​​obtained from each key point group are used for replacement and correction to obtain the corrected key point set.

9. The method according to claim 8, characterized in that, The step of replacing and correcting the untrusted points using the corresponding points in the first keypoint group or the statistical center values ​​obtained from each keypoint group includes: Calculate the deviation between the corresponding point in the first key point group and the statistical center value; If the deviation is greater than the preset deviation threshold, the unreliable point is corrected using the statistical center value; Otherwise, the untrusted point is corrected using the corresponding point in the first key point group.

10. The method according to any one of claims 1 to 9, characterized in that, The charging dock includes an inner frame and an outer frame, and the fusion key point group includes inner frame corner points and outer frame corner points; the method further includes: When the preset rescue triggering conditions are met, the image is corrected by perspective using the corner points of the outer frame to obtain a front view; In the front view, the positions of the inner frame corner points are extracted based on image features; The extracted inner frame corner points are back-projected back to the original image, and the corresponding inner frame corner points in the fused key point group are replaced with the back-projected inner frame corner points.

11. The method according to claim 10, characterized in that, The rescue triggering conditions include: The number of key points corrected in the point-by-point consistency check reaches a preset number, or the shape formed by the corner points of the outer frame in the fused key point group deviates from the angle of the standard rectangle by more than a preset angle.

12. The method according to claim 1 or 10, characterized in that, The charging dock includes an inner frame and an outer frame, and the key point group includes corner points of the inner frame and corner points of the outer frame; the method further includes: Detect whether the geometric relationship between the inner frame corner point and the outer frame corner point in the key point group conforms to a preset abnormal pattern; When the abnormal pattern is met, an ideal key point is calculated based on the reference key point where no abnormality occurs and the standard geometry of the charging dock, and the key point where the abnormality occurs is replaced with the ideal key point.

13. A lawnmower robot, characterized in that, The lawnmower robot includes: at least two sensors, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the recharge positioning method of the lawnmower robot as described in any one of claims 1 to 12.

Images