Method for correcting binocular camera of mowing robot and mowing robot

Abstract

The invention provides a correction method for a binocular camera of a mowing robot and the mowing robot. The correction method comprises the following steps: adjusting the mowing robot to a target pose; after the mowing robot is adjusted to the target pose, the binocular camera is used for shooting the front environment, and two images including the charging station are obtained; correcting the two images according to the camera parameters of the binocular camera, obtaining a charging station disparity map by using the corrected two images, comparing the deviation between the charging station disparity map and a standard disparity map, and adjusting the camera parameters to reduce the deviation when the deviation is greater than a preset value; and repeating the step until the deviation is not greater than a preset value. By correcting the camera parameters of the binocular camera, it can be ensured that the mowing robot accurately recognizes the surrounding environment, and the mowing robot can conveniently carry out the next step of operation such as obstacle avoiding or high grass area finding. In addition, the accurate charging station disparity map can provide more reliable depth information to help the mowing robot to navigate.

Description

Technical Field

[0001] This application relates to the field of lawn mowing robot technology, and in particular to a method for calibrating a binocular camera for a lawn mowing robot and a lawn mowing robot. Background Technology

[0002] Lawn-mowing robots often work in outdoor environments and are easily affected by various environmental factors, such as temperature changes, humidity fluctuations, and mechanical vibrations. These environmental factors may cause changes in the camera parameters of the binocular camera, resulting in binocular parallax degradation, which in turn affects the lawn-mowing robot's perception of its surroundings. For example, it may be difficult to accurately estimate the position of the target object, which may cause the lawn-mowing robot to have difficulty avoiding obstacles or finding tall grass areas, thus affecting its navigation accuracy and motion stability. In severe cases, it may even cause the lawn-mowing robot to malfunction.

[0003] Based on this, this application provides a method for calibrating the binocular camera of a lawnmower robot and a lawnmower robot, in order to improve the related technology. Summary of the Invention

[0004] The purpose of this application is to provide a method for calibrating a binocular camera for a lawnmower robot and a lawnmower robot that can improve the accuracy of the lawnmower robot in recognizing its surrounding environment.

[0005] The objective of this application is achieved through the following technical solution:

[0006] Currently, this application provides a method for calibrating a binocular camera of a lawnmower robot, the lawnmower robot being able to depart from a charging station to perform lawnmowing operations and return to the charging station for charging when the battery is low, the method comprising:

[0007] Adjust the lawnmower robot to the target pose;

[0008] Once the lawnmower robot has adjusted to the target pose, it uses the binocular camera to capture images of the environment in front of it, obtaining two images that include the charging station.

[0009] The two images are corrected according to the camera parameters of the binocular camera, and a disparity map of the charging station is obtained using the corrected two images. The deviation between the disparity map of the charging station and the standard disparity map is compared. If the deviation is greater than a preset value, the camera parameters are adjusted to reduce the deviation. This step is repeated until the deviation is not greater than the preset value.

[0010] In some embodiments, adjusting the lawnmower robot to the target pose includes:

[0011] Control the lawnmower robot to move to a target location, which is located within a target area near the charging station;

[0012] When the robot moves to the target location, it adjusts its posture until the charging station in the image captured by the left or right eye camera of the binocular camera meets the preset conditions.

[0013] In some embodiments, the preset condition is that the coordinates of a preset reference point on the charging station in the image captured by the left or right eye camera of the binocular camera are consistent with the target coordinates.

[0014] In some embodiments, the outline of the charging station or the markings on the charging station determine the centerline of the charging station. The centerline intersects with the bottom edge of the image captured by the left or right eye camera of the binocular camera. The centerline and the bottom edge also form an angle. The preset condition is that the angle is within a preset angle range, and the distance between the intersection point and a preset point on the centerline meets a preset distance requirement.

[0015] In some embodiments, the step of correcting the two images according to the camera parameters of the binocular camera and obtaining a charging station disparity map using the corrected two images includes:

[0016] The two images are corrected according to the camera parameters of the binocular camera;

[0017] The two corrected images are calculated using a stereo matching algorithm to obtain an overall disparity map;

[0018] Based on one of the two images, a charging station outline map is obtained, and the charging station outline map and the overall disparity map are ANDed to obtain the charging station disparity map.

[0019] In some embodiments, the standard disparity map is obtained in advance by:

[0020] The binocular camera is calibrated to obtain its intrinsic and extrinsic parameters, and these parameters are used as the camera parameters.

[0021] After the binocular camera is calibrated, and when the lawnmower robot is in the target pose, the binocular camera is used to acquire two images containing the charging station.

[0022] The two images are corrected according to the camera parameters obtained from the calibration, and a standard charging station disparity map is obtained using the corrected two images. The standard charging station disparity map is then used as the standard disparity map.

[0023] In some embodiments, adjusting the camera parameters to reduce the deviation when the deviation is greater than a preset value includes:

[0024] When the deviation exceeds a preset value, the following processing is performed:

[0025] S1: Adjust the camera parameters of the binocular camera according to the target step size;

[0026] S2: Correct the two images according to the adjusted camera parameters, and use the corrected two images to re-obtain the disparity map of the charging station;

[0027] S3: Compare the deviation between the newly obtained charging station disparity map and the standard disparity map. If the deviation obtained by comparison is greater than the preset value, execute S1; if the deviation obtained by comparison is not greater than the preset value, end the correction.

[0028] In some embodiments, controlling the lawnmower robot to move to the target location includes:

[0029] The charging station is photographed using either the left or right camera of the binocular camera to obtain a first image of the charging station; the outline of the charging station corresponding to the first image is acquired, and based on the outline, the lawnmower robot is controlled to move to the target location; or...

[0030] The charging station is photographed using either the left or right camera of the binocular camera to obtain a second charging station image, which includes color bands; based on the color bands in the second charging station image, the lawnmower robot is controlled to move to the target location.

[0031] In some embodiments, before the step of adjusting the lawn mowing robot to the target pose, the method further includes: determining whether the target correction conditions are met, wherein the target correction conditions include one or more of the following: the lawn mowing robot's working time is greater than the target time, the number of lawn mowing operations is greater than the target number, or an anomaly occurs during lawn mowing operations;

[0032] The step of adjusting the lawn mowing robot to the target pose includes: adjusting the lawn mowing robot to the target pose when the target correction conditions are met.

[0033] Secondly, this application provides a lawnmower robot, including a control module for performing any of the methods described above.

[0034] This application provides a method for calibrating a binocular camera for a lawnmower robot and the lawnmower robot itself. After adjusting the lawnmower robot to the target pose, the binocular camera captures images of a charging station, obtaining two images containing the charging station. The two images are then calibrated using the current camera parameters. A disparity map of the charging station is calculated based on the calibrated images. This disparity map is then compared with a known standard disparity map. If the deviation between the charging station disparity map and the standard disparity map exceeds a preset value, the camera parameters need to be adjusted, and the above steps are repeated until the deviation is no greater than the preset value. By calibrating the camera parameters of the binocular camera, this application ensures that the lawnmower robot accurately identifies its surrounding environment, facilitating subsequent operations such as avoiding obstacles or finding tall grass areas. Furthermore, calibrating the camera parameters improves the accuracy of the binocular camera in identifying its surroundings, thereby enhancing the lawnmower robot's navigation accuracy and obstacle avoidance capabilities. Secondly, using the charging station as a calibration reference simplifies the calibration process and reduces reliance on external equipment or complex calibration procedures, making the calibration operation simpler and faster. Furthermore, by ensuring that the lawnmower maintains good performance under various conditions, user satisfaction and product market competitiveness are improved, and maintenance needs due to changes in camera parameters are reduced, thus lowering the overall maintenance cost of the lawnmower. Attached Figure Description

[0035] This application will be further described below with reference to the accompanying drawings and specific embodiments.

[0036] Figure 1 This is a flowchart illustrating a method for calibrating a binocular camera on a lawnmower robot, as provided in an embodiment of this application.

[0037] Figure 2 This is a schematic diagram illustrating the determination of a preset reference point provided in an embodiment of this application.

[0038] Figure 3 This is a schematic diagram of determining the center line and intersection point provided in an embodiment of this application.

[0039] Figure 4 This is a schematic diagram of another method for determining the center line and intersection points provided in an embodiment of this application.

[0040] Figure 5 This is a left-eye image provided in an embodiment of this application.

[0041] Figure 6 This is a right eye image provided in an embodiment of this application.

[0042] Figure 7 This is a first overall disparity map provided in an embodiment of this application.

[0043] Figure 8This is a outline diagram of a charging station provided in an embodiment of this application.

[0044] Figure 9 This is a parallax map of a first charging station provided in an embodiment of this application.

[0045] Figure 10 This is a second overall disparity map provided in the embodiments of this application.

[0046] Figure 11 This is a parallax map of a second charging station provided in an embodiment of this application.

[0047] Figure 12 This is a third overall disparity map provided in the embodiments of this application.

[0048] Figure 13 This is a parallax map of a third charging station provided in an embodiment of this application.

[0049] In the picture: 100, charging station; 200, sign. Detailed Implementation

[0050] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] In the description of the embodiments of this application, it should be understood that the terms "current" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined as "current" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0052] Binocular cameras are widely used in lawnmower robots for functions such as obstacle avoidance, locating tall grass areas, and positioning. To ensure proper operation, high-precision calibration methods are typically required to accurately calibrate camera parameters. Lawnmower robots often operate outdoors and are susceptible to various environmental factors, such as temperature changes, humidity fluctuations, and mechanical vibrations. These factors can cause changes in the camera parameters, leading to degraded parallax and affecting the robot's perception of its surroundings. For example, it may become difficult to accurately estimate the position of objects, making it harder to avoid obstacles or locate tall grass areas, thus impacting navigation accuracy and motion stability. In severe cases, this can even cause the robot to malfunction.

[0053] See Figure 1 , Figure 1 This is a flowchart illustrating a method for calibrating a binocular camera on a lawnmower robot, as provided in an embodiment of this application.

[0054] In related technologies, known geometric parameters of objects (such as the planar characteristics of the ground, parallel lines, and the width of parallel lines) are used for automatic correction and even automatic calibration of lawn mowing robots. However, since lawn mowing robots work in unstructured lawn environments, although lawns are mostly flat, they have uneven surfaces and variations in the height of the grass, which cannot guarantee 100% that the area in front of the lawn mowing robot is flat. To improve related technologies, this application provides a method for calibrating a binocular camera for a lawn mowing robot. The lawn mowing robot can start from a charging station to perform mowing operations and return to the charging station to recharge when its battery is low. The method includes steps S101 to S103.

[0055] Step S101: Adjust the lawnmower robot to the target pose.

[0056] Step S102: After the lawnmower robot adjusts to the target pose, it uses the binocular camera to capture the environment in front of it and obtain two images containing the charging station.

[0057] Step S103: Correct the two images according to the camera parameters of the binocular camera, and obtain a charging station disparity map using the corrected two images. Compare the deviation between the charging station disparity map and the standard disparity map. If the deviation is greater than a preset value, adjust the camera parameters to reduce the deviation. Repeat this step until the deviation is not greater than the preset value.

[0058] In this system, a binocular camera is a system consisting of two cameras, similar to human eyes. Target pose refers to the pose of the two images taken by the binocular cameras of the foreground environment while acquiring a standard disparity map. Correction, for example, refers to distortion correction (i.e., eliminating lens distortion) and geometric alignment (i.e., aligning overlapping fields of view in the two images) performed on the two images based on camera parameters. The standard disparity map can be obtained under laboratory conditions or before the lawnmower leaves the factory for subsequent correction. Camera parameters, for example, refer to the intrinsic and extrinsic parameters of the binocular camera, where the intrinsic parameters include the focal length f along the x-axis. x Focal length f along the y-axis y The pixel coordinates of the camera's optical center in the image (u x u yExternal parameters include distortion parameters, and the rotation matrix R and translation matrix T of the camera coordinate system relative to the world coordinate system. Deviation refers to the difference between the actual obtained disparity map of the charging station and the standard disparity map; the larger the deviation, the less accurate the camera parameter settings are considered. The preset value is a pre-set threshold used to determine if the deviation is small enough. If the deviation is not greater than the preset value, the camera is considered to have been calibrated. The preset value can be, for example, 10; this application does not impose any limitation on this.

[0059] In the above embodiment, after adjusting the lawnmower robot to the target pose, a binocular camera is used to capture images of the charging station, obtaining two images containing the charging station. The two images are then corrected using the current camera parameters. A disparity map of the charging station is calculated based on the corrected images. This calculated disparity map is then compared to a known standard disparity map. If the deviation between the charging station disparity map and the standard disparity map exceeds a preset value, the camera parameters need to be adjusted, and the above steps are repeated. That is, after adjusting the camera parameters, the two images are corrected again using the adjusted camera parameters, and the disparity map of the charging station is obtained using the two images obtained after the second correction. The deviation between the charging station disparity map and the standard disparity map is compared again. If the newly obtained deviation is still greater than the preset value, the above steps of adjusting the camera parameters and performing the steps after adjusting the camera parameters are repeated until the deviation is less than or equal to the preset value. In the above embodiment, using the charging station as a reference to correct the binocular camera of the lawnmower robot fully utilizes the planar characteristic of the charging station's base. In one possible implementation, the binocular camera calibration method can be performed in OpenCV (Open Source Computer Vision Library).

[0060] In the above embodiments, by calibrating the camera parameters of the binocular camera, the lawnmower robot can accurately identify its surroundings, facilitating its next operations, such as avoiding obstacles or finding tall grass areas. Secondly, an accurate charging station parallax map provides more reliable depth information, aiding the lawnmower robot in navigating complex environments. Furthermore, by optimizing the camera parameters of the binocular camera, the lawnmower robot can automatically adjust, reducing the need for manual intervention and improving calibration efficiency.

[0061] To further define the specific process of adjusting the lawnmower robot to the target pose, in some embodiments, adjusting the lawnmower robot to the target pose may include: controlling the lawnmower robot to move to a target position, the target position being located within a target range near the charging station; when moving to the target position, adjusting the posture of the lawnmower robot until the charging station in the image captured by the left or right eye camera of the binocular camera meets preset conditions.

[0062] The target location refers to the specific location that the lawnmower needs to move to. The target location can be any location within the target range. The ground within the target range is on the same plane as the base of the charging station. This can be understood as the ground within the target range being roughly flat with no significant height changes. For example, the target range can be a 1m x 1m area in front of the charging station.

[0063] In the above embodiments, the lawnmower robot is controlled to move to the target location. After moving to the target location, the robot's posture (posture in this document can be understood as attitude or orientation) is adjusted until the charging station in the image captured by the left or right eye camera meets the preset conditions. When moving to the target location, the lawnmower robot's posture may not be facing the charging station. In this case, the robot's posture needs to be adjusted so that the binocular camera can capture the charging station. Either of the binocular cameras can be used to obtain an image containing the charging station. By analyzing whether the charging station in the image meets the preset conditions, it is determined whether further posture adjustment is needed. If the preset conditions are not met, the lawnmower robot's posture can be adjusted according to the image. After each posture adjustment, an image containing the charging station is re-acquired until the charging station in the image meets the preset conditions.

[0064] In the above embodiment, after the lawnmower moves to the target location, the robot's posture is continuously adjusted to ensure that the binocular camera on the lawnmower can capture an image containing a charging station that meets the preset conditions.

[0065] See Figure 2 , Figure 2 This is a schematic diagram illustrating the determination of a preset reference point provided in an embodiment of this application.

[0066] To ensure the lawnmower robot can capture two suitable images including the charging station, so that camera parameters can be subsequently corrected based on the two images, in some embodiments, the preset condition is that the coordinates of a preset reference point on the charging station in the image captured by the left or right camera of the binocular camera are consistent with the target coordinates.

[0067] Here, the preset reference point refers to one or more specific locations on the charging station. These locations have obvious features in the image and can be easily identified by a stereo camera, for example, such as... Figure 2As shown, the preset reference points can be the two endpoints on the base of the charging station near the charging end. Target coordinates refer to the preset coordinates of the reference points in the image. For example, these can be pre-set according to the design of the charging station to guide the lawnmower robot in adjusting its posture. The fact that the preset reference point coordinates are consistent with the target coordinates can mean that the coordinates of the preset reference point and the target coordinates are the same, or it can mean that there is a coordinate deviation between the preset reference point coordinates and the target coordinates, but the coordinate deviation is controlled within a certain range.

[0068] In the above embodiments, images containing the charging station are captured by either the left or right camera in the binocular camera system. The coordinates of a preset reference point in the image are analyzed to ensure that the coordinates of the preset reference point are consistent with the target coordinates. If the coordinates of the preset reference point in the image are inconsistent with the target coordinates, the posture of the lawnmower robot is adjusted, the image is retaken, and the coordinates of the preset reference point are re-determined based on the retaken image until the coordinates of the preset reference point are consistent with the target coordinates.

[0069] The above embodiments, by identifying preset reference points on the charging station and comparing the coordinates of the preset reference points with the target coordinates, can ensure that the lawnmower robot adjusts to the target pose. In the event that the coordinates of the preset reference points are inconsistent with the target coordinates, the lawnmower robot's posture is adjusted until the coordinates of the preset reference points on the charging station in the image captured by the left or right eye camera are consistent with the target coordinates. This allows the lawnmower robot to automatically adjust based on visual feedback, reducing the need for manual intervention and improving the automation level of the system.

[0070] See Figure 3 and Figure 4 , Figure 3 This is a schematic diagram illustrating the determination of a centerline and its intersection point, provided in an embodiment of this application. Figure 4 This is a schematic diagram of another method for determining the center line and intersection points provided in an embodiment of this application.

[0071] In some embodiments, the outline of the charging station or the markings on the charging station determine the centerline of the charging station. The centerline intersects with the bottom edge of the image captured by the left or right eye camera of the binocular camera. The centerline and the bottom edge also form an angle. The preset condition is that the angle is within a preset angle range, and the distance between the intersection point and a preset point on the centerline meets a preset distance requirement.

[0072] The markings on the charging station refer to special marks or patterns on the charging station, such as colored stripes. The preset point refers to a point pre-defined on the center line; the location of this point can be determined, for example, based on the design of the charging station.

[0073] In the above embodiments, the outline of the charging station is identified by images captured by the left or right eye camera (e.g., ...). Figure 3 (as shown) or logo (such as) Figure 4 As shown, the centerline of the charging station is determined based on its outline or markings. Then, the angle between the centerline and the bottom edge of the image, as well as the intersection point of the centerline and the bottom edge of the image, are determined. If the angle is within a preset range and the distance between the intersection point and a preset point on the centerline meets a preset distance requirement, the charging station in the captured image is deemed to meet the preset conditions. If the angle exceeds the preset range or the distance between the intersection point and the preset point on the centerline does not meet the preset distance requirement, the robot's posture is adjusted until both the angle and the distance between the intersection point and the preset point meet the corresponding requirements. For example, the preset angle range could be between 90±2 degrees, and the preset distance could be between 50±10 mm; this application does not impose any limitations on these parameters.

[0074] The above embodiments ensure that the angle between the centerline of the charging station and the bottom edge of the image is within a preset angle range, and that the distance between the intersection point and the preset point meets the preset distance requirement. This enables the lawnmower robot to automatically adjust based on visual feedback, reducing the need for manual intervention and improving the automation level of the system.

[0075] To clarify the specific process of obtaining the charging station disparity map, in some embodiments, the step of correcting the two images according to the camera parameters of the binocular camera and using the corrected two images to obtain the charging station disparity map includes: correcting the two images according to the camera parameters of the binocular camera; calculating the overall disparity map using a stereo matching algorithm on the corrected two images; obtaining a charging station outline map based on one of the two images, and performing an AND operation between the charging station outline map and the overall disparity map to obtain the charging station disparity map.

[0076] Stereo matching algorithms are used to determine the horizontal pixel difference (i.e., disparity) between corresponding feature points in two images to calculate depth information. Examples include block matching algorithms, feature point matching algorithms, and semi-global block matching (SGBM). The overall disparity map refers to the disparity map calculated using stereo matching algorithms based on the two corrected images, containing the disparity value of each pixel in the captured scene. In this paper, the calculation refers to calculating the disparity of the charging station outline map and the overall disparity map, retaining only the disparity information within the charging station outline.

[0077] In the above embodiment, firstly, the camera parameters of the binocular camera can be used to correct the distortion of the two images, eliminating the influence of lens distortion, ensuring that straight lines in the images remain straight, and improving the accuracy of subsequent disparity calculations. Then, the two images after distortion correction are precisely aligned to ensure geometric consistency. Next, a stereo matching algorithm is used to calculate the horizontal pixel difference between corresponding feature points in the aligned images, i.e., to calculate the disparity, and a corresponding disparity value is assigned to each pixel in the scene. These disparity values ​​are then integrated to generate an overall disparity map. When generating the overall disparity map, a charging station outline map can also be obtained. The charging station outline map is then ANDed with the overall disparity map, retaining only the disparity information within the charging station outline to obtain the charging station disparity map.

[0078] The above embodiments improve the accuracy of the binocular camera in recognizing the surrounding environment by calibrating the camera parameters, thereby enhancing the navigation accuracy and obstacle avoidance capabilities of the lawnmower robot. Secondly, using the charging station as a calibration reference simplifies the calibration process and reduces reliance on external equipment or complex calibration procedures, making calibration operations more convenient and faster. Furthermore, by ensuring the lawnmower robot maintains good performance under various conditions, user satisfaction and product market competitiveness are improved, and maintenance needs due to changes in camera parameters are reduced, lowering the overall maintenance cost of the lawnmower robot.

[0079] To clarify the process of obtaining the standard disparity map, in some embodiments, the standard disparity map is obtained in advance by: calibrating the binocular camera to obtain the intrinsic and extrinsic parameters of the binocular camera, and using the intrinsic and extrinsic parameters as the camera parameters; after the binocular camera is calibrated, and when the lawnmower robot is in the target pose, using the binocular camera to acquire two images containing the charging station; correcting the two images according to the calibrated camera parameters, and using the corrected two images to obtain a standard charging station disparity map, and using the standard charging station disparity map as the standard disparity map.

[0080] In the above embodiment, the binocular camera of the lawnmower robot is first calibrated, for example using a high-precision classical calibration method (such as the Zhang Zhengyou calibration method), to obtain the intrinsic and extrinsic parameters of the binocular camera, which are then used as standard camera parameters. After the binocular camera is calibrated, when the lawnmower robot is in the target pose, two images containing the charging station are acquired using the binocular camera. These two images are then corrected according to the standard camera parameters to eliminate lens distortion and achieve geometric alignment between the two images. A stereo matching algorithm can then be used to calculate the standard global disparity map from the corrected two images. The charging station contour map is extracted from one of the images, and a bitwise AND operation is performed between the charging station contour map and the standard global disparity map, retaining only the disparity information within the charging station contour, thus obtaining a standard charging station disparity map. This standard charging station disparity map is stored as a standard disparity map, and the disparities in the standard disparity map are used as standard disparities.

[0081] The above embodiments provide a reference for the subsequent calibration process by acquiring a standard disparity map. By comparing the charging station disparity map with the standard disparity map, it is possible to quickly determine whether calibration is needed, simplifying the calibration process. Furthermore, acquiring the standard disparity map and performing the subsequent calibration process reduces the need for manual intervention and improves the system's automation level.

[0082] To clarify how to reduce the deviation between the charging station disparity map and the standard disparity map, in some embodiments, adjusting the camera parameters to reduce the deviation when the deviation is greater than a preset value includes: when the deviation is greater than the preset value, performing the following processing: S1: adjusting the camera parameters of the binocular camera by a target step size; S2: correcting the two images according to the adjusted camera parameters, and re-obtaining the charging station disparity map using the corrected two images; S3: comparing the deviation between the re-obtained charging station disparity map and the standard disparity map, and performing S1 when the deviation obtained by comparison is greater than the preset value; ending the correction when the deviation obtained by comparison is not greater than the preset value.

[0083] In the above embodiments, if the deviation is greater than a preset value, a camera parameter adjustment loop is entered. The camera parameters of the binocular camera are adjusted with a target step size. The target step size can be a preset fixed value or dynamically adjusted according to the magnitude of the deviation. After adjusting the camera parameters, the two previously acquired images are corrected using the adjusted camera parameters, and the charging station disparity map is re-obtained using the corrected two images. The deviation between the re-obtained charging station disparity map and the standard disparity map is compared again. If the deviation is still greater than a preset value, the camera parameters are adjusted again, and the process after adjusting the camera parameters is repeated. If the deviation is not greater than a preset value, the correction ends. In some embodiments, the range of camera parameter variation can be set, and a corresponding target step size can be set. For example, taking the focal length and the pixel coordinates of the camera's optical center in the image as examples, the range of focal length variation along the x-axis is (f xmin f xmax The target step size along the focal length of the x-axis is Δf. x The range of focal length variation along the y-axis is (f ymin f ymax The target step distance along the focal length of the y-axis is Δf. y The range of pixel coordinates of the camera's optical center along the x-axis in the image is (u xmin u xmax The corresponding target step size is Δu. x The range of pixel coordinates of the camera's optical center along the y-axis in the image is (u ymin u ymax The corresponding target step size is Δu. y Within the range of variation of the corresponding camera parameters, the camera parameters can be adjusted with the corresponding target step size. Then, the disparity map of the charging station can be obtained again based on the adjusted camera parameters. The deviation between the obtained disparity map of the charging station and the standard disparity map can be calculated and compared with the preset value.

[0084] In some embodiments, the deviation can be calculated, for example, by the following formula.

[0085] For example, the first type is Where M and N are the width and height of the image, respectively, and P... cij P represents the disparity corresponding to each pixel in the disparity map of the charging station. oij This represents the disparity corresponding to each pixel in the standard disparity map.

[0086] The second type is, for example, the absolute value of the statistical deviation |P c —P o |P greater than a given value g Quantity S g .

[0087] The specific formula is as follows: in Where P cij P represents the disparity corresponding to each pixel in the disparity map of the charging station. oij This represents the disparity corresponding to each pixel in the standard disparity map.

[0088] The third type is, for example, the absolute value of the statistical deviation |P c —P o | Greater than the given value P g The percentage of pixels in the total number of pixels.

[0089] The specific formula is R g :

[0090] The above embodiments, by automatically adjusting camera parameters, can reduce the deviation between the charging station's disparity map and the standard disparity map. Furthermore, the automatic adjustment of camera parameters during cyclic calibration simplifies the calibration process and reduces the need for manual intervention. In addition, by automatically correcting deviations, performance degradation caused by changes in camera parameters is reduced, thus lowering maintenance costs.

[0091] To enable the lawnmower robot to accurately move to a target location, in some embodiments, controlling the lawnmower robot to move to the target location includes: using the left or right camera of the binocular camera to photograph the charging station and obtain a first charging station image; acquiring the charging station outline corresponding to the first charging station image, and controlling the lawnmower robot to move to the target location based on the charging station outline; or, using the left or right camera of the binocular camera to photograph the charging station and obtain a second charging station image, the second charging station image including color bands; and controlling the lawnmower robot to move to the target location based on the color bands in the second charging station image.

[0092] In the above embodiments, there are two methods for controlling the lawnmower robot to move to the target location. One method is to use either of the binocular cameras to photograph the charging station, obtaining a first image of the charging station. This can be understood as assuming there are no markers on the charging station. By recognizing the outline of the charging station in the first image, the lawnmower robot moves to the target location based on the outline. The other method is to use either of the binocular cameras to photograph the charging station, obtaining a second image of the charging station. Based on the color bands in the second image, the lawnmower robot moves to the target location based on the color bands.

[0093] The above embodiments, by recognizing the outline or color stripe of the charging station, can control the lawnmower robot to move to the target location based on the charging station's outline or color stripe, even under different lighting conditions or in the presence of obstructions. Furthermore, by recognizing the charging station's outline or color stripe, the lawnmower robot can move to the target location without relying on complex path planning algorithms, simplifying the navigation process.

[0094] To determine when to adjust the lawnmower robot to the target pose, in some embodiments, before the step of adjusting the lawnmower robot to the target pose, the method further includes: determining whether a target correction condition is met, wherein the target correction condition includes one or more of the following: the lawnmower robot's working time is greater than a target time, the number of lawnmower operations is greater than a target number, or an anomaly occurs during lawnmower operations; adjusting the lawnmower robot to the target pose includes: adjusting the lawnmower robot to the target pose if the target correction condition is met.

[0095] In the above embodiments, it is first determined whether the target correction conditions are met. If the target correction conditions are met, such as the lawnmower robot's working time exceeding the target time, the number of lawnmower operations exceeding the target number, or an anomaly occurring during lawnmower operations, the lawnmower robot is controlled to adjust to the target pose. In some embodiments, the target time is, for example, one month; the target number is, for example, 60 lawnmower operations by the lawnmower robot; and an anomaly during lawnmower operations is, for example, an area is determined to be a lawn based on images taken by either monocular camera in the binocular camera system using AI or traditional algorithms, but the binocular camera images indicate that the area contains many protruding obstacles. The above embodiments, through a correction mechanism triggered periodically or based on specific conditions, can reduce performance degradation caused by changes in camera parameters and lower maintenance costs.

[0096] See Figures 5 to 11 , Figure 5 This is a left-eye image provided in an embodiment of this application. Figure 6 This is a right eye image provided in an embodiment of this application. Figure 7 This is a first overall disparity map provided in an embodiment of this application. Figure 8 This is a outline diagram of a charging station provided in an embodiment of this application. Figure 9 This is a parallax map of a first charging station provided in an embodiment of this application. Figure 10 This is a second overall disparity map provided in the embodiments of this application. Figure 11 This is a parallax map of a second charging station provided in an embodiment of this application.

[0097] The following example illustrates the calibration method for the binocular camera of a lawnmower robot.

[0098] Because the intrinsic parameters in camera parameters have a greater impact on the deviation, this example only adjusts the intrinsic parameters to save computation.

[0099] Under the condition that the target correction is met, the lawnmower robot is first controlled to move to the target position, which is located within the target range near the charging station. The target range is a 1m×1m area in front of the charging station.

[0100] When the lawnmower moves to the target location, it adjusts its posture and uses its binocular cameras to capture two images of the surrounding environment, including the charging station. This image is recorded as the left-eye image (e.g., ...). Figure 5 (as shown) and right eye image (as shown) Figure 6 As shown), the left and right eye images are then corrected based on the camera parameters of the stereo camera. A semi-global block matching algorithm is then used to calculate the first global disparity map (as shown). Figure 7 (As shown). Simultaneously, a charging station outline map can also be obtained from the left-eye image (e.g., Figure 8 As shown), and perform AND calculations on the first overall disparity map and the charging station outline map to obtain the first charging station disparity map (as shown). Figure 9 (As shown). The first deviation between the disparity map of the first charging station and the standard disparity map is calculated, and it is found that the first deviation is greater than a preset value.

[0101] After determining that the first deviation is greater than a preset value, the intrinsic parameters in the camera parameters are adjusted. Based on the adjusted intrinsic parameters, the left and right eye images are then subjected to secondary correction. A semi-global block matching algorithm is then used to calculate the second global disparity map (e.g., ...) on the corrected left and right eye images. Figure 10 (As shown). The second overall disparity map and the charging station outline map are compared and calculated to obtain the second charging station disparity map (as shown). Figure 11 (As shown). Calculate the second deviation between the disparity map of the second charging station and the standard disparity map. If the second deviation is not greater than a preset value, the correction ends.

[0102] See Figure 12 and Figure 13 , Figure 12 This is a third overall disparity map provided in an embodiment of this application. Figure 13 This is a parallax map of a third charging station provided in an embodiment of this application.

[0103] The disparity maps of the first, second, and third charging stations were compared with the standard disparity map to obtain the first, second, and third deviations. The third deviation was greater than the first deviation, and the first deviation was greater than the second deviation. Figure 9 , Figure 11 and Figure 13 From the parallax diagram, we can see that the plane of the charging station is completely invisible in the third charging station parallax diagram, while a portion of the charging station plane can be seen in the first charging station parallax diagram, and the charging station plane can be clearly seen in the second charging station parallax diagram. It is believed that the smaller the deviation, the clearer and easier to identify the planar features in the charging station parallax diagram; the larger the deviation, the more blurred the planar features in the charging station parallax diagram, and they may even disappear completely.

[0104] This application also provides a lawnmower robot, including a control module, which is used to execute any of the above methods.

[0105] It should be noted that although some embodiments of this application use a lawnmower robot as an example, this application can be applied to other self-moving devices, and this application does not set any limitations on them.

[0106] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, etc.) involved in various embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws and regulations and standards of the relevant countries and regions, and corresponding instruction entry points shall be provided for the user to choose to authorize or refuse.

[0107] It is understood that the specific examples in this specification are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0108] It is understood that in the various embodiments of this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0109] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and this application does not limit them.

[0110] Unless otherwise stated, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0112] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0113] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0115] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0116] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The above are merely specific embodiments described in this specification, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for calibrating a binocular camera of a lawnmower robot, the lawnmower robot being able to depart from a charging station to perform lawnmowing operations and return to the charging station for charging when its battery is low, characterized in that, The method includes: Adjust the lawnmower robot to the target pose; Once the lawnmower robot has adjusted to the target pose, it uses the binocular camera to capture images of the environment in front of it, obtaining two images that include the charging station. The two images are corrected according to the camera parameters of the binocular camera, and a disparity map of the charging station is obtained using the corrected two images. The deviation between the disparity map of the charging station and the standard disparity map is compared. If the deviation is greater than a preset value, the camera parameters are adjusted to reduce the deviation. This step is repeated until the deviation is not greater than the preset value.

2. The method according to claim 1, characterized in that, Adjusting the lawnmower robot to the target pose includes: Control the lawnmower robot to move to a target location, which is located within a target area near the charging station; When the robot moves to the target location, it adjusts its posture until the charging station in the image captured by the left or right eye camera of the binocular camera meets the preset conditions.

3. The method according to claim 2, characterized in that, The preset condition is that the coordinates of a preset reference point on the charging station in the image captured by the left or right eye camera of the binocular camera are consistent with the target coordinates.

4. The method according to claim 2, characterized in that, The outline of the charging station or the markings on the charging station determine the centerline of the charging station. The centerline intersects with the bottom edge of the image captured by the left or right eye camera of the binocular camera. The centerline and the bottom edge also form an angle. The preset condition is that the angle is within a preset angle range, and the distance between the intersection point and a preset point on the centerline meets a preset distance requirement.

5. The method according to any one of claims 1-4, characterized in that, The step of correcting the two images based on the camera parameters of the binocular camera and obtaining a disparity map of the charging station using the corrected two images includes: The two images are corrected according to the camera parameters of the binocular camera; The two corrected images are calculated using a stereo matching algorithm to obtain an overall disparity map; Based on one of the two images, a charging station outline map is obtained, and the charging station outline map and the overall disparity map are ANDed to obtain the charging station disparity map.

6. The method according to any one of claims 1-4, characterized in that, The standard disparity map is obtained in advance through the following methods: The binocular camera is calibrated to obtain its intrinsic and extrinsic parameters, and these parameters are used as the camera parameters. After the binocular camera is calibrated, and when the lawnmower robot is in the target pose, the binocular camera is used to acquire two images containing the charging station. The two images are corrected according to the camera parameters obtained from the calibration, and a standard charging station disparity map is obtained using the corrected two images. The standard charging station disparity map is then used as the standard disparity map.

7. The method according to claim 1, characterized in that, When the deviation is greater than a preset value, adjusting the camera parameters to reduce the deviation includes: When the deviation exceeds a preset value, the following processing is performed: S1: Adjust the camera parameters of the binocular camera according to the target step size; S2: Correct the two images according to the adjusted camera parameters, and use the corrected two images to re-obtain the disparity map of the charging station; S3: Compare the deviation between the newly obtained charging station disparity map and the standard disparity map. If the deviation obtained by comparison is greater than the preset value, execute S1; if the deviation obtained by comparison is not greater than the preset value, end the correction.

8. The method according to claim 2, characterized in that, Controlling the lawnmower robot to move to the target location includes: The charging station is photographed using either the left or right camera of the binocular camera to obtain a first image of the charging station; the outline of the charging station corresponding to the first image is acquired, and based on the outline, the lawnmower robot is controlled to move to the target location; or... The charging station is photographed using either the left or right camera of the binocular camera to obtain a second charging station image, which includes color bands; based on the color bands in the second charging station image, the lawnmower robot is controlled to move to the target location.

9. The method according to claim 1, characterized in that, Before the step of adjusting the lawn mowing robot to the target pose, the method further includes: determining whether the target correction conditions are met, wherein the target correction conditions include one or more of the following: the lawn mowing robot's working time is greater than the target time, the number of lawn mowing operations is greater than the target number, or an anomaly occurs during lawn mowing operations; The step of adjusting the lawn mowing robot to the target pose includes: adjusting the lawn mowing robot to the target pose when the target correction conditions are met.

10. A lawnmower robot, the lawnmower robot comprising a control module, characterized in that, The control module is used to execute the method according to any one of claims 1-9.

Images