Position determination method and device, mowing robot and medium

Abstract

The invention relates to the technical field of robot positioning, in particular to a position determination method and device, a mowing robot and a medium, and the method comprises the following steps: after the mowing robot completes mowing, determining a current position with a scale of the mowing robot by using a visual odometer on the mowing robot; the method comprises the following steps of: acquiring a transverse correction value which is obtained by comparing a transverse distance value between the position of the visual odometer at a target moment and mowing trace information at the target moment with a preset standard distance value due to the fact that the visual odometer possibly has an error, in particular an error in a transverse position; the current position is corrected according to the transverse correction amount, and the more accurate corrected position of the mowing robot is obtained; not only is the cost low, but also the position determination precision is improved, and the positioning capability of the mowing robot in a complex environment is enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of robot positioning, and in particular to a position determination method, device, lawn mowing robot and medium. Background Art

[0002] With the rapid development of modern technology, intelligent equipment has been widely used in various fields. Among them, lawn mowing robots are important tools for garden maintenance, and their technology iteration and upgrading are particularly critical. Lawn mowing robots not only need to complete lawn mowing tasks efficiently, but also need to have accurate navigation and positioning capabilities to ensure safety and efficiency during the operation.

[0003] Among the existing lawn mowing robot technologies, most products use a combination of RTK (Real Time Kinematic) technology and cameras, or UWB (Ultra Wide Band) technology and cameras for target recognition and positioning navigation. RTK technology can provide centimeter-level positioning accuracy by receiving satellite signals and performing differential calculations, while the camera is used to identify environmental information such as obstacles and boundary lines. Similarly, UWB technology uses ultra-wideband signals for ranging and positioning, and combined with the visual information of the camera, it can also achieve high-precision navigation.

[0004] However, the installation process of RTK and UWB technologies is relatively complicated and requires professional debugging and calibration, which increases the user's usage cost and time cost. Secondly, these technologies rely on external signals for positioning and navigation. Once the signal is interfered or blocked, the robot's positioning accuracy will drop significantly, and may even cause navigation errors.

[0005] Therefore, how to reduce costs while maintaining high-precision positioning and navigation has become an urgent problem to be solved in current lawn mowing robot technology. Summary of the invention

[0006] The purpose of this application is to provide a position determination method, device, lawn mowing robot and medium, which can reduce costs and maintain high-precision positioning and navigation.

[0007] In a first aspect, a method for determining a position is provided, comprising:

[0008] After the lawn mowing robot completes mowing, obtaining a current position of the lawn mowing robot with a scale determined by a visual odometer on the lawn mowing robot;

[0009] Obtaining a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target time and mowing trace information after mowing is completed at the target time;

[0010] The current position is corrected according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

[0011] In a preferred example, the present application may be further configured as follows: the current time is the target time, the current position is the target position, and after obtaining the current position of the mowing robot with a scale determined by the visual odometer on the mowing robot, the method further includes:

[0012] After obtaining the current position of the mowing robot with scale determined by the visual odometer on the mowing robot, it also includes:

[0013] Acquire an image to be identified of the lawn at a target time captured by a camera device on the lawn mowing robot;

[0014] Identifying mowing traces on the image to be identified to obtain mowing trace information;

[0015] Determine a lateral distance value between a target position determined by a visual odometer at a target time and the mowing trace information;

[0016] According to the lateral distance value and the preset standard distance value, the lateral correction amount of the visual odometer is determined. In a preferred example, the present application can be further configured as follows: before obtaining the current position of the mowing robot with scale determined by the visual odometer on the mowing robot, it also includes:

[0017] Acquire a reverse image including a marker captured by a camera device on the lawn mower robot when the lawn mower robot reverses and starts from the charging pile;

[0018] Determine, according to the reverse image, the scaled coordinates of the robot in the world coordinate system as first position information;

[0019] Obtain the scale-free coordinates of the robot in the world coordinate system calculated by the visual odometer as the second position information;

[0020] A restoration scale factor is determined according to the initial position, the first position information, and the second position information; the restoration scale factor is used to restore the scale of the position of the output of the control visual odometer to obtain the actual position, and the initial position is the initial position of the robot or the camera in the world coordinate system.

[0021] In a preferred example, the present application may be further configured as follows: determining the scaled coordinates of the robot in the world coordinate system according to the backward image as the first position information, including:

[0022] Acquire a position recognition model; the position recognition model is obtained by training a deep learning model through a plurality of training images including a marker and position information labels of the robots corresponding to the respective training images;

[0023] According to the reverse image, the position recognition model of the marker is used to perform position recognition, and the scaled coordinates of the marker in the robot coordinate system are obtained as the first position information.

[0024] In a preferred example, the present application may be further configured as follows: before obtaining the location recognition model, the application may further include:

[0025] Acquire a plurality of training images including markers and their corresponding position information labels;

[0026] Input the training image into the deep learning model to obtain the training position information;

[0027] Determine a loss value based on the training position information and the position information label;

[0028] When the loss value is greater than the preset loss threshold, training is performed based on the training images and their corresponding location information labels, and the network parameters of the deep learning model are updated until the loss value is no greater than the preset loss threshold, thereby obtaining a location recognition model.

[0029] In a preferred example, the present application can be further configured as follows:

[0030] When the mowing robot walks according to the planned path and the corrected position, a real-time image of the lawn captured by a camera device on the mowing robot is obtained;

[0031] Performing image recognition based on the real-time image to determine the pixel coordinates of the lowest point of the outline of the obstacle in the drivable area;

[0032] According to the posture parameters of the mowing robot and the pixel coordinates of the lowest point of the contour, the actual distance between the obstacle and the mowing robot is determined;

[0033] When the actual distance is less than the set threshold, the mowing robot is controlled to perform obstacle avoidance operations.

[0034] In a preferred example, the present application can be further configured as follows: when the actual distance is less than a set threshold, the lawn mowing robot is controlled to perform an obstacle avoidance operation, including:

[0035] Determine the obstacle area in the left region and the obstacle area in the right region in the real-time image;

[0036] When the area of ​​the obstacles in the left region is larger than the area of ​​the obstacles in the right region, the lawn mowing robot is controlled to detour to the right;

[0037] When the obstacle area in the left region is smaller than the obstacle area in the right region, the lawn mowing robot is controlled to detour to the left;

[0038] When the obstacle area in the left region is equal to the obstacle area in the right region, the lawn mowing robot is controlled to detour to the left or to the right.

[0039] In a second aspect, a position determination device is provided, comprising:

[0040] A first acquisition module is used to acquire a current position of the lawn mower robot with a scale determined by a visual odometer on the lawn mower robot after the lawn mower robot finishes mowing;

[0041] a second acquisition module, configured to acquire a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target moment and mowing trace information after mowing is completed at the target moment;

[0042] The position correction module is used to correct the current position according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

[0043] In a third aspect, a lawn mowing robot is provided, comprising:

[0044] Visual odometry;

[0045] Camera device;

[0046] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and one or more programs are configured to: perform operations corresponding to the method shown in any possible implementation of the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executes the steps of the method shown in any possible implementation of the first aspect.

[0048] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, operations corresponding to the method shown in any possible implementation manner in the first aspect are implemented.

[0049] In summary, the method provided by this application includes the following beneficial technical effects:

[0050] After the lawn mowing robot finishes mowing, the visual odometer on the lawn mowing robot is used to determine the current position of the lawn mowing robot with scale; the visual odometer may have errors, especially errors in the lateral position, so a lateral correction amount is obtained, which is based on the lateral distance value between the visual odometer position at the target moment and the mowing trace information before the target moment, and compared with a preset standard distance value; the current position is corrected according to the lateral correction amount to obtain a more accurate corrected position of the lawn mowing robot; not only is the cost low, but also the accuracy of position determination is improved, and the positioning ability of the lawn mowing robot in complex environments is enhanced.

[0051] In addition, the present application also provides a position determination device, a lawn mowing robot and a medium, all of which have the above-mentioned beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 is a flow chart of a location determination method provided in an embodiment of the present application;

[0054] Figure 2 is a flow chart of another location determination method provided in an embodiment of the present application;

[0055] Figure 3 is a flow chart of a method for determining a restoration scale factor provided in an embodiment of the present application;

[0056] Figure 4 It is a flowchart of an obstacle avoidance method provided in an embodiment of the present application;

[0057] Figure 5 It is a schematic diagram of the software and hardware structure of a lawn mowing robot provided in an embodiment of the present application;

[0058] Figure 6 is a structural schematic diagram of a position determination device provided in an embodiment of the present application;

[0059] Figure 7 It is a structural schematic diagram of a lawn mowing robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, a person skilled in the art may make non-creative modifications to the present embodiment as needed, but such modifications are protected by the patent law as long as they are within the scope of the present application.

[0061] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0064] Among the related technologies, lawn mowing robot technology includes the fusion application of multiple sensors and technologies, most of which rely on RTK+camera or UWB+camera for target recognition and positioning navigation. Although these technologies have the advantage of high precision, they have problems such as complex installation process, high cost, reliance on external signals, and environmental interference.

[0065] Therefore, pure vision technology, with its rich visual information, does not rely on external signal sources, has high autonomy and flexibility, and shows great potential in lawn maintenance scenarios. It is particularly important to develop an efficient, low-cost lawn mowing robot based on pure vision technology. However, vision technology also faces challenges such as changes in ambient light and complex obstacle recognition. How to overcome the limitations of its depth perception has become a hot topic in current research.

[0066] Based on this, the embodiment of the present application relates to the technical field of gardening automation equipment, specifically a lawn mowing robot and position determination method based on pure vision technology. By optimizing the image processing algorithm and enhancing the visual recognition capability, the problems of high cost, reliance on external signals and poor environmental adaptability in the prior art are solved, and the robot can achieve efficient, convenient and accurate intelligent lawn maintenance operations.

[0067] In some embodiments, the lawn mower robot provided in the embodiments of the present application includes a lawn mower robot visual perception system and an image processing and control unit. The visual perception system is composed of front and rear monocular high-definition camera devices, which can establish the charging pile as the starting coordinate of the lawn mower robot, capture the image of the lawn and the surrounding environment in real time, and decompose the key working area and non-working area of ​​the lawn mower robot. The image processing and control unit uses advanced computer vision algorithms to pre-process, extract features, identify and locate the image, and control the precise positioning and precise and efficient operation of the robot based on the recognition results.

[0068] The present application embodiment provides a method for determining a position, such as Figure 1 , Figure 2 As shown, the method provided in the embodiment of the present application can be executed by a lawn mowing robot, which is a server. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal device and the lawn mowing robot can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. The method includes:

[0069] S101, after the lawn mowing robot has finished mowing, obtaining a current position of the lawn mowing robot with a scale determined by a visual odometer on the lawn mowing robot;

[0070] The mowing robot generally performs the mowing task at a fixed mowing distance. After the mowing robot completes the mowing task, the visual odometer function can be activated; the visual odometer calculates and determines the position of the mowing robot by analyzing the images captured during the movement of the mowing robot, which is a position with scale.

[0071] S102, obtaining a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target time and mowing trace information after mowing is completed at the target time;

[0072] The lateral correction refers to the correction amount of the lateral (i.e., perpendicular to the forward direction) position deviation of the lawn mower robot made by the visual odometer during the estimation of the lawn mower robot's motion. It is used to adjust the motion trajectory of the lawn mower robot so that it can move along the predetermined path more accurately.

[0073] The target time may be the current time, or any time before the current time when mowing is completed.

[0074] The process of determining the lateral correction amount includes: SA1-SA4, wherein:

[0075] SA1, obtaining an image to be recognized of a lawn at a target time captured by a camera device on the lawn mowing robot;

[0076] The image to be identified is an image after lawn mowing is completed.

[0077] SA2, identifying the mowing traces on the image to be identified, and obtaining the mowing trace information;

[0078] The mowing traces left by the mowing robot during the mowing process (traces formed by the height difference of the grass before and after mowing) can be regarded as a series of point sets, which can be extracted from the image taken by the monocular camera and fitted into a straight line or curve as the mowing trace information. In some embodiments, the mowing trace recognition of the image to be recognized can be realized in a variety of ways:

[0079] Optionally, one implementation method is based on traditional image processing algorithms. Use tools such as Gaussian filters to preprocess the image to be identified to reduce noise interference; use algorithms such as Canny edge detection to detect edge information in the image to be identified, which is likely to be mowing marks; further process the edge information through morphological operations (such as expansion, corrosion, etc.) to extract more complete mowing marks; finally, perform linear fitting on the extracted mowing marks to obtain mowing mark information.

[0080] Alternatively, another implementation method is based on deep learning. Collect a large amount of image data containing mowing marks and annotate them; use these data to train a deep learning model (such as a convolutional neural network) so that it can automatically learn the characteristics of mowing marks; after the model training is completed, input the image to be identified into the model, identify the mowing marks through the model's reasoning process, output the mowing mark coordinates, and perform limited fitting on the mowing mark coordinates to obtain the mowing mark information.

[0081] It is understandable that other methods may be used to implement the mowing trace recognition on the image to be recognized, which is not limited here.

[0082] SA3, determining a lateral distance value between a target position determined by a visual odometer at a target time and the mowing trace information;

[0083] In the embodiment of the present application, when the mowing trace information is a straight line (0=Ax+By+C), the lateral distance between the current position (estimated by the visual SLAM (Simultaneous Localization and Mapping) odometer) and the last mowing trace information is It can be calculated using the distance formula from a point to a line: ;in is the current target position of the robot in the world coordinate system.

[0084] Exemplarily, after the mowing robot completes mowing at time a, the mowing lateral distance value identified at this time should be a preset standard distance value, and the target position at time a and the lateral distance value at time a are obtained.

[0085] When the mowing trace information is the curve y=f(x) and the current position is (x0,y0); find the point on the curve closest to the given point, that is, find the point such that (x−x0) 2 +(f(x)−y0) 2 The smallest x value; then, calculate the distance between the two points, which is (x−x0) 2 +(f(x)−y0) 2 , and the result is the distance from the point to the curve.

[0086] SA4. Determine the lateral correction amount of the visual odometer based on the lateral distance value and the preset standard distance value.

[0087] The preset standard distance value is a preset standard value, that is, a set distance between two adjacent mowing marks.

[0088] If the lateral distance value d exceeds the preset standard distance value, it means that the current visual odometer has lateral drift. At this time, the drift can be reduced by adjusting the output of the SLAM odometer. If the lateral distance value d is less than or equal to the preset standard distance value, it is considered that the trajectory estimation of the current visual odometer is accurate and there is no lateral drift.

[0089] The corrected robot position can be expressed as:

[0090] ;

[0091] in, is the position of the current visual odometry output, Based on the horizontal distance value The lateral correction vector calculated from the preset standard distance value.

[0092] In the embodiment of the present application, the lawn image to be identified at the current moment is obtained, and the mowing traces are identified to obtain the actual motion trajectory of the mowing robot, and then the lateral distance value from the visual odometer position is calculated. Then the lateral distance value is compared with the preset standard distance value to determine whether the current visual odometer has lateral drift, and if so, the lateral correction amount of the visual odometer is determined.

[0093] S103: Correct the current position according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

[0094] It can be seen that in the embodiment of the present application, after the lawn mowing robot finishes mowing, the visual odometer on the lawn mowing robot is used to determine the current position of the lawn mowing robot with a scale; the visual odometer may have errors, especially errors in the lateral position, and thus, a lateral correction amount is obtained, which is based on the lateral distance value between the visual odometer position at the target moment and the mowing trace information before the target moment, and compared with a preset standard distance value; the current position is corrected according to the lateral correction amount to obtain a more accurate corrected position of the lawn mowing robot; not only is the cost low, but also the accuracy of position determination is improved, and the positioning capability of the lawn mowing robot in complex environments is enhanced.

[0095] A possible implementation of the embodiment of the present application, before obtaining the current position of the lawn mower robot with scale determined by the visual odometer on the lawn mower robot, further includes:

[0096] Acquire a reverse image including a marker captured by a camera device on the lawn mower robot when the lawn mower robot reverses and starts from the charging pile;

[0097] According to the backward image, the scaled coordinates of the robot in the world coordinate system are determined as the first position information;

[0098] Obtain the scale-free coordinates of the robot in the world coordinate system calculated by the visual odometer as the second position information;

[0099] A restoration scale factor is determined according to the initial position, the first position information, and the second position information; the restoration scale factor is used to restore the scale of the position of the output of the control visual odometer to obtain the actual position, and the initial position is the initial position of the robot or the camera in the world coordinate system.

[0100] Before controlling the lawn mowing robot to perform the lawn mowing task, the monocular camera device needs to be initialized so as to convert the scale-free data of the visual odometer into scaled positions.

[0101] Take charging pile as an example, see Figure 3,The process of determining the scale factor includes: identifying the charging pile; initializing the position and posture of the world coordinates; outputting the position without scale information; restoring the scale factor through the algorithm and updating the initial position. Generally, the initial position is set to (0,0), and other positions are adaptively adjusted. Specifically,

[0102] The scaled coordinates of a robot in the world coordinate system calculated using the backward image and deep learning model The scale-free coordinates of the robot in the world coordinate system calculated by the visual odometry The scale factor is recovered by :

[0103] ;

[0104] in is the initial position of the robot or camera in the world coordinate system. Then, use To scale all positions output by the SLAM system:

[0105] .

[0106] In some possible implementations, based on the reverse image, the scaled coordinates of the robot in the world coordinate system are determined as the first position information, including: obtaining a position recognition model; the position recognition model is obtained by training a deep learning model through multiple training images including markers and their corresponding position information labels of the robots; based on the reverse image, position recognition is performed using the marker position recognition model to obtain the scaled coordinates of the marker in the robot coordinate system as the first position information.

[0107] In some possible implementations, before obtaining the location recognition model, it also includes: obtaining multiple training images including identifiers and their corresponding location information tags; inputting the training images into the deep learning model to obtain training location information; determining a loss value based on the training location information and the location information tags; when the loss value is greater than a preset loss threshold, training is performed based on the training images and their corresponding location information tags, and the network parameters of the deep learning model are updated until the loss value is no greater than the preset loss threshold, thereby obtaining a location recognition model, wherein the preset loss threshold can be custom set.

[0108] Specifically, a large number of images containing markers are collected, and the precise position and orientation of the robot is annotated based on the markers in these images (such as charging pile markers). A deep learning model is trained using the annotated dataset. The model can receive images captured by a camera device (such as a monocular camera) as input and obtain the position and orientation of the mowing robot camera, that is, the four-dimensional vector ,in are the scaled coordinates of the robot, is the angle of the charging pile logo relative to the robot's orientation, and s is the confidence score. Through training, a position recognition model is obtained.

[0109] It can be seen that in the embodiment of the present application, when the lawn mowing robot starts to move backward, the backward image captured by the camera device is input into the model, so as to quickly and accurately identify the position information of the robot in the world coordinate system.

[0110] See also Figure 4 The lawn mowing robot provided in the embodiment of the present application also has an obstacle avoidance function. Specifically, a possible implementation method of the embodiment of the present application also includes:

[0111] When the mowing robot walks according to the planned path and the corrected position, a real-time image of the lawn captured by a camera device on the mowing robot is obtained;

[0112] Perform image recognition based on real-time images to determine the pixel coordinates of the lowest point of the obstacle outline within the drivable area;

[0113] According to the posture parameters of the mowing robot, the camera internal parameters, the height information and the pixel coordinates of the lowest point of the contour, the actual distance between the obstacle and the mowing robot is determined;

[0114] When the actual distance is less than the set threshold, the mowing robot is controlled to perform obstacle avoidance operations.

[0115] Among them, a real-time image of the lawn captured by a camera device on the lawn mowing robot is obtained. Further, the image can be pre-processed, including denoising (such as using median filtering), color correction (ensuring stable image quality under different lighting conditions), and possible image enhancement (such as edge enhancement), format conversion, image color stripe processing, and proportional scaling to improve the accuracy of subsequent image recognition.

[0116] Furthermore, deep learning algorithms (such as YOLO, SSD and other target detection models) or traditional image processing techniques (such as edge detection, Hough transform, etc.) can be used to identify obstacles in the preprocessed images, or to infer semantic segmentation models; and morphological operations (such as erosion, dilation) and contour search algorithms can be used to extract the contours of obstacles; then the contour points are traversed to find the lowest point of the contour and determine the pixel coordinates.

[0117] The current posture parameters (including pitch angle, yaw angle, roll angle) and height information of the mowing robot collected by the built-in sensors (such as IMU, gyroscope, altimeter) of the mowing robot are obtained. The height information is the distance between the camera and the ground. The intrinsic parameters of the camera (focal length, optical center coordinates, etc.) and the known ground height information are used, combined with the pixel coordinates of the lowest point of the obstacle outline, and the pinhole camera model is used to convert the pixel coordinates into actual three-dimensional space coordinates, and then calculate the actual distance between the obstacle and the mowing robot.

[0118] In addition, a set threshold is set, which is a safety distance threshold. The set threshold can be set according to the size of the lawn mower robot and the walking speed of the lawn mower robot. For example, a first value is determined according to the size and a first preset relationship, and a second value is determined according to the speed and a second preset relationship. The sum of the first value and the second value is used as the safety distance threshold. The first preset relationship and the second preset relationship are set according to experience. The first preset relationship is a correspondence relationship between multiple sizes and multiple values, and the second preset relationship is a correspondence relationship between multiple speeds and multiple values.

[0119] Furthermore, if an obstacle suddenly appears in front of the lawn mower robot within a short period of time, a simple obstacle avoidance strategy such as emergency braking, rotating on the spot, or circling the edge of the obstacle can be adopted to ensure the safety of the obstacle and the lawn mower robot.

[0120] Furthermore, after the obstacle avoidance operation is completed, the obstacle avoidance effect can be evaluated to determine whether the obstacle is successfully avoided, whether the lawn mowing robot is scratched, etc.; if the obstacle avoidance effect is not good, the set threshold value can be increased to improve the subsequent obstacle avoidance effect.

[0121] It can be seen that in the embodiment of the present application, while the lawn mower robot is walking according to the planned path and the corrected position, a real-time image of the lawn captured by the camera device on the lawn mower robot is obtained; image recognition is performed based on the real-time image, and the pixel coordinates of the lowest point of the outline of the obstacle in the drivable area are determined; based on the posture parameters of the lawn mower robot and the pixel coordinates of the lowest point of the outline, the actual distance between the obstacle and the lawn mower robot is determined; when the actual distance is less than the set threshold, the lawn mower robot is controlled to perform obstacle avoidance operations; the distance measurement is completed, and precise obstacle avoidance is achieved.

[0122] A possible implementation of the embodiment of the present application is to control the lawn mowing robot to perform an obstacle avoidance operation when the actual distance is less than a set threshold, including:

[0123] Determine the obstacle area in the left area and the obstacle area in the right area in the real-time image;

[0124] When the obstacle area in the left area is larger than that in the right area, the mowing robot is controlled to go around to the right;

[0125] When the obstacle area in the left area is smaller than that in the right area, the mowing robot is controlled to go around to the left;

[0126] When the obstacle area in the left region is equal to the obstacle area in the right region, the lawn mowing robot is controlled to bypass to the left or to the right.

[0127] The left and right area boundaries in the real-time image are determined. Specifically, the left and right areas can be determined by a preset image width ratio. Then, the pixel areas of obstacles in the left and right areas are calculated respectively, and the robot is controlled to bypass the obstacles according to the pixel areas of the obstacles.

[0128] It is understandable that if the obstacle is a boundary obstacle such as a fence, a wall, etc., the vehicle will brake in time and stop running to wait for user instructions.

[0129] Furthermore, a path to avoid obstacles can be planned according to the detour direction. Specifically, the detour radius is determined according to the turning radius of the lawn mowing robot; and the detour path is planned based on the distribution of obstacles and the detour radius.

[0130] It can be seen that in the embodiment of the present application, by comparing the obstacle areas of the left and right regions, the side with a smaller area is intelligently selected as the detour direction, thereby improving the obstacle avoidance efficiency and enabling the device to cope with complex environments more flexibly.

[0131] Based on any of the above embodiments, it can be seen that Figure 5 The solution provided in the embodiment of the present application is a lawn mowing robot based on pure vision, which is a robust visual decision system, a precise actuator, and an efficient and stable software system. It is composed of a visual decision system, an actuator, and a software system. In the end, the lawn mowing robot's visual algorithm will generate the optimal mowing path to ensure that the lawn is evenly and efficiently covered; at the same time, the algorithm can adjust the path in real time to avoid obstacles and ensure operation safety.

[0132] The embodiments of the present application achieve low-cost, high-autonomy, and strong environmental adaptability lawn maintenance operations through innovative deep learning visual recognition and positioning technology; it not only improves the efficiency and quality of mowing, but also reduces the user's usage cost and maintenance difficulty, bringing new technological breakthroughs and development directions to the field of automated gardening equipment.

[0133] The following is an introduction to a device provided in an embodiment of the present application. The device described below and the method described above can be referenced to each other. The device of this embodiment is set in a lawn mowing robot. Figure 6 , Figure 6 It is a structural block diagram of a device of one embodiment of the present application, comprising:

[0134] A first acquisition module 210 is used to acquire a current position of the lawn mower robot with a scale determined by a visual odometer on the lawn mower robot after the lawn mower robot has finished mowing;

[0135] A second acquisition module 220 is used to acquire a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target time and mowing trace information after mowing is completed at the target time;

[0136] The position correction module 230 is used to correct the current position according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

[0137] In one achievable manner, the present invention further includes:

[0138] The module for determining the lateral correction is used for:

[0139] Acquire an image to be identified of the lawn at a target time captured by a camera device on the lawn mowing robot;

[0140] Identify the mowing traces on the image to be identified, and obtain the mowing trace information;

[0141] Determine the lateral distance value between the target position determined by the visual odometer at the target time and the mowing trace information;

[0142] The lateral correction amount of the visual odometer is determined according to the lateral distance value and the preset standard distance value.

[0143] In an achievable manner, the method further includes: a restoration scale factor determination module, configured to:

[0144] Acquire a reverse image including a marker captured by a camera device on the lawn mower robot when the lawn mower robot reverses and starts from the charging pile;

[0145] According to the backward image, the scaled coordinates of the robot in the world coordinate system are determined as the first position information;

[0146] Obtain the scale-free coordinates of the robot in the world coordinate system calculated by the visual odometer as the second position information;

[0147] A restoration scale factor is determined according to the initial position, the first position information, and the second position information; the restoration scale factor is used to restore the scale of the position of the output of the control visual odometer to obtain the actual position, and the initial position is the initial position of the robot or the camera in the world coordinate system.

[0148] In one achievable manner, the scale factor determination module is restored to:

[0149] Acquire a position recognition model; the position recognition model is obtained by training a deep learning model with a plurality of training images including a marker and respective corresponding robot position information labels;

[0150] According to the backward image, the position recognition model of the marker is used to perform position recognition, and the scaled coordinates of the marker in the robot coordinate system are obtained as the first position information.

[0151] In one achievable manner, the present invention further includes:

[0152] A training module is used to: obtain a plurality of training images including markers and their corresponding position information labels;

[0153] Input the training image into the deep learning model to obtain the training position information;

[0154] Determine the loss value based on the training position information and the position information label;

[0155] When the loss value is greater than the preset loss threshold, training is performed based on the training images and their corresponding location information labels, and the network parameters of the deep learning model are updated until the loss value is no greater than the preset loss threshold, thereby obtaining a location recognition model.

[0156] In one achievable manner, the present invention further includes:

[0157] The obstacle avoidance module is used to obtain a real-time image of the lawn captured by a camera device on the lawn mower robot while the lawn mower robot is walking along the planned path and the corrected position;

[0158] Perform image recognition based on real-time images to determine the pixel coordinates of the lowest point of the obstacle outline within the drivable area;

[0159] According to the posture parameters of the mowing robot and the pixel coordinates of the lowest point of the contour, the actual distance between the obstacle and the mowing robot is determined;

[0160] When the actual distance is less than the set threshold, the mowing robot is controlled to perform obstacle avoidance operations.

[0161] In one achievable manner, the obstacle avoidance module is used to determine the obstacle area in the left region and the obstacle area in the right region in the real-time image;

[0162] When the obstacle area in the left area is larger than that in the right area, the mowing robot is controlled to go around to the right;

[0163] When the obstacle area in the left area is smaller than that in the right area, the mowing robot is controlled to go around to the left;

[0164] When the obstacle area in the left region is equal to the obstacle area in the right region, the lawn mowing robot is controlled to bypass to the left or to the right.

[0165] The present application provides a lawn mowing robot, such as Figure 7 As shown, Figure 7 The lawn mower robot 300 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the lawn mower robot 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the lawn mower robot 300 does not constitute a limitation on the embodiments of the present application.

[0166] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0167] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0168] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0169] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.

[0170] Figure 7 The lawn mowing robot shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0171] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0172] An embodiment of the present application provides a computer program product, including a computer program, which implements the corresponding contents of the aforementioned method embodiment when the computer program is executed by a processor.

[0173] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0174] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a position, characterized in that: include: After the lawn mowing robot completes mowing, obtaining a current position of the lawn mowing robot with a scale determined by a visual odometer on the lawn mowing robot; Obtaining a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target time and mowing trace information after mowing is completed at the target time; The current position is corrected according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

2. The method according to claim 1, characterized in that After obtaining the current position of the mowing robot with scale determined by the visual odometer on the mowing robot, it also includes: Acquire an image to be identified of the lawn at a target time captured by a camera device on the lawn mowing robot; Identifying mowing traces on the image to be identified to obtain mowing trace information; Determine a lateral distance value between a target position determined by a visual odometer at a target time and the mowing trace information; A lateral correction amount of the visual odometer is determined according to the lateral distance value and a preset standard distance value.

3. The method according to claim 1, characterized in that Before obtaining the current position of the lawn mowing robot with scale determined by the visual odometer on the lawn mowing robot, the method further includes: Acquire a reverse image including a marker captured by a camera device on the lawn mower robot when the lawn mower robot reverses and starts from the charging pile; Determine, according to the reverse image, the scaled coordinates of the robot in the world coordinate system as first position information; Obtain the scale-free coordinates of the robot in the world coordinate system calculated by the visual odometer as the second position information; A restoration scale factor is determined according to the initial position, the first position information, and the second position information; the restoration scale factor is used to restore the scale of the position of the output of the control visual odometer to obtain the actual position, and the initial position is the initial position of the robot or the camera in the world coordinate system.

4. The method according to claim 3, characterized in that Determining the scaled coordinates of the robot in the world coordinate system according to the backward image as first position information includes: Acquire a position recognition model; the position recognition model is obtained by training a deep learning model through a plurality of training images including a marker and position information labels of the robots corresponding to the respective training images; According to the reverse image, the position recognition model of the marker is used to perform position recognition, and the scaled coordinates of the marker in the robot coordinate system are obtained as the first position information.

5. The method according to claim 4, characterized in that Before obtaining the location recognition model, it also includes: Acquire a plurality of training images including markers and their corresponding position information labels; Input the training image into the deep learning model to obtain the training position information; Determine a loss value based on the training position information and the position information label; When the loss value is greater than the preset loss threshold, training is performed based on the training images and their corresponding location information labels, and the network parameters of the deep learning model are updated until the loss value is no greater than the preset loss threshold, thereby obtaining a location recognition model.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: When the mowing robot walks according to the planned path and the corrected position, a real-time image of the lawn captured by a camera device on the mowing robot is obtained; Performing image recognition based on the real-time image to determine the pixel coordinates of the lowest point of the outline of the obstacle in the drivable area; According to the posture parameters of the mowing robot, the camera internal parameters, the height information and the pixel coordinates of the lowest point of the contour, the actual distance between the obstacle and the mowing robot is determined; When the actual distance is less than the set threshold, the mowing robot is controlled to perform obstacle avoidance operations.

7. The method according to claim 6, characterized in that When the actual distance is less than the set threshold, the mowing robot is controlled to perform obstacle avoidance operations, including: Determine the obstacle area in the left region and the obstacle area in the right region in the real-time image; When the area of ​​the obstacles in the left region is larger than the area of ​​the obstacles in the right region, the lawn mowing robot is controlled to detour to the right; When the obstacle area in the left region is smaller than the obstacle area in the right region, the lawn mowing robot is controlled to detour to the left; When the obstacle area in the left region is equal to the obstacle area in the right region, the lawn mowing robot is controlled to detour to the left or to the right.

8. A position determination device, characterized in that: include: A first acquisition module is used to acquire a current position of the lawn mower robot with a scale determined by a visual odometer on the lawn mower robot after the lawn mower robot finishes mowing; a second acquisition module, configured to acquire a lateral correction value of the visual odometer, wherein the lateral correction value is determined according to a lateral distance value and a preset standard distance value, and the lateral distance value is a lateral distance value between a target position determined by the visual odometer at a target moment and mowing trace information after mowing is completed at the target moment; The position correction module is used to correct the current position according to the lateral correction amount to obtain a corrected position of the lawn mowing robot.

9. A lawn mowing robot, characterized in that: include: Visual odometry; Camera device; one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by the processor and executes the steps of the method according to any one of claims 1 to 7.

Images