Control method and device of mowing robot, electronic equipment and storage medium

Abstract

The embodiment of the present application provides a control method of a mowing robot, first detection is carried out in the running process of the mowing robot through a first detection mode, if an obstacle is detected, a first distance of the obstacle is obtained; when the first distance is greater than a preset distance threshold, second detection is carried out in the running process of the mowing robot through a second detection mode, and when the obstacle is detected, a second distance of the obstacle is obtained, the first detection mode and the second detection mode are different modal detection modes; the mowing robot is controlled to avoid the obstacle based on the first distance or the second distance. The present application fully utilizes the second detection mode to further perceive the obstacle, avoids the perception deviation of the first detection mode, thereby improving the perception accuracy of the mowing robot for the obstacle, and further improving the obstacle avoidance success rate of the mowing robot.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology, and in particular to a control method, device, electronic equipment and storage medium for a lawnmower robot. Background Technology

[0002] A lawnmower robot is a robot capable of autonomously mowing lawns along a planned route. Lawnmower robots typically use sensors such as cameras or radar to perceive external information. However, because lawnmower robots rely on trained image detection models for obstacle perception, they sometimes experience inaccuracies in distance measurement during autonomous mowing. Furthermore, they often fail to accurately perceive untrained image categories, frequently resulting in collisions with obstacles. Therefore, existing lawnmower robots have low obstacle perception accuracy, leading to a low obstacle avoidance success rate. Summary of the Invention

[0003] This invention provides a control method for a lawnmower robot, aiming to solve the problem of low obstacle perception accuracy in existing lawnmower robots, resulting in low obstacle avoidance success rates. A first detection method is used to obtain a first distance to the obstacle. When the first distance exceeds a preset distance threshold, a second detection method is used to obtain a second distance to the obstacle. Since the first and second detection methods are different modalities, the second detection method is fully utilized to further perceive the obstacle, avoiding perception deviations in the first detection method. This improves the obstacle perception accuracy of the lawnmower robot, thereby increasing its obstacle avoidance success rate.

[0004] In a first aspect, embodiments of the present invention provide a control method for a lawnmower robot, the control method comprising the following steps:

[0005] The first detection is performed during the operation of the lawnmower robot using a first detection method. If an obstacle is detected, the first distance of the obstacle is obtained.

[0006] When the first distance is greater than a preset distance threshold, a second detection is performed during the operation of the lawnmower robot using a second detection method. When the obstacle is detected, the second distance of the obstacle is obtained. The first detection method and the second detection method are detection methods of different modalities.

[0007] The lawnmower robot is controlled to avoid obstacles based on the first distance or the second distance.

[0008] Optionally, the first detection method is a visual detection method. The step of performing a first detection during the operation of the lawnmower robot using the first detection method, and obtaining a first distance to the obstacle if an obstacle is detected, further includes:

[0009] During the operation of the lawnmower robot, images to be detected are acquired;

[0010] The image to be detected is visually detected using a preset visual detection model;

[0011] If the obstacle is detected, the obstacle is identified by distance using a preset distance recognition model.

[0012] Optionally, before the step of performing visual detection on the image to be detected using a preset visual detection model, the method further includes:

[0013] Obtain sample images of various types of lawns;

[0014] The obstacles in the sample image are labeled to obtain a first labeled image;

[0015] A training set is formed based on the first labeled image;

[0016] Constructing a convolutional neural network;

[0017] The convolutional neural network is trained using the training set to obtain a visual detection model.

[0018] Optionally, the second detection method is an ultrasonic detection method. The step of performing a second detection during the operation of the lawnmower robot by means of the second detection method when the first distance is greater than a preset distance threshold, and obtaining the second distance of the obstacle when the obstacle is detected, includes:

[0019] When the first distance is greater than a preset distance threshold, the ultrasonic sensing result is obtained;

[0020] If the obstacle is detected in the ultrasonic sensing results, then the second distance of the obstacle is obtained.

[0021] Optionally, the step of controlling the lawnmower robot to avoid obstacles based on the first distance or the second distance includes:

[0022] If the first distance is less than the preset distance threshold, then obstacle avoidance control is performed on the lawnmower robot based on the first distance;

[0023] If the first distance is greater than the preset distance threshold, then obstacle avoidance control is performed on the lawnmower robot based on the second distance.

[0024] Optionally, after the step of performing obstacle avoidance control on the lawnmower robot based on the first distance if the first distance is less than the preset distance threshold, the method further includes:

[0025] The image to be detected is labeled with obstacles to obtain a second labeled image.

[0026] The visual detection model is trained online using the second labeled image.

[0027] Optionally, after the step of performing obstacle avoidance control on the lawnmower robot based on the second distance if the first distance is greater than the preset distance threshold, the method further includes:

[0028] The location of the obstacle is determined based on the ultrasonic sensing results;

[0029] The position of the obstacle in the image to be detected is determined based on the position of the obstacle;

[0030] The obstacles are marked according to their positions in the image to be detected, resulting in a third marked image;

[0031] The visual detection model is trained online using the third labeled image.

[0032] Secondly, embodiments of the present invention also provide a control device for a lawnmower robot, the control device for the lawnmower robot comprising the following steps:

[0033] The first detection module is used to perform a first detection during the operation of the lawnmower robot using a first detection method. If an obstacle is detected, the first distance of the obstacle is obtained.

[0034] The second detection module is used to perform a second detection during the operation of the lawnmower robot by means of a second detection method when the first distance is greater than a preset distance threshold, and to obtain the second distance of the obstacle when the obstacle is detected. The first detection method and the second detection method are detection methods of different modes.

[0035] The control module is used to control the lawnmower robot to avoid obstacles based on the first distance or the second distance.

[0036] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the control method for a lawnmower robot as described in any one of the embodiments of the present invention.

[0037] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the lawnmower control method as described in any one of the embodiments of the present invention.

[0038] In this embodiment of the invention, a first detection method is used during the operation of the lawnmower robot. If an obstacle is detected, a first distance to the obstacle is obtained. When the first distance is greater than a preset distance threshold, a second detection method is used during the operation of the lawnmower robot, and when the obstacle is detected, a second distance to the obstacle is obtained. The first and second detection methods are different modal detection methods. Obstacle avoidance control is performed on the lawnmower robot based on the first or second distance. By obtaining the first distance to the obstacle through the first detection method and obtaining the second distance to the obstacle through the second detection method when the first distance is greater than the preset distance threshold, the second detection method is fully utilized to further perceive the obstacle, avoiding the perception deviation of the first detection method. This improves the lawnmower robot's accuracy in perceiving obstacles and thus improves the obstacle avoidance success rate of the lawnmower robot. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a control method for a lawnmower robot provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of a ranging device provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Please see Figure 1 , Figure 1 This is a flowchart of a control method for a lawnmower robot provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the control method for this lawnmower robot is specifically used for lawnmower robots, and the control method for this lawnmower robot includes the following steps:

[0045] 101. The first detection is performed during the operation of the lawnmower robot using the first detection method. If an obstacle is detected, the first distance of the obstacle is obtained.

[0046] In this embodiment of the invention, the first detection method can be one of visual detection, ultrasonic detection, laser detection, etc. The lawnmower robot integrates a first detection device corresponding to the first detection method, such as a visual detection device corresponding to the visual detection method, an ultrasonic detection device corresponding to the ultrasonic detection method, and a laser detection device corresponding to the laser detection method. The first detection device includes an acquisition device and a processing chip. For example, the visual detection device may include an image acquisition device and an image processing chip, wherein the image processing chip is signal-connected to the controller of the lawnmower robot; the ultrasonic detection device may include an ultrasonic transceiver device and an ultrasonic signal processing chip, wherein the ultrasonic signal processing chip is signal-connected to the controller of the lawnmower robot; the laser detection device may include a laser transceiver device and a laser signal processing chip, wherein the laser signal processing chip is signal-connected to the controller of the lawnmower robot.

[0047] During the operation of the lawnmower robot, the first detection device senses the target grass and obtains first sensing information. The processing chip processes the first sensing information to detect whether there are obstacles in the first sensing information. If no obstacles are detected, the lawnmower continues to run along the planned route. If an obstacle is detected in the first sensing information, the first distance of the obstacle is extracted from the first sensing information.

[0048] The aforementioned distance threshold can be determined based on the detection accuracy of the first detection method. It can be the product of the detection accuracy and the maximum effective detection distance of the first detection method. The maximum effective detection distance of the first detection method can be determined based on the factory parameters of the first detection device. For example, if the factory parameters of the first detection device are a maximum effective detection distance of 10m and a detection accuracy of 95%, then the distance threshold can be set to 10m * 95% = 9.5m.

[0049] 102. When the first distance is greater than the preset distance threshold, a second detection is performed during the operation of the lawnmower robot using a second detection method, and when an obstacle is detected, the second distance of the obstacle is obtained.

[0050] In this embodiment of the invention, the first detection method and the second detection method are detection methods of different modes. The aforementioned different modes refer to signals with different modes, such as image signals, ultrasonic signals, laser signals, and other signals of different modes.

[0051] The aforementioned distance threshold can be used as a trigger condition for the second detection method. When the first distance is greater than the preset distance threshold, it indicates that the accuracy of the first detection method has decreased, thereby triggering the second detection method.

[0052] The second detection method mentioned above can be one of the following: visual detection, ultrasonic detection, laser detection, etc., and the second detection method is different from the first detection method mentioned above. The lawnmower robot integrates a second detection device corresponding to the second detection method, such as a visual detection device corresponding to the visual detection method, an ultrasonic detection device corresponding to the ultrasonic detection method, and a laser detection device corresponding to the laser detection method. The second detection device includes an acquisition device and a processing chip. For example, the visual detection device may include an image acquisition device and an image processing chip, wherein the image processing chip is connected to the controller of the lawnmower robot; the ultrasonic detection device may include an ultrasonic transceiver device and an ultrasonic signal processing chip, wherein the ultrasonic signal processing chip is connected to the controller of the lawnmower robot; the laser detection device may include a laser transceiver device and a laser signal processing chip, wherein the laser signal processing chip is connected to the controller of the lawnmower robot.

[0053] During the operation of the lawnmower robot, when the first distance exceeds a preset distance threshold, the target grass can be sensed by the acquisition device in the second detection device to obtain second sensing information. The processing chip processes the second sensing information to detect whether there are obstacles in the second sensing information. If no obstacles are detected in the second sensing information, the lawnmower robot continues to run along the planned route. If obstacles are detected in the second sensing information, the second distance of the obstacle is extracted from the second sensing information.

[0054] 103. Based on the first distance or the second distance, perform obstacle avoidance control on the lawnmower robot.

[0055] In this embodiment of the invention, the first distance is the obstacle distance obtained based on the first detection method, and the second distance is the obstacle distance obtained based on the second detection method, provided that the first distance is greater than a preset distance threshold. When the first distance is less than the preset distance threshold, the route of the lawnmower robot can be directly modified based on the first distance to obtain a modified route, and obstacle avoidance control of the lawnmower robot can be performed based on the modified route. When the first distance is greater than the preset distance threshold, the route of the lawnmower robot can be directly modified based on the second distance to obtain a modified route, and obstacle avoidance control of the lawnmower robot can be performed based on the modified route.

[0056] In this embodiment of the invention, a first detection method is used during the operation of the lawnmower robot. If an obstacle is detected, a first distance to the obstacle is obtained. When the first distance is greater than a preset distance threshold, a second detection method is used during the operation of the lawnmower robot, and when the obstacle is detected, a second distance to the obstacle is obtained. The first and second detection methods are different modal detection methods. Obstacle avoidance control is performed on the lawnmower robot based on the first or second distance. By obtaining the first distance to the obstacle through the first detection method and obtaining the second distance to the obstacle through the second detection method when the first distance is greater than the preset distance threshold, the second detection method is fully utilized to further perceive the obstacle, avoiding the perception deviation of the first detection method. This improves the lawnmower robot's accuracy in perceiving obstacles and thus improves the obstacle avoidance success rate of the lawnmower robot.

[0057] Optionally, the first detection method is a visual detection method. In the step of performing the first detection during the operation of the lawnmower robot and obtaining the first distance of the obstacle if an obstacle is detected, the image to be detected can be acquired during the operation of the lawnmower robot; the image to be detected can be visually detected using a preset visual detection model; if an obstacle is detected, the obstacle can be identified by distance using a preset distance recognition model.

[0058] In this embodiment of the invention, the first detection method is preferably a visual detection method. During the operation of the lawnmower robot, an image acquisition device in the visual detection equipment acquires a lawn image. The image processing chip processes the lawn image to detect whether there are obstacles in the lawn image. If no obstacles are detected in the lawn image, the lawnmower robot continues to run along the planned route. If an obstacle is detected in the lawn image, the first distance of the obstacle is extracted from the lawn image.

[0059] The image processing chip incorporates a visual detection model and a distance recognition model. The visual detection model performs visual detection on the grass image to identify whether there are obstacles in the grass image, and the distance recognition model extracts the first distance of the obstacle.

[0060] Optionally, before performing visual detection on the image to be detected using a preset visual detection model, sample images of various types of lawns can be obtained; obstacles in the sample images can be labeled to obtain a first labeled image; a training set can be formed based on the first labeled image; a convolutional neural network can be constructed; and the convolutional neural network can be trained using the training set to obtain a visual detection model.

[0061] In this embodiment of the invention, the above-mentioned visual detection model can be constructed based on a convolutional neural network. By training the convolutional neural network with a training set, a visual detection model for detecting obstacles can be obtained.

[0062] Specifically, sample images of various types of lawns are acquired. These sample images include both grass and obstacles, with obstacles labeled in the sample images to form the first labeled images. These first labeled images can be used as a training set to train a convolutional neural network (CNN). During training, the CNN learns obstacle classification and regression, resulting in a trained CNN that serves as a visual detection model. This model is then used to detect the presence of obstacles in the grass images.

[0063] Optionally, the distance recognition model described above can also be built based on a convolutional neural network. By training the convolutional neural network with a training set, a distance recognition model for extracting obstacle distances can be obtained.

[0064] Specifically, sample images of various types of lawns captured by the lawnmower robot are acquired. These sample images include both grass and obstacles. The actual distance between the obstacles and the lawnmower robot is labeled in the sample images, forming a fourth labeled image. This fourth labeled image can be used as a training set to train a convolutional neural network (CNN). During training, the CNN learns to regress obstacle distances, resulting in a trained CNN that serves as a distance recognition model. This model is then used to extract the first distance of obstacles from the grass images.

[0065] Currently, the aforementioned visual detection model and distance recognition model can be constructed based on the same convolutional neural network. Specifically, sample images of various types of lawns captured by the lawnmower robot are acquired. These sample images include both grass and obstacles. Obstacles are labeled in the sample images, and the actual distance between the obstacles and the lawnmower robot is also labeled, forming a fifth labeled image. This fifth labeled image can be used as a training set to train the convolutional neural network. During training, the convolutional neural network learns obstacle classification and regression, and simultaneously learns obstacle distance regression calculation. The resulting trained convolutional neural network serves as the visual detection model. This model detects the presence of obstacles in the grass images and extracts the first distance of the obstacles from the grass images.

[0066] Optionally, the second detection method is ultrasonic detection. In the step of performing a second detection during the operation of the lawnmower robot when the first distance is greater than a preset distance threshold, and obtaining the second distance of the obstacle when an obstacle is detected, the ultrasonic sensing result can be obtained when the first distance is greater than the preset distance threshold; if there is an obstacle in the ultrasonic sensing result, the second distance of the obstacle is obtained.

[0067] In this embodiment of the invention, the second detection method is preferably an ultrasonic detection method. The ultrasonic sensing result includes an ultrasonic signal. When the first distance is greater than a preset distance threshold, the second detection method is activated. The ultrasonic signal is acquired by the ultrasonic transceiver in the ultrasonic detection device, and processed by the ultrasonic signal processing chip to detect whether there is an obstacle signal in the ultrasonic signal. If no obstacle signal is detected, the lawnmower continues to run along the planned route. If an obstacle signal is detected, a second distance to the obstacle is calculated based on the ultrasonic signal. In calculating the second distance to the obstacle using the ultrasonic signal, the flight time and flight speed of the ultrasonic signal are mainly used to calculate the flight distance, thereby obtaining the second distance to the obstacle.

[0068] Optionally, in the step of controlling the lawnmower robot to avoid obstacles based on a first distance or a second distance, if the first distance is less than a preset distance threshold, then the lawnmower robot is controlled to avoid obstacles based on the first distance; if the first distance is greater than the preset distance threshold, then the lawnmower robot is controlled to avoid obstacles based on the second distance.

[0069] In this embodiment of the invention, the first distance is the obstacle distance obtained based on the first detection method, and the second distance is the obstacle distance obtained based on the second detection method, provided that the first distance is greater than a preset distance threshold. When the first distance is less than the preset distance threshold, the route of the lawnmower robot can be directly modified based on the first distance to obtain a modified route, and obstacle avoidance control of the lawnmower robot can be performed based on the modified route. When the first distance is greater than the preset distance threshold, the route of the lawnmower robot can be directly modified based on the second distance to obtain a modified route, and obstacle avoidance control of the lawnmower robot can be performed based on the modified route.

[0070] Optionally, after the step of controlling the lawnmower robot to avoid obstacles based on the first distance if the first distance is less than a preset distance threshold, the image to be detected can be labeled with obstacles to obtain a second labeled image; the visual detection model can then be trained online using the second labeled image.

[0071] In this embodiment of the invention, the first detection method is a visual detection method, and the first distance is a distance value obtained based on the visual detection method. When the first distance is less than a preset distance threshold, it indicates that the obstacle is within the effective detection range of the first detection method. The image to be detected is the grass image corresponding to the moment the obstacle is detected. The obstacle is marked in the grass image in the form of a label to obtain a second labeled image. The pre-trained visual detection model is trained online using the second labeled image, so that the visual detection model can continuously optimize for the current grass and improve the obstacle detection accuracy of the visual detection model.

[0072] Optionally, after the step of controlling the lawnmower robot to avoid obstacles based on the second distance if the first distance is greater than a preset distance threshold, the position of the obstacle can be determined based on the ultrasonic sensing results; the position of the obstacle in the image to be detected can be determined based on the position of the obstacle; the obstacle can be labeled based on its position in the image to be detected to obtain a third labeled image; and the visual detection model can be trained online using the third labeled image.

[0073] In this embodiment of the invention, the second detection method is ultrasonic detection, and the second distance is a distance value obtained based on ultrasonic detection. The first detection method is visual detection, and the first distance is a distance value obtained based on visual detection. When the first distance is greater than a preset distance threshold, it indicates that the obstacle is not within the effective detection range of the first detection method. The image to be detected is the grass image corresponding to the moment the obstacle is detected. The obstacle is marked in the grass image with a label to obtain a second labeled image. The second labeled image is used to train the pre-trained visual detection model online, so that the visual detection model can be continuously optimized for the current grass, thereby improving the effective detection range of the visual detection model.

[0074] In one possible embodiment, obstacles can be labeled in the grass image, and a second distance can be used as the actual distance of the obstacles and labeled in the grass image to obtain a sixth labeled image. The trained visual detection model can be trained online using the sixth labeled image, so that the visual detection model can be continuously optimized for the current grass and further improve the effective detection range of the visual detection model.

[0075] In this embodiment of the invention, a first distance to an obstacle can be obtained through a first detection method. When the first distance is greater than a preset distance threshold, a second distance to the obstacle can be obtained through a second detection method. Since the first detection method and the second detection method are different modal detection methods, the second detection method is fully utilized to further perceive the obstacle, avoiding the perception deviation of the first detection method, thereby improving the perception accuracy of the lawnmower robot for obstacles, and thus improving the obstacle avoidance success rate of the lawnmower robot.

[0076] It should be noted that the control method for the lawnmower robot provided in this embodiment of the invention can be applied to devices such as smartphones, computers, and servers.

[0077] Optional, please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a control device for a lawnmower robot provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the device includes:

[0078] The first detection module 201 is used to perform a first detection during the operation of the lawnmower robot using a first detection method. If an obstacle is detected, the first distance of the obstacle is obtained.

[0079] The second detection module 202 is used to perform a second detection during the operation of the lawnmower robot by means of a second detection method when the first distance is greater than a preset distance threshold, and to obtain the second distance of the obstacle when the obstacle is detected. The first detection method and the second detection method are detection methods of different modes.

[0080] The control module 203 is used to control the lawnmower robot to avoid obstacles based on the first distance or the second distance.

[0081] Optionally, the first detection method is a visual detection method, and the first detection module 201 includes:

[0082] The first acquisition submodule is used to acquire the image to be detected during the operation of the lawnmower robot;

[0083] The first detection submodule is used to perform visual detection on the image to be detected using a preset visual detection model;

[0084] The identification submodule is used to identify the obstacle by distance using a preset distance identification model if the obstacle is detected.

[0085] Optionally, the device further includes:

[0086] The first acquisition module is used to acquire sample images of various types of lawns;

[0087] The first annotation module is used to annotate the obstacles in the sample image to obtain a first annotated image;

[0088] The body shape formation module forms a training set based on the first labeled image;

[0089] Modules for building convolutional neural networks;

[0090] The first training module is used to train the convolutional neural network using the training set to obtain a visual detection model.

[0091] Optionally, the second detection method is an ultrasonic detection method, and the second detection module 202 includes:

[0092] The second acquisition submodule is used to acquire ultrasonic sensing results when the first distance is greater than a preset distance threshold.

[0093] The third acquisition submodule is used to acquire the second distance of the obstacle if the obstacle is present in the ultrasonic sensing result.

[0094] Optionally, the control module 203 includes:

[0095] The first control submodule is used to control the lawnmower robot to avoid obstacles based on the first distance if the first distance is less than the preset distance threshold.

[0096] The second control submodule is used to control the lawnmower robot to avoid obstacles based on the second distance if the first distance is greater than the preset distance threshold.

[0097] Optionally, the device further includes:

[0098] The second annotation module is used to annotate the image to be detected by detecting the obstacle, thereby obtaining a second annotated image;

[0099] The second training module is used to train the visual detection model online using the second labeled image.

[0100] Optionally, the device further includes:

[0101] The first determining module is used to determine the position of the obstacle based on the ultrasonic sensing results;

[0102] The second determining module is used to determine the position of the obstacle in the image to be detected based on the position of the obstacle;

[0103] The third annotation module is used to annotate the obstacles according to their positions in the image to be detected, thereby obtaining a third annotated image;

[0104] The third training module is used to train the visual detection model online using the third labeled image.

[0105] It should be noted that the control device for the lawnmower robot provided in this embodiment of the invention can be applied to devices such as smartphones, computers, and servers that can perform layer-level business analysis.

[0106] The ranging device provided in this embodiment of the invention can realize all the processes implemented by the control method of the lawnmower robot in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0107] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program for a vehicle obstacle avoidance method stored in the memory 302 and executable on the processor 301, wherein:

[0108] The processor 301 is used to call the computer program stored in the memory 302 and perform the following steps:

[0109] The first detection is performed during the operation of the lawnmower robot using a first detection method. If an obstacle is detected, the first distance of the obstacle is obtained.

[0110] When the first distance is greater than a preset distance threshold, a second detection is performed during the operation of the lawnmower robot using a second detection method. When the obstacle is detected, the second distance of the obstacle is obtained. The first detection method and the second detection method are detection methods of different modalities.

[0111] The lawnmower robot is controlled to avoid obstacles based on the first distance or the second distance.

[0112] Optionally, the first detection method is a visual detection method. The step of performing the first detection through the first detection method during the operation of the lawnmower robot, executed by the processor 301, and obtaining the first distance of the obstacle if an obstacle is detected, further includes:

[0113] During the operation of the lawnmower robot, images to be detected are acquired;

[0114] The image to be detected is visually detected using a preset visual detection model;

[0115] If the obstacle is detected, the obstacle is identified by distance using a preset distance recognition model.

[0116] Optionally, before the step of performing visual detection on the image to be detected using a preset visual detection model, the method executed by the processor 301 further includes:

[0117] Obtain sample images of various types of lawns;

[0118] The obstacles in the sample image are labeled to obtain a first labeled image;

[0119] A training set is formed based on the first labeled image;

[0120] Constructing a convolutional neural network;

[0121] The convolutional neural network is trained using the training set to obtain a visual detection model.

[0122] Optionally, the second detection method is an ultrasonic detection method. The step executed by the processor 301, which involves performing a second detection during the operation of the lawnmower robot when the first distance is greater than a preset distance threshold, and obtaining the second distance of the obstacle when the obstacle is detected, includes:

[0123] When the first distance is greater than a preset distance threshold, the ultrasonic sensing result is obtained;

[0124] If the obstacle is detected in the ultrasonic sensing results, then the second distance of the obstacle is obtained.

[0125] Optionally, the steps performed by the processor 301 to control the lawnmower robot for obstacle avoidance based on the first distance or the second distance include:

[0126] If the first distance is less than the preset distance threshold, then obstacle avoidance control is performed on the lawnmower robot based on the first distance;

[0127] If the first distance is greater than the preset distance threshold, then obstacle avoidance control is performed on the lawnmower robot based on the second distance.

[0128] Optionally, after the step of performing obstacle avoidance control on the lawnmower robot based on the first distance if the first distance is less than the preset distance threshold, the method executed by the processor 301 further includes:

[0129] The image to be detected is labeled with obstacles to obtain a second labeled image.

[0130] The visual detection model is trained online using the second labeled image.

[0131] Optionally, after the step of performing obstacle avoidance control on the lawnmower robot based on the second distance if the first distance is greater than the preset distance threshold, the method executed by the processor 301 further includes:

[0132] The location of the obstacle is determined based on the ultrasonic sensing results;

[0133] The position of the obstacle in the image to be detected is determined based on the position of the obstacle;

[0134] The obstacles are marked according to their positions in the image to be detected, resulting in a third marked image;

[0135] The visual detection model is trained online using the third labeled image.

[0136] It should be noted that the electronic device provided in the embodiments of the present invention can be applied to devices such as smartphones, navigation devices, computers, and servers that can control lawnmower robots.

[0137] The electronic device provided in this embodiment of the invention can realize all the processes implemented by the control method of the lawnmower robot in the above method embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0138] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the lawnmower control method or the application-side lawnmower control method provided in this invention, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0140] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A control method for a lawnmower robot, characterized in that, The method comprises the following steps: The first detection is performed on the lawn mowing robot during operation by a first detection method, and if an obstacle is detected, a first distance of the obstacle is obtained; When the first distance is greater than a preset distance threshold, the second detection is performed on the lawn mowing robot during operation by a second detection method, and when the obstacle is detected, a second distance of the obstacle is obtained, and the first detection method and the second detection method are different modal detection methods; The lawn mowing robot is controlled to avoid obstacles based on the first distance or the second distance; The step of controlling the lawn mowing robot to avoid obstacles based on the first distance or the second distance comprises: If the first distance is less than the preset distance threshold, the lawn mowing robot is controlled to avoid obstacles based on the first distance; If the first distance is greater than the preset distance threshold, the lawn mowing robot is controlled to avoid obstacles based on the second distance; The distance threshold can be determined according to the detection accuracy of the first detection method, and is the product of the detection accuracy and the maximum effective detection distance of the first detection method; The real distance of the obstacle from the lawn mowing robot is labeled in the form of a label in the sample image to form a fourth labeled image, and the fourth labeled image is used as a training set to train a distance recognition model based on a convolutional neural network; the distance recognition model is used to extract the first distance; The obstacle is labeled in the form of a label in the sample image, and the real distance of the obstacle from the lawn mowing robot is labeled in the form of a label in the sample image to form a fifth labeled image, and the fifth labeled image is used as a training set to train a visual detection model and a distance recognition model based on a convolutional neural network; the visual detection model is used to detect whether there is an obstacle.

2. The control method of the mowing robot according to claim 1, wherein The first detection method is a visual detection method, and the step of performing the first detection on the lawn mowing robot during operation by the first detection method, and if an obstacle is detected, obtaining a first distance of the obstacle further comprises: During operation of the lawn mowing robot, a to-be-detected image is obtained; The to-be-detected image is visually detected by a preset visual detection model; If the obstacle is detected, the distance of the obstacle is recognized by a preset distance recognition model.

3. The control method of the mowing robot according to claim 2, wherein Before the step of visually detecting the to-be-detected image by the preset visual detection model, the method further comprises: Obtaining sample images of various types of lawns; Labeling the obstacles in the sample images to obtain first labeled images; Forming a training set according to the first labeled images; Constructing a convolutional neural network; Training the convolutional neural network by the training set to obtain a visual detection model.

4. The control method of the mowing robot according to claim 3, wherein The second detection method is an ultrasonic detection method, and the step of performing the second detection on the lawn mowing robot during operation by the second detection method when the first distance is greater than a preset distance threshold, and obtaining a second distance of the obstacle when the obstacle is detected comprises: acquiring an ultrasonic sensing result when the first distance is greater than a preset distance threshold; if the obstacle exists in the ultrasonic sensing result, acquiring a second distance of the obstacle. 5.The control method of the mowing robot of claim 1, wherein, After the step of, if the first distance is less than the preset distance threshold, performing obstacle avoidance control on the mowing robot based on the first distance, the method further comprises: performing obstacle labeling on the to-be-detected image in which the obstacle is detected to obtain a second labeled image; performing online training on the visual detection model through the second labeled image. 6.The control method of the mowing robot of claim 1, wherein, After the step of, if the first distance is greater than the preset distance threshold, performing obstacle avoidance control on the mowing robot based on the second distance, the method further comprises: determining a position of the obstacle according to the ultrasonic sensing result; determining a position of the obstacle in the to-be-detected image according to the position of the obstacle; performing labeling on the obstacle according to the position in the to-be-detected image to obtain a third labeled image; performing online training on the visual detection model through the third labeled image.

7. A control device of a mowing robot, characterized in that The control device of the mowing robot comprises: a first detection module configured to perform first detection on the mowing robot during operation through a first detection manner, and acquire a first distance of an obstacle if the obstacle is detected; a second detection module configured to perform second detection on the mowing robot during operation through a second detection manner when the first distance is greater than a preset distance threshold, and acquire a second distance of the obstacle if the obstacle is detected, the first detection manner and the second detection manner being different modal detection manners; a control module configured to perform obstacle avoidance control on the mowing robot based on the first distance or the second distance; wherein the control module is specifically configured to: perform obstacle avoidance control on the mowing robot based on the first distance if the first distance is less than the preset distance threshold; perform obstacle avoidance control on the mowing robot based on the second distance if the first distance is greater than the preset distance threshold. The distance threshold can be determined according to the detection accuracy of the first detection manner, and is a product of the detection accuracy and the maximum effective detection distance of the first detection manner. The real distance of the obstacle from the mowing robot is labeled in the form of a label in a sample image to form a fourth labeled image, and the fourth labeled image is taken as a training set to train a distance recognition model constructed based on a convolutional neural network; the distance recognition model is used to extract the first distance. The obstacle is labeled in the form of a label in a sample image, and the real distance of the obstacle from the mowing robot is labeled in the form of a label in the sample image to form a fifth labeled image, and the fifth labeled image is taken as a training set to train a visual detection model and a distance recognition model constructed based on a convolutional neural network; the visual detection model is used to detect whether an obstacle exists.

8. An electronic device, comprising: comprises: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the control method of the mowing robot according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the control method of the mowing robot according to any one of claims 1 to 6.

Images