Method and device for determining the direction of travel of a grass clearing device

Abstract

The embodiment of the present application provides a kind of grassland cleaning equipment's advancing direction determination method and device, wherein the method comprises: obtaining the target image that the camera of grassland cleaning equipment is shot;In target image, according to the distortion degree of the field angle image of camera, the first area included in target image is determined, wherein the first area is used to indicate the image area of image distortion degree greater than preset distortion threshold;Image recognition is carried out to the first area, and image recognition result is obtained;According to image recognition result, the advancing direction of grassland cleaning equipment is determined.Through the present application, the problem that the accuracy of determining the advancing direction of grassland cleaning equipment is low in the related art is solved, and the effect of improving the accuracy of determining the advancing direction of grassland cleaning equipment is achieved.

Description

[Technical Field]

[0001] The present invention relates to the field of communications, and more specifically, to a method and apparatus for determining the direction of travel of a lawn cleaning device. [Background Technology]

[0002] With the advancement of technology, various intelligent robot devices are becoming increasingly widespread and are gradually entering more and more fields. For example, intelligent robot devices can be used to clean lawns, enabling intelligent operation of lawn cleaning.

[0003] In existing technologies, lawn mowing equipment requires the installation of ultrasonic or millimeter-wave radar around the equipment to detect obstacles during the entire lawn mowing process. Simultaneously, a color sensor needs to be installed beneath the equipment to identify the grass. For example, the edges of the grass are first identified, and the direction of travel of the lawn mowing equipment is controlled based on the identification results, or other methods are used to control the direction of travel. It should be noted that the accuracy of determining the direction of travel of lawn mowing equipment in these related technologies is relatively low.

[0004] There is currently no effective solution to the problem of low accuracy in determining the direction of travel of lawn cleaning equipment in related technologies. [Summary of the Invention]

[0005] This invention provides a method and apparatus for determining the direction of travel of a lawn cleaning device, thereby at least solving the problem of low accuracy in determining the direction of travel of lawn cleaning devices in related technologies.

[0006] According to an embodiment of the present invention, a method for determining the travel direction of a lawn cleaning device is provided, comprising: acquiring a target image captured by a camera of the lawn cleaning device; determining, based on the distortion degree of the field of view image of the camera, that the target image includes a first region, wherein the first region is used to indicate an image region where the image distortion degree is greater than a preset distortion threshold; performing image recognition on the first region to obtain an image recognition result; and determining the travel direction of the lawn cleaning device based on the image recognition result.

[0007] In an exemplary embodiment, performing image recognition on the first region to obtain an image recognition result includes: dividing the first region into N sub-regions, where N is an integer greater than 1; determining the target sub-region where the target object included in the target image is located based on the image features of the N sub-regions; determining the target distance between the target object and the lawn cleaning device based on the coordinates of the target object in the target sub-region, wherein the target object is an object other than lawn, and the image recognition result includes the target distance.

[0008] In an exemplary embodiment, determining the target sub-region where the target object is located in the target image based on the image features of the N sub-regions includes: determining a first comparison result between the color feature value and a first preset feature value of each of the N sub-regions; determining M first sub-regions from the N sub-regions based on the determined first comparison results, wherein M is an integer greater than 0 and M is less than or equal to N, and the similarity between the color feature value of the first sub-region and the first preset feature value is greater than a preset similarity threshold; and determining L second sub-regions from the M first sub-regions if the M first sub-regions include a first sub-region in the forward direction of the lawn cleaning device, wherein L is an integer greater than 0 and L is less than or equal to M, and the target sub-region includes the L second sub-regions.

[0009] In an exemplary embodiment, when the M first sub-regions include a first sub-region along the forward direction of the lawn mowing device, determining L second sub-regions from the M first sub-regions includes: determining, within the M first sub-regions, a target first sub-region whose distance from the first sub-region along the forward direction of the lawn mowing device is less than or equal to a preset distance threshold, wherein the target first sub-region includes one or more first sub-regions; determining a second comparison result between the image position feature value of each target first sub-region and the image position feature value of the first sub-region along the forward direction of the lawn mowing device; and determining L second sub-regions from the target first sub-regions based on the determined plurality of second comparison results, wherein L is an integer greater than 0 and L is less than or equal to M, and the Euclidean distance between the image position feature value of the second sub-region and the image position feature value of the first sub-region along the forward direction of the lawn mowing device is less than a preset distance value.

[0010] In an exemplary embodiment, determining the target distance between the target object and the lawn cleaning device based on the coordinates of the target object in the target sub-region includes: performing the following operations for each second sub-region to determine multiple target distances: expanding the second sub-region to obtain a target region image; traversing the target region image to determine target pixels in the target region image whose color feature values ​​are greater than preset color feature values; and, if the target region image is determined to include a first target region, determining the target distance between the first target region and the camera based on the coordinates of the first target region, wherein the number of target pixels included in the first target region is less than a first preset number, and the number of target pixels included in a second target region adjacent to the first target region is greater than or equal to a second preset number, the target region image includes the second target region, and the target object includes the first target region.

[0011] In an exemplary embodiment, determining the direction of travel of the lawn mowing device based on the image recognition result includes: controlling the mowing device to change direction when the target object is located in the current direction of travel and the target distance between the target object and the lawn mowing device is less than or equal to a first preset distance; controlling the mowing device to maintain the forward direction when the target object is located in the current direction of travel and the target distance between the target object and the lawn mowing device is greater than the first preset distance; controlling the mowing device to change direction when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn mowing device is less than or equal to a second preset distance; and controlling the mowing device to maintain the forward direction when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn mowing device is greater than the second preset distance.

[0012] In one exemplary embodiment, after determining M first sub-regions from N sub-regions based on the determined plurality of first comparison results, the method further includes: if it is determined that the M first sub-regions include a first sub-region in the direction of travel of the lawn cleaning device, controlling the lawn cleaning device to maintain the direction of travel.

[0013] According to another embodiment of the present invention, a device for determining the travel direction of a lawn mowing device is provided, comprising: an acquisition module for acquiring a target image captured by a camera of the lawn mowing device; a first determination module for determining, based on the distortion degree of the field of view image of the camera, that the target image includes a first region, wherein the first region is used to indicate an image region where the image distortion degree is greater than a preset distortion threshold; an identification module for performing image identification on the first region to obtain an image identification result; and a second determination module for determining the travel direction of the lawn mowing device based on the image identification result.

[0014] According to yet another embodiment of the present invention, a lawn cleaning device is also provided, comprising the apparatus described above.

[0015] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0016] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0017] This invention, through the use of a lawn mowing device's camera, identifies a first region in the target image whose distortion exceeds a preset distortion threshold, based on the distortion degree of the camera's field of view. The direction of travel of the lawn mowing device is then determined based on this first region. By using the distorted portion of the image captured by the lawn mowing device, the direction of travel is determined solely through visual recognition of the grass edges and obstacles, without the need for radar or color sensors. This improves the efficiency of obstacle or grass edge recognition and addresses the problem of low accuracy in determining the direction of travel, thus enhancing the overall accuracy of determining the direction of travel. [Attached Image Description]

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a mobile terminal hardware structure block diagram of a method for determining the travel direction of a lawn cleaning device according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of a method for determining the travel direction of a lawn cleaning device according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of an optional image region division according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of an optional image sub-region division according to an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of an optional image sub-region numbering according to an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of an optional image sub-region expansion according to an embodiment of the present invention;

[0025] Figure 7 This is a schematic diagram illustrating an optional image sub-region determination according to an embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram of the camera field of view of an optional lawn mowing device according to an embodiment of the present invention;

[0027] Figure 9 This is a structural block diagram of a method and apparatus for determining the travel direction of a lawn cleaning device according to an embodiment of the present invention.

Detailed Implementation Methods

[0028] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] The method embodiments provided in this application can be executed in a mobile robot or a similar computing device. Taking running on a mobile robot as an example, Figure 1 This is a hardware structure block diagram of a method for determining the travel direction of a lawn cleaning device according to an embodiment of the present invention. Figure 1 As shown, a mobile robot may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. In one exemplary embodiment, the mobile robot may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile robot described above. For example, the mobile robot may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0031] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the method for determining the direction of travel of the lawn mowing equipment in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile robot via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile robot's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0033] First, the application scenarios of this invention will be explained:

[0034] During operation, lawn mowing equipment needs to determine its direction of travel. For example, obstacles and grass edges can be detected first to determine the distance between the obstacle or edge and the lawn mowing equipment. During the entire lawn mowing process, ultrasonic or millimeter-wave radar needs to be installed around the lawn mowing equipment to detect obstacles, and a color sensor needs to be installed below the equipment to identify grass edges. By identifying obstacles or grass edges, the direction of travel of the lawn mowing equipment is controlled. Using two separate sensors to detect obstacles and grass edges leads to high costs and complex detection methods. Furthermore, the accuracy of determining the direction of travel of lawn mowing equipment in related technologies is relatively low. To address this problem, this invention proposes a method for determining the direction of travel of lawn mowing equipment, which can ensure that the accuracy of determining the direction of travel of lawn mowing equipment is avoided. The lawn mowing equipment involved in this invention can be a lawnmower or other equipment with lawn mowing and autonomous driving capabilities.

[0035] The present invention will now be described with reference to the embodiments:

[0036] This embodiment provides a method for determining the direction of travel of a lawn cleaning device, such as... Figure 2 As shown, the method includes the following steps:

[0037] Step S202: Acquire the target image captured by the camera of the lawn cleaning equipment;

[0038] The lawn cleaning equipment can be a lawnmower or other equipment capable of cleaning lawns. It should also be noted that this invention can be applied to the cleaning of other crops, such as corn stalks, wheat stalks, or other crops. This invention is illustrated using lawn cleaning as an example. The type of lawn cleaning equipment is not limited here; the camera can be a fisheye lens or other lens with a field of view greater than 180 degrees.

[0039] Step S204: In the target image, a first region is determined based on the distortion degree of the field of view image of the camera, wherein the first region is used to indicate an image region where the image distortion degree is greater than a preset distortion threshold.

[0040] The camera can be a fisheye camera, and the target image captured by the fisheye camera can be divided into multiple regions according to the field of view. For example, the image region corresponding to a field of view greater than 200 degrees. The preset distortion threshold can be 0.8. Assuming that the image distortion value of the image region corresponding to a field of view of 200 degrees is 0.8, the image region corresponding to a field of view greater than 200 degrees is determined as the first region.

[0041] Step S206: Perform image recognition on the first region to obtain the image recognition result;

[0042] In this process, image features in the first region can be identified. Since the first region has large distortion and it is difficult to extract feature points from the surrounding area, the grass cleaning equipment can achieve pure visual obstacle avoidance or edge-following by identifying the position of obstacles or grass edges in the first region image.

[0043] Step S208: Determine the direction of travel of the lawn cleaning equipment based on the image recognition results.

[0044] The image recognition result can be the location of the identified obstacle or the edge of the grass, so as to determine the direction of travel of the grass cleaning equipment based on the identified obstacle avoidance location.

[0045] The entity performing the above operations can be a lawn cleaning device (e.g., an intelligent robot, or other devices with lawn cleaning and autonomous driving capabilities), a processor installed within the lawn cleaning device, or other devices with similar processing capabilities.

[0046] In an exemplary embodiment, the entity performing the above steps may be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least a data processing device. The data processing device may include, but is not limited to, terminals such as computers and mobile phones.

[0047] Through the above steps, in the target image captured by the camera of the lawn cleaning equipment, a first region with an image distortion degree greater than a preset distortion threshold is determined based on the distortion degree of the camera's field of view image. The direction of travel of the lawn cleaning equipment is then determined based on this first region. By using the image portion with a higher degree of distortion captured by the lawn cleaning equipment, without the need for radar or color sensors, the direction of travel of the lawn cleaning equipment is determined solely through visual recognition of the grass edges and obstacle information. This improves the efficiency of the lawn cleaning equipment in recognizing obstacles or grass edges, solving the problem of low accuracy in determining the direction of travel of the lawn cleaning equipment, and achieving the effect of improving the accuracy of determining the direction of travel of the lawn cleaning equipment.

[0048] As an optional implementation, the camera of the lawn cleaning device can be a fisheye lens, specifically a fisheye lens with a field of view greater than 180 degrees, such as a fisheye lens with a field of view of 230 degrees. Figure 3As shown, the captured target image can be divided into three regions based on the field of view of the fisheye lens. The innermost region is considered unusable, as it lacks effective feature points outdoors, and processing it would require significant computational resources. Secondly, the central annular region is the feature point extraction region, capable of extracting effective feature points from the surrounding area without significant distortion. Finally, the outermost region can serve as the primary area for grass monitoring, as it suffers from significant distortion, hindering effective feature point extraction from the surrounding area. Therefore, this region can be used for purely visual obstacle avoidance and edge recognition in intelligent lawnmowers.

[0049] In an exemplary embodiment, performing image recognition on the first region to obtain an image recognition result includes: dividing the first region into N sub-regions, where N is an integer greater than 1; determining the target sub-region where the target object included in the target image is located based on the image features of the N sub-regions; determining the target distance between the target object included in the target image and the lawn cleaning device based on the coordinates of the target object in the target sub-region, wherein the target object is an object other than lawn, and the image recognition result includes the target distance.

[0050] As an optional implementation, the first region can be an area in the image captured by the fisheye lens where the distortion level is greater than a preset distortion threshold. The lawn cleaning device can be a lawnmower, and the target object can be the boundary area between the obstacle and the grass, or the edge area of ​​the grass. The value of N can be determined according to the size of the lawnmower and the anti-collision expansion radius. Here, there is no restriction on the method of determining the value of N. Assuming that the value of N is 16, the first region can be divided into 16 sub-regions, such as... Figure 4 The diagram shows the division of the first region into 16 sub-regions. Assuming the target object to be identified is the boundary between the obstacle and the grass, after identifying the image features of each of the 16 sub-regions, the target sub-region where the target object is located can be obtained based on the identification results of each sub-region. Based on the coordinates of the target object in the target sub-region, the boundary between the obstacle and the grass, and the target distance between the boundary and the grass cleaning equipment can be determined. Assuming that an obstacle A and the grass are identified in the forward direction of the grass cleaning equipment, the target distance between obstacle A and the grass cleaning equipment can be determined based on the coordinates of the boundary in the target sub-region.

[0051] In an exemplary embodiment, determining the target sub-region where the target object is located in the target image based on the image features of the N sub-regions includes: determining a first comparison result between the color feature value and a first preset feature value of each of the N sub-regions; determining M first sub-regions from the N sub-regions based on the determined first comparison results, wherein M is an integer greater than 0 and M is less than or equal to N, and the similarity between the color feature value of the first sub-region and the first preset feature value is greater than a preset similarity threshold; and determining L second sub-regions from the M first sub-regions if the M first sub-regions do not include the sub-regions in the forward direction of the lawn cleaning device, wherein L is an integer greater than 0 and L is less than or equal to M, and the target sub-region includes the L second sub-regions.

[0052] As an optional implementation, assuming N is 16, the color feature value can be the HSV feature value of the image. The HSV color model can represent the color features of the image, where the HSV color parameters are hue (H), saturation (S), and lightness (V). The HSV feature value of each of the 16 sub-regions can be compared with a first preset feature value to obtain 16 first comparison results. The first preset feature value can be the HSV feature value of a grass image taken by the lawn cleaning device at the center of the grass, or it can be the HSV feature value of a grass image obtained by other shooting devices. The image source of the first preset feature value is not limited here. Based on the 16 first comparison results, M first sub-regions are determined. For example, 14 sub-regions with HSV feature values ​​greater than the first preset feature value can be selected, resulting in 14 first sub-regions, i.e., M equals 14. It should be noted that an HSV feature value greater than the first preset feature value indicates that the HSV color feature value of the region is highly similar to the grass, and can be considered a feasible region for the lawn cleaning device. Based on the image region with a certain proportion of grass, the location of the grass edge is further determined. Assuming that sub-region 1 is the current direction of travel of the lawn cleaning equipment, if sub-region 1 is not included in the current direction of travel of the lawn cleaning equipment among the 14 first sub-regions, the image position of sub-region 1 can be compared with the image positions of the 14 first sub-regions to obtain feasible first sub-regions within a preset range of distance from the direction of travel. This allows for the determination of the location of the boundary area between the obstacle and the lawn or the edge area of ​​the lawn, and the determination of L second sub-regions where the boundary area between the obstacle and the lawn or the edge area of ​​the lawn exists. These L second sub-regions can then be identified as target sub-regions.

[0053] In an exemplary embodiment, if the first sub-regions in the direction of the lawn mowing device's movement are not included in the M first sub-regions, determining L second sub-regions from the M first sub-regions includes: determining, within the M sub-regions, a target first sub-region whose distance from the sub-regions in the direction of the lawn mowing device's movement is less than or equal to a preset distance threshold, wherein the target first sub-region includes one or more first sub-regions; determining a second comparison result between the image position feature value of each target first sub-region and the image position feature value of the sub-regions in the direction of the lawn mowing device's movement, and determining L second sub-regions from the target first sub-regions based on the determined plurality of second comparison results, wherein L is an integer greater than 0, and L is less than or equal to M, and the Euclidean distance between the image position feature value of the second sub-region and the image position feature value of the sub-regions in the direction of the lawn mowing device's movement is less than a preset distance value.

[0054] As an alternative implementation, assuming M equals 14 and N equals 16, the 16 sub-regions can be numbered sequentially as Region 1 to Region 16, such as... Figure 5 As shown, the first sub-region of M can correspond to regions 1-14. Assuming the sub-region in the forward direction of the lawn cleaning equipment is region 15, and the preset distance is 10 cm, the first sub-regions with a distance less than 10 cm from the center point of region 15 can be identified as regions 1, 13, and 14. At this point, regions 1, 13, and 14 can be identified as the target first sub-region. The image position feature value can be the center coordinate point of the image. The image center points of regions 1, 13, and 14 are compared with the image center point of region 15 using Euclidean distance, resulting in three second comparison results. Assuming the comparison results are 2, 2, and 1 respectively, and the preset distance value is 2, the region with an Euclidean distance less than 2 from the image center point of region 15 is region 14. Region 14 is then identified as L second sub-regions, where L equals 1. Based on the obtained second sub-region region 14, the distance between the lawn cleaning equipment and the edge of the lawn can be determined.

[0055] Assuming the first sub-region of M corresponds to regions 1-14, and the sub-region in the forward direction of the lawn clearing equipment is region 15, with a preset distance of 10 cm, the first sub-regions with a distance of less than 20 cm from the center point of region 15 can be identified as regions 1, 2, 12, 13, and 14. It should be noted that if the proportion of the lawn image in regions 1 and 2 is greater than a certain threshold, regions 1 and 2 can be considered as a feasible region A. In this case, regions A, 12, 13, and 14 can be identified as the target first sub-region. Image location feature values ​​can be the center coordinates of the image. The center points of regions A, 12, 13, and 14 are compared with the center point of region 15 using Euclidean distance. Weights can be assigned to the four Euclidean distances. For example, smaller weights can be assigned to the Euclidean distances corresponding to larger feasible regions. It should be noted that when the feasible region is larger, it means that the grass area accounts for a larger proportion of the feasible region. When cleaning the grass, corner areas with a smaller proportion of grass area should be selected first to improve the accuracy of cleaning and achieve better cleaning results. It should be noted that a smaller Euclidean distance indicates a smaller turning angle for the lawn mower. Therefore, the first sub-region with the smallest Euclidean distance from region 15 should be selected as the optimal region. Conversely, a smaller area indicates a potential corner region, which should be prioritized. Therefore, the first sub-region with both a small Euclidean distance from the lawn mower's forward direction and a small area should be selected as the optimal region. Assuming the Euclidean distances between the image center points of regions A, 12, 13, and 14 and the image center point of region 15 are 2.5, 3, 2, and 1, respectively, smaller areas should be assigned smaller weights to obtain smaller Euclidean distances. The area weights can be 2, 1, 1, and 1. Thus, the four comparison results are 2.5*2, 3*1, 2*1, and 1*1, which are 5, 3, 2, and 1. Assuming a preset distance value of 2, the region with an Euclidean distance less than 2 from the image center point of region 15 is region 14. Region 14 is then determined as L second sub-regions, where L equals 1. Based on the obtained second sub-region 14, the distance between the lawn mower and the edge of the lawn or obstacles can be determined.

[0056] In an exemplary embodiment, determining the target distance between the target object and the lawn cleaning device based on the coordinates of the target object in the target sub-region includes: performing the following operations for each second sub-region to determine multiple target distances: expanding the second sub-region to obtain a target region image; traversing the target region image to determine target pixels in the target image whose color feature values ​​are greater than preset color feature values; and, if the target region image is determined to include a first target region, determining the target distance between the first target region and the camera based on the coordinates of the first target region, wherein the number of target pixels included in the first target region is less than a first preset number, and the number of target pixels included in a second target region adjacent to the first target region is greater than or equal to a second preset number, the target region image includes the second target region, and the target object includes the first target region.

[0057] As an optional implementation, assuming the determined L second sub-regions are region 1 and region 2, and both region 1 and region 2 are edge images captured by a fisheye lens, region 1 and region 2 can be expanded to obtain a standard image. Taking the expansion of region 1 as an example, ... Figure 6 As shown, the color feature value can be the HSV feature value of the image. The image in region 1 can be expanded to obtain a standard image, and the pixels in the obtained standard image can be traversed to find target pixels whose HSV feature values ​​are greater than a preset HSV feature value. The preset HSV feature value can be the HSV feature value representing the grass color in a pre-captured grass image; that is, finding the pixels in region 1 that can represent grass. For example, the color feature value can be... Figure 6 The pixel marked A is taken as the target pixel. The first target region can be a pixel row, a pixel column, or a pixel block of any shape composed of pixels. There is no restriction on the first target region here. Assuming the first target region is a pixel row, the pixel row containing fewer than a first preset number of target pixels is taken as the first target region. Assuming the first preset number is 1, the pixel row containing fewer than 1 target pixel A can be taken as the first target region. Figure 6As shown, assuming the second target region is also a single pixel row, and the second preset number is 3, then for the two adjacent pixel rows of the third pixel row, there is no pixel row containing the target pixel A with a number greater than or equal to 3. That is, the third pixel row is not the first target region. However, for the two adjacent pixel rows of the fourth pixel row, there is a fifth pixel row containing the target pixel A with a number greater than or equal to 3. At this time, the fourth pixel row can be identified as the first target region. After determining the location of the first target region, the actual target distance between the first target region and the camera can be obtained based on the coordinates of the first target region in the target region image. For example, a monocular ranging method or a binocular ranging method can be used. Here, the ranging method is not limited, and an appropriate ranging algorithm can be selected according to the actual situation.

[0058] In an exemplary embodiment, determining the direction of travel of the lawn mowing device based on the image recognition result includes: controlling the mowing device to change direction when the target object is located in the current direction of travel and the target distance between the target object and the lawn mowing device is less than or equal to a first preset distance; controlling the mowing device to maintain the forward direction when the target object is located in the current direction of travel and the target distance between the target object and the lawn mowing device is greater than the first preset distance; controlling the mowing device to change direction when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn mowing device is less than or equal to a second preset distance; and controlling the mowing device to maintain the forward direction when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn mowing device is greater than the second preset distance.

[0059] As an optional implementation, assuming the target object is the edge of a lawn, the first preset distance can be 20 cm, 30 cm, etc., and there is no limitation on the first preset distance. When the edge of the lawn is detected to be in the current direction of the lawn cleaning device, and the target distance between the edge of the lawn and the lawn cleaning device is less than or equal to 20 cm, the lawn cleaning device is controlled to change direction. For example, it can turn 90 degrees to the right or 60 degrees to the left. Here, there is no limitation on the direction of the lawn cleaning device's change, and it can be determined according to the size of the lawn cleaning device or the length of the lawn cleaning components.

[0060] If the edge of the grass is detected to be in the direction the grass cleaning equipment is currently moving, and the target distance between the edge of the grass and the grass cleaning equipment is greater than 20 centimeters, the grass cleaning equipment is controlled to maintain its forward direction. At this time, there is a certain safe distance between the grass cleaning equipment and the edge of the grass, so it can continue to maintain its forward direction.

[0061] If the edge of the grass is detected to be on the side of the grass cleaning device's current direction of travel, and the target distance between the edge of the grass and the grass cleaning device is less than or equal to 20 centimeters, the grass cleaning device is controlled to change direction. For example, it can turn 90 degrees to the right or 60 degrees to the left. There is no restriction on the direction of grass cleaning device change. The side of the current direction of travel can be 30 degrees to the right or 20 degrees to the left. There is no restriction on the side of the current direction of travel.

[0062] If the edge of the grass is detected to be on the side of the grass cleaning equipment in the direction it is currently moving, and the target distance between the edge of the grass and the grass cleaning equipment is greater than 20 centimeters, the grass cleaning equipment is controlled to maintain the forward direction. At this time, there is a certain safe distance between the grass cleaning equipment and the edge of the grass, so it can continue to maintain the forward direction.

[0063] In one exemplary embodiment, after determining M first sub-regions from N sub-regions based on the determined plurality of first comparison results, the method further includes: if it is determined that the M first sub-regions include a first sub-region in the direction of travel of the lawn cleaning device, controlling the lawn cleaning device to maintain the direction of travel.

[0064] As an alternative implementation, it is assumed that the determined M first sub-regions can correspond to Figure 7 Regions 2-16 are defined, where M first sub-regions represent the areas where the lawn mower can travel. Assume the first sub-region in the direction the lawn mower travels is... Figure 7 If we consider region 16, then we can determine that region 2-region 16 includes region 16. In this case, if... Figure 7 As shown, region 16 in the target image represents the area in the forward direction of the lawn cleaning equipment. Since the forward direction is a feasible area, the lawn cleaning equipment can be controlled to maintain its forward movement.

[0065] As an optional implementation, for grassland monitoring areas, this invention provides an algorithm for rapid scene identification. This method can effectively segment obstacles and boundaries containing color differences. Figure 8As can be seen, a certain range of visible information can be obtained within the grass monitoring area. The blind spot of the fisheye lens can be calculated using the formula L = tanθ * H, where H is the height of the grass clearing device, L is the radius of the blind spot, and θ is the angle corresponding to the blind spot. To process the grass feature monitoring area while minimizing computational resources, this invention specifically designs a fast edge-avoidance search strategy. The strategy includes: 1. Quickly traversing to find absolutely infeasible areas; 2. Obtaining the optimal area based on Euclidean distance and expanding this area to approximate obstacles and boundaries as closely as possible.

[0066] Step 1: Quickly traverse to find absolutely infeasible regions: such as Figure 4 The grassland monitoring area shown is a ring, which can be divided into N fine-grained regions. The similarity of each region is measured by similarity (HSV feature), and regions containing a certain amount of grassland are fitted out from the N regions (this part requires the initial area where the robot is located to be grassland, so that the machine can effectively detect inconsistent regions).

[0067] Step 2: Obtain the optimal region based on Euclidean distance, and expand the region to get as close as possible to obstacles and boundaries.

[0068] After step one, a portion of the annular region has been eliminated. This portion may contain areas in the robot's forward direction. The selection strategy at this point is to iterate through the area to find the nearest un-eliminated region. Then, as follows... Figure 6 As shown, the region is expanded and then traversed horizontally to find the edge of the grassland, and noisy regions are filtered out using median filtering. Figure 6 Assuming A is the HSV pixel of the grass, then the fourth row represents the edge region of the grass. The distance to the edge region of the grass is then roughly estimated based on the image information.

[0069] As an optional implementation, the overall obstacle or edge detection method of the lawn cleaning equipment is as follows: Step 1: While performing V-SLAM, use a fisheye camera to detect lawn information. Step 2: Divide the lawn monitoring area into coarse and fine divisions according to preset values. These preset values ​​are not fixed and can be dynamically adjusted to meet actual user needs. This ensures the robot operates within the preset boundaries while dynamically adjusting its sensitivity to different colored areas. Step 3: Adjust the lawn cleaning equipment's posture based on the identified obstacle or lawn edge positions, plan the cutting route, and proceed to the charging station area after the lawn cleaning equipment completes cleaning or the battery level falls below a preset threshold.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0071] This embodiment also provides a control device for rotating the side brush, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0072] Figure 9 This is a structural block diagram of a device for determining the travel direction of a lawn mowing equipment according to an embodiment of the present invention, as shown in the diagram. Figure 9 As shown, the device includes:

[0073] Acquisition module 92 is used to acquire target images captured by the camera of the lawn cleaning equipment;

[0074] The first determining module 94 is used to determine, based on the degree of distortion of the field of view image of the camera, that the target image includes a first region, wherein the first region is used to indicate an image region whose image distortion degree is greater than a preset distortion threshold.

[0075] The recognition module 96 is used to perform image recognition on the first region and obtain the image recognition result;

[0076] The second determining module 98 is used to determine the direction of travel of the lawn cleaning equipment based on the image recognition result.

[0077] In an optional embodiment, the recognition module 96 can perform image recognition on the first region and obtain image recognition results by: dividing the first region into N sub-regions, where N is an integer greater than 1; determining the target sub-region where the target object included in the target image is located based on the image features of the N sub-regions; determining the target distance between the target object included in the target image and the lawn cleaning device based on the coordinates of the target object in the target sub-region, wherein the target object is an object other than the lawn, and the image recognition result includes the target distance.

[0078] In an optional embodiment, the identification module 96 can determine the target sub-region where the target object is located in the target image based on the image features of the N sub-regions in the following manner: determine the first comparison result of the color feature value and the first preset feature value of each of the N sub-regions; determine M first sub-regions from the N sub-regions based on the determined first comparison results, where M is an integer greater than 0 and M is less than or equal to N, and the similarity between the color feature value of the first sub-region and the first preset feature value is greater than a preset similarity threshold; if the M first sub-regions do not include the sub-regions in the forward direction of the lawn cleaning device, determine L second sub-regions from the M first sub-regions, where L is an integer greater than 0 and L is less than or equal to M, and the target sub-region includes L second sub-regions.

[0079] In an optional embodiment, the identification module 96 can determine L second sub-regions from the M first sub-regions when the sub-regions in the direction of the lawn mowing device's movement are not included in the M first sub-regions in the following manner: In the M first sub-regions, a target first sub-region is determined whose distance from the sub-regions in the direction of the lawn mowing device's movement is less than or equal to a preset distance threshold, wherein the target first sub-region includes one or more first sub-regions; a second comparison result is determined between the image position feature value of each target first sub-region and the image position feature value of the sub-regions in the direction of the lawn mowing device's movement; based on the determined second comparison results, L second sub-regions are determined from the target first sub-regions, where L is an integer greater than 0, and L is less than or equal to M, and the Euclidean distance between the image position feature value of the second sub-region and the image position feature value of the sub-regions in the direction of the lawn mowing device's movement is less than a preset distance value.

[0080] In an optional embodiment, the identification module 96 can achieve the above-mentioned determination of the target distance between the target object and the lawn cleaning device based on the coordinates of the target object in the target sub-region in the following manner: For each second sub-region, the following operations are performed to determine multiple target distances: the second sub-region is expanded to obtain a target region image; the target region image is traversed to determine target pixels in the target region image whose color feature values ​​are greater than preset color feature values; if the target region image is determined to include a first target region, the target distance between the first target region and the camera is determined based on the coordinates of the first target region, wherein the number of target pixels included in the first target region is less than a first preset number, and the number of target pixels included in the second target region adjacent to the first target region is greater than or equal to a second preset number, the target region image includes the second target region, and the target object includes the first target region.

[0081] In an optional embodiment, the second determining module 98 can determine the direction of travel of the lawn cleaning device based on the image recognition result in the following ways: when the target object is located in the current direction of travel and the target distance between the target object and the lawn cleaning device is less than or equal to a first preset distance, control the lawn cleaning device to change direction; when the target object is located in the current direction of travel and the target distance between the target object and the lawn cleaning device is greater than the first preset distance, control the lawn cleaning device to maintain the forward direction; when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn cleaning device is less than or equal to a second preset distance, control the lawn cleaning device to change direction; when the target object is located to the side of the current direction of travel and the target distance between the target object and the lawn cleaning device is greater than the second preset distance, control the lawn cleaning device to maintain the forward direction.

[0082] In an optional embodiment, the device for determining the direction of travel of the lawn cleaning equipment is further configured to, after determining M first sub-regions from the N sub-regions based on the determined plurality of first comparison results, control the lawn cleaning equipment to maintain the forward direction of travel if it is determined that the M first sub-regions include the first sub-regions in the direction of travel of the lawn cleaning equipment.

[0083] This embodiment also provides a lawn cleaning device, which may include a device for determining the direction of travel of the lawn cleaning device described in any of the above embodiments.

[0084] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0085] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0086] In this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0087] S1, acquire the target image captured by the camera of the lawn cleaning equipment;

[0088] S2, in the target image, a first region is determined based on the distortion degree of the field of view image of the camera, wherein the first region is used to indicate an image region where the image distortion degree is greater than a preset distortion threshold;

[0089] S3, perform image recognition on the first region to obtain the image recognition result;

[0090] S4. Based on the image recognition results, determine the direction of travel of the lawn cleaning equipment.

[0091] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0092] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0093] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0094] In one exemplary embodiment, the processor described above may be configured to perform the following steps via a computer program:

[0095] S1, acquire the target image captured by the camera of the lawn cleaning equipment;

[0096] S2, in the target image, a first region is determined based on the distortion degree of the field of view image of the camera, wherein the first region is used to indicate an image region where the image distortion degree is greater than a preset distortion threshold;

[0097] S3, perform image recognition on the first region to obtain the image recognition result;

[0098] S4. Based on the image recognition results, determine the direction of travel of the lawn cleaning equipment.

[0099] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining the direction of travel of a lawn cleaning device, characterized in that, include: Acquire target images captured by the camera of the lawn mowing equipment; In the target image, a first region is determined based on the degree of distortion of the field of view image of the camera, wherein the first region is used to indicate an image region where the degree of image distortion is greater than a preset distortion threshold; Image recognition is performed on the first region to obtain the image recognition result; Based on the image recognition results, the direction of travel of the lawn cleaning equipment is determined; The image recognition process for the first region yields the following results: The first region is divided into N sub-regions, where N is an integer greater than 1; Based on the image features of the N sub-regions, determine the target sub-region where the target object is located in the target image; The target distance between the target object and the lawn cleaning device is determined based on the coordinates of the target object in the target sub-region, wherein the target object is an object other than lawn, and the image recognition result includes the target distance.

2. The method for determining the direction of travel of the lawn cleaning equipment according to claim 1, characterized in that, The step of determining the target sub-region containing the target object in the target image based on the image features of the N sub-regions includes: The color feature value and the first preset feature value of each of the N sub-regions are determined respectively. Based on the determined first comparison results, M first sub-regions are determined from the N sub-regions, where M is an integer greater than 0 and M is less than or equal to N. The similarity between the color feature value of the first sub-region and the first preset feature value is greater than a preset similarity threshold. If the first sub-regions do not include the sub-regions in the forward direction of the lawn cleaning equipment, L second sub-regions are determined from the M first sub-regions, where L is an integer greater than 0 and L is less than or equal to M, and the target sub-regions include the L second sub-regions.

3. The method for determining the direction of travel of the lawn cleaning equipment according to claim 2, characterized in that, If the M first sub-regions do not include the sub-regions in the forward direction of the lawn clearing equipment, then L second sub-regions are determined from the M first sub-regions, including: In the M first sub-regions, a target first sub-region is determined whose distance from the sub-region in the forward direction of the lawn cleaning device is less than or equal to a preset distance threshold. The target first sub-region includes one or more first sub-regions. The image position feature value of each of the target first sub-regions is determined by a second comparison result with the image position feature value of the sub-region in the forward direction of the lawn cleaning device. Based on the determined second comparison results, L second sub-regions are determined from the target first sub-region, where L is an integer greater than 0 and L is less than or equal to M. The Euclidean distance between the image position feature value of the second sub-region and the image position feature value of the sub-region in the forward direction of the lawn cleaning device is less than a preset distance value.

4. The method for determining the direction of travel of the lawn cleaning equipment according to claim 3, characterized in that, Determining the target distance between the target object and the lawn cleaning equipment based on the coordinates of the target object in the target sub-region includes: For each of the second sub-regions, the following operations are performed to determine the multiple target distances: Expand the second sub-region to obtain the target region image; The target region image is traversed to determine the target pixels in the target region image whose color feature values ​​are greater than a preset color feature value. If the target area image is determined to include a first target area, the target distance between the first target area and the camera is determined based on the coordinates of the first target area. The number of target pixels included in the first target area is less than a first preset number, and the number of target pixels included in the second target area adjacent to the first target area is greater than or equal to a second preset number. The target area image includes the second target area, and the target object includes the first target area.

5. The method for determining the direction of travel of the lawn cleaning equipment according to claim 1, characterized in that, Determining the direction of travel of the lawn cleaning equipment based on the image recognition result includes: If the target object is located in the current direction of travel, and the target distance between the target object and the lawn mowing device is less than or equal to a first preset distance, the lawn mowing device is controlled to change direction; and / or, If the target object is located in the current forward direction and the target distance between the target object and the lawn mowing device is greater than a first preset distance, the lawn mowing device is controlled to maintain the forward direction; and / or, If the target object is located to the side of the current direction of travel, and the target distance between the target object and the lawn mowing device is less than or equal to a second preset distance, the lawn mowing device is controlled to change direction; and / or, When the target object is located to the side of the current direction of travel, and the target distance between the target object and the lawn cleaning device is greater than a second preset distance, the lawn cleaning device is controlled to maintain the forward direction of travel.

6. The method for determining the direction of travel of the lawn cleaning equipment according to claim 2, characterized in that, After determining M first sub-regions from the N sub-regions based on the determined plurality of first comparison results, the method further includes: If it is determined that the first sub-regions in the M first sub-regions include the first sub-regions in the direction of the lawn cleaning equipment's forward movement, the lawn cleaning equipment is controlled to maintain the forward movement direction.

7. A device for determining the direction of travel of a lawn cleaning equipment, characterized in that, include: The acquisition module is used to acquire target images captured by the camera of the lawn cleaning equipment; The first determining module is used to determine a first region in the target image based on the degree of distortion of the field of view image of the camera, wherein the first region is used to indicate an image region where the degree of image distortion is greater than a preset distortion threshold. The recognition module is used to perform image recognition on the first region and obtain the image recognition result; The second determining module is used to determine the direction of travel of the lawn cleaning equipment based on the image recognition results; The recognition module is further configured to perform image recognition on the first region in the following manner to obtain an image recognition result: dividing the first region into N sub-regions, where N is an integer greater than 1; determining the target sub-region where the target object is located in the target image based on the image features of the N sub-regions; determining the target distance between the target object and the lawn cleaning device based on the coordinates of the target object in the target sub-region, wherein the target object is an object other than lawn, and the image recognition result includes the target distance.

8. A lawn cleaning device, characterized in that, The device includes a means for determining the direction of travel of the lawn cleaning equipment as described in claim 7.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the program can be executed by a terminal device or a computer to determine the direction of travel of the lawn cleaning equipment as described in any one of claims 1 to 6.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform a method for determining the direction of travel of the lawn cleaning equipment as described in any one of claims 1 to 6.

Images