Method and device for providing information related to lawn management by using sensing information collected by robot

Abstract

Disclosed are a method and a device, the method comprising the steps of: determining, on the basis of map information, a search area to undergo lawn management; determining, on the basis of the search area and the map information, a movement path for a mobile robot to move within the search area; acquiring sensing information that includes vision information acquired from a vision sensor included in the mobile robot moving on the basis of the movement path, and non-vision information acquired from a sensor other than the vision sensor; acquiring lawn state information about the search area by processing the vision information and the non-vision information in different network models; and providing lawn management solution information for lawn management about the search area on the basis of the lawn state information.

Description

Method and device for providing information related to lawn care using sensing information collected by a robot

[0001] The technical field of the present disclosure relates to a method and device for providing information related to lawn care, and more specifically, to a technical field that enables determination of lawn care solution information optimized for lawn care based on various sensing information.

[0002] The use of lawn robots for lawn care at golf courses, soccer fields, and parks is on the rise. These robots deploy various sensors to collect a variety of lawn information and provide it to users, who then analyze this information to make informed decisions about lawn care.

[0003] However, because the information collected by lawn robots is so vast, it takes a long time for users to analyze it and make decisions about lawn management. Therefore, the need for technology that analyzes the information collected by lawn robots using artificial intelligence to manage lawns is increasing.

[0004] The problem to be solved in this disclosure is to disclose a method and device for controlling the driving of a lawn care robot, and to provide information on a method for determining a driving path that enables the lawn care robot to manage lawn more efficiently.

[0005] As a technical means for achieving the above-described technical task, a method for providing information related to lawn care using a robot by a device according to the first aspect of the present disclosure may include the steps of: determining a search area to be a target of lawn care based on map information; determining a movement path for a mobile robot to move within the search area based on the search area and the map information; obtaining sensing information including vision information obtained from a vision sensor included in the mobile robot moving based on the movement path and non-vision information obtained from a sensor other than the vision sensor; obtaining lawn condition information for the search area by processing the vision information and the non-vision information in different network models; and providing lawn care solution information for lawn care for the search area based on the lawn condition information.

[0006] In addition, the step of obtaining the grass condition information may include a step of obtaining grass diagnosis information based on a result of processing the vision information using a first network model; and a step of obtaining the grass condition information by applying the grass diagnosis information and the non-vision information to a second network model.

[0007] Additionally, the lawn care solution information may include lawn care know-how information and task command information for the mobile robot to perform a task corresponding to the lawn condition information of the search area.

[0008] In addition, the step of obtaining the grass condition information further includes a step of obtaining sunlight information based on a result of processing the vision information using a 3D map processing model, and the step of obtaining the grass condition information by applying the grass diagnosis information and the non-vision information to the second network model may obtain the grass condition information by applying the grass diagnosis information, the non-vision information, and the sunlight information to the second network model.

[0009] In addition, the method may further include a step of updating the first network model and the second network model based on the grass state information.

[0010] In addition, the method further includes a step of obtaining obstacle information on the movement path based on the sensing information; and a step of updating the movement path based on the obstacle information; wherein the obstacle information may include obstacle location information on the movement path, slope information on the movement path, and path status information on the movement path.

[0011] In addition, the method may further include a step of visualizing and displaying the movement path; a step of activating a confirmation request button for the displayed movement path; and a step of providing a movement command for the mobile robot based on an approval input for the confirmation request button.

[0012] In addition, the method further includes a step of obtaining a 3D map representing the search area as at least one voxel using the 3D map processing model, and the sunlight amount information can be determined based on GPS coordinate information and sun position information corresponding to each vertex of the voxel.

[0013] Additionally, the above work order information may include at least one of weeding work, fertilizer spreading work, pest control work, watering work, divot repair work, and sowing work.

[0014] A device for providing information related to lawn care using a robot according to a second aspect of the present disclosure may include a processor for determining a search area to be a target of lawn care based on map information, determining a movement path for a mobile robot to move within the search area based on the search area and the map information, obtaining sensing information including vision information obtained from a vision sensor included in the mobile robot moving based on the movement path and non-vision information obtained from a sensor other than the vision sensor, processing the vision information and the non-vision information in different network models to obtain lawn condition information for the search area, and providing lawn care solution information for lawn care for the search area based on the lawn condition information.

[0015] According to one embodiment of the present disclosure, efficient lawn management is possible because sensing information acquired by a mobile robot is processed in different network models to provide lawn management solution information.

[0016] In addition, according to another embodiment of the present disclosure, the convenience of the manager can be improved because the mobile robot is controlled by acquiring lawn condition information based on the sensing information acquired by the mobile robot and generating corresponding lawn management know-how information as well as work command information for the mobile robot.

[0017] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0018] Figure 1 is a block diagram schematically illustrating the configuration of a device according to one embodiment.

[0019] FIG. 2 is a flowchart illustrating a method for a device to provide information related to lawn care using a robot according to one embodiment.

[0020] FIG. 3 is a diagram illustrating an example of a device providing lawn care solution information using different network models according to one embodiment.

[0021] FIG. 4 is a diagram showing a screen of a lawn care system according to one embodiment.

[0022] The advantages and features of the present disclosure, and the methods for achieving them, will become clearer with reference to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure.

[0023] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present disclosure. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and "and / or" includes each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present disclosure.

[0024] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art. Furthermore, terms defined in commonly used dictionaries should not be interpreted ideally or excessively unless explicitly defined otherwise.

[0025] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" can be used to easily describe the relationship between one component and other components as depicted in the drawings. Spatially relative terms should be understood to include different orientations of the components during use or operation in addition to the orientations depicted in the drawings. For example, if a component depicted in the drawings were flipped over, a component described as "below" or "beneath" another component could end up "above" the other component. Thus, the exemplary term "below" can include both the above and below orientations. Components can also be oriented in other directions, and thus spatially relative terms can be interpreted accordingly.

[0026] Below, various embodiments are described in detail with reference to the drawings.

[0027]

[0028] Figure 1 is a block diagram schematically illustrating the configuration of a device (100) according to one embodiment.

[0029] Referring to FIG. 1, the device (100) may include a memory (110) and a processor (120).

[0030] According to an embodiment, a processor (120) may determine a search area to be a target of lawn care based on map information. In addition, the processor (120) may determine a movement path for a mobile robot to move within the search area based on the search area and map information. In addition, the processor (120) may obtain sensing information including vision information obtained from a vision sensor included in the moving mobile robot based on the movement path and non-vision information obtained from a sensor other than the vision sensor. In addition, the processor (120) may process the vision information and non-vision information in different network models to obtain lawn condition information for the search area. In addition, it should be noted that the processor (120) may utilize various combinations of conventional networks, such as the Internet or a mobile communication network, in the process of providing lawn care solution information for lawn care for the search area based on the lawn condition information, and there is no particular limitation thereto.

[0031] In addition, it will be understood by those skilled in the art that, in addition to the components illustrated in FIG. 1, other general components may be further included in the device (100). For example, the device (100) may include a drive control module, a path detection module, a data processing module, etc. for performing each step. In addition, the device (100) may control a mobile robot through a control unit, and may include a receiving unit (not illustrated) for receiving a plurality of pieces of information, or a transmitting unit (not illustrated) for transmitting lawn care solution information. In addition, the device (100) may be one module among the devices included in the mobile robot, or may be a device or server for controlling the mobile robot from the outside. Alternatively, it will be understood by those skilled in the art that, according to another embodiment, some of the components illustrated in FIG. 1 may be omitted.

[0032] A device (100) according to one embodiment and a method for providing information related to lawn care can be utilized by a user and can be linked with all kinds of handheld-based wireless communication devices equipped with a touch screen panel, such as a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), a tablet PC, etc. In addition, it can be included in or linked with a device that has a foundation for installing and executing applications, such as a desktop PC, a tablet PC, a laptop PC, an IPTV including a set-top box.

[0033] Additionally, the device (100) can be implemented as a terminal such as a computer that operates through a computer program to realize the functions described in this specification.

[0034] A device (100) according to one embodiment may include, but is not limited to, a system (not shown) for controlling a mobile robot and a related server (not shown). The server according to one embodiment may support an application that provides information related to lawn care.

[0035] Hereinafter, the present invention will be described focusing on an embodiment in which a device (100) according to one embodiment provides information related to lawn care. However, as previously described, this may also be performed through linkage with a server. In other words, the device (100) and the server providing information related to lawn care according to one embodiment may be implemented in an integrated manner in terms of their functions, and the server may be omitted, and it can be understood that the present invention is not limited to any one embodiment.

[0036] In one embodiment, the device (100) and a server providing information related to lawn care may be linked, and the configuration of the system providing information related to lawn care may be performed by the server or may be automatically performed by the device (100). For example, the device (100) may operate as a server, and will be described below unified as the device (100).

[0037]

[0038] FIG. 2 is a flowchart illustrating a method in which a device (100) according to one embodiment provides information related to lawn care using a robot.

[0039] Referring to step S210, the device (100) according to one embodiment can determine a search area to be subjected to lawn care based on map information. In one embodiment, the map information may refer to a satellite map of a golf course or soccer field, and the search area may refer to an area requiring lawn care on the satellite map. Specifically, the search area can be determined in the form of a boundary by an administrator drawing on the map area (420) of the lawn care system described below or through touch input.

[0040] Referring to step S220, the device (100) according to one embodiment may determine a movement path for the mobile robot to move within the search area based on the search area and map information. Specifically, the movement path may refer to a path along which the mobile robot travels within the search area to collect information related to lawn or perform lawn care. In addition, the movement path may refer to a shortest path for performing lawn care or a path that moves throughout the search area, and a path planning algorithm for autonomous driving, such as Dijkstra's algorithm, A star algorithm, or RRT algorithm, may be used based on map information and information about the search area to determine the movement path.

[0041]

[0042] Referring to step S230, the device (100) according to one embodiment can obtain sensing information including vision information obtained from a vision sensor included in a mobile robot moving based on a movement path and non-vision information obtained from a sensor other than the vision sensor. Specifically, the mobile robot according to one embodiment can obtain sensing information on a search area while driving along a movement path by having vision sensors such as an RGB camera, a lidar sensor, an ultrasonic sensor, and non-vision sensors such as a temperature sensor, a humidity sensor, and a GPS sensor arranged together.

[0043] For example, a device (100) according to one embodiment can obtain a grass image for determining pests and diseases in the grass through an RGB camera among vision sensors placed on a mobile robot, and a lidar sensor can obtain point cloud data of a search area for generating a three-dimensional map.

[0044] In addition, the device (100) according to one embodiment can calculate the amount of moisture contained in the grass through a humidity sensor among non-vision sensors placed on a mobile robot, for example, and based on this, the manager can decide whether to water the grass.

[0045] Additionally, the device (100) according to one embodiment can obtain obstacle information on a moving path based on sensing information and update the moving path based on the obstacle information. Specifically, the obstacle information can include obstacle location information on the moving path, slope information on the moving path, and path status information on the moving path.

[0046]

[0047] Referring to step S240, the device (100) according to one embodiment may acquire grass condition information for a search area by processing vision information and non-vision information in different network models. Specifically, the network model may refer to an artificial intelligence learning model, and the network model may include a first network model that processes vision information and a second network model that processes grass diagnosis information and non-vision information acquired from the first network model.

[0048] For example, a device (100) according to one embodiment may obtain grass diagnosis information based on the result of processing vision information applied to a first network model. Specifically, the grass diagnosis information may include data on grass color, grass shape, grass length, grass density, etc., obtained by analyzing grass image information obtained through a vision sensor through the first network model. In addition, the grass diagnosis information may be obtained in the form of an image or a JSON (JavaScript Object Notation) file.

[0049] Additionally, the device (100) according to one embodiment can obtain lawn condition information by applying lawn diagnostic information and non-vision information acquired through the first network model to the second network model. Specifically, the lawn condition information may include data on whether the lawn is infected with pests or diseases, the type of pests or diseases, the growth status of the lawn, the extent of lawn damage, etc.

[0050] For example, a device (100) according to an embodiment can obtain grass diagnosis information indicating that there is an irregular color area of ​​reddish brown / yellow / brown in the grass by applying a grass image acquired through an RGB camera among vision sensors to a first network model. In addition, non-vision information indicating that the relative humidity is 80 to 90% can be obtained through a humidity sensor among non-vision sensors. Thereafter, the device (100) according to an embodiment can obtain grass condition information indicating that the grass is infected with anthrax by applying the grass diagnosis information and non-vision information to a second network model.

[0051] In addition, the device (100) according to one embodiment can further obtain sunlight information based on the result of processing vision information collected from a mobile robot using a 3D map processing model to obtain lawn condition information. Thereafter, the device (100) according to one embodiment can obtain lawn condition information by applying lawn diagnosis information, non-vision information, and sunlight information to a second network model. This is because lawn with insufficient sunlight may have a slow growth rate due to a decrease in photosynthetic efficiency and may be infected with pests such as yellow wilt. In this way, by determining the lawn condition by including sunlight information, lawn condition information can be obtained more accurately and efficiently.

[0052] In addition, the device (100) according to one embodiment can further obtain a 3D map represented by at least one voxel using a 3D map processing model to obtain sunlight information, and can determine sunlight information based on GPS coordinate information and sun position information corresponding to each vertex of the voxel.

[0053] Below, an example of the process for determining sunlight information is described in detail.

[0054] For example, a device (100) according to one embodiment may scan a search area using a lidar sensor placed on a mobile robot to generate 3D point cloud data, and apply the 3D point cloud data to a SLAM (Simultaneous Localization And Mapping) algorithm to generate a 3D map of the search area. Thereafter, the 3D map may be voxelized into at least one rectangular parallelepiped.

[0055] Hereinafter, the device (100) according to one embodiment can calibrate the X-coordinate value, Y-coordinate value, and Z-coordinate value of each vertex of the voxel to GPS coordinates consisting of latitude, longitude, and altitude values. Specifically, the upper left vertex of the voxel is ( ), the upper right corner is ( ), the lower left corner is ( ), the lower right corner is ( ) can be calibrated.

[0056] Afterwards, the device (100) according to one embodiment calculates the elevation angle α and azimuth angle γ of the sun, and calculates the amount of sunlight at each vertex of the voxel using the mathematical formula below.

[0057]

[0058] is the solar constant, which is approximately 1361 W / m2, is the atmospheric transmittance, which is calculated as follows depending on the altitude:

[0059]

[0060] A device (100) according to one embodiment can determine sunlight information by calculating in this way, and can obtain lawn condition information by applying the determined sunlight information, lawn diagnosis information, and non-vision information to a second network model.

[0061] Additionally, the device (100) according to one embodiment can update the first network model and the second network model based on the acquired lawn condition information. This can improve reliability by reducing errors in the lawn condition information output by the network model.

[0062]

[0063] Referring to step 250, the device (100) according to one embodiment may provide lawn care solution information for lawn care in a search area based on lawn condition information. Specifically, the lawn care solution information may include lawn care know-how information and task command information for a mobile robot to perform a task corresponding to the lawn condition information in the search area. Specifically, the task command information may include at least one of a weeding operation, a fertilizer application operation, a pest control operation, a watering operation, a divot repair operation, and a seeding operation.

[0064] Additionally, the device (100) according to one embodiment can obtain lawn management solution information along with lawn condition information by applying lawn diagnostic information and non-vision information to a second network model. The second network model for this purpose may be a generative artificial intelligence model such as a Large Language Model (LLM).

[0065] For example, if the grass condition information acquired from the second network model is that the grass on the golf course is in a divot state, the second network model according to an embodiment can provide a grass restoration method as a task corresponding to the divot state as grass management know-how information along with the grass condition information. In addition, the second network model according to an embodiment can provide specific execution steps for the grass restoration method, such as ① removing damaged grass, ② preparing soil, ③ sowing new grass seeds, ④ watering, and ⑤ protecting from external stimuli. In addition, the second network model according to an embodiment can determine task command information corresponding to each of the specific execution steps of the grass restoration method, ① removing damaged grass, ② preparing soil, ③ sowing new grass seeds, ④ watering, and ⑤ protecting from external stimuli, and transmit the determined task command information to the mobile robot. The mobile robot can receive the corresponding task command information and autonomously perform the grass restoration task.

[0066]

[0067] FIG. 3 is a drawing for explaining an example of a device (100) providing lawn care solution information using a different network model according to one embodiment, and is the same as the above, so a detailed description is omitted.

[0068]

[0069] FIG. 4 is a drawing showing a screen of a lawn management system according to one embodiment.

[0070]

[0071] A device (100) according to one embodiment may be implemented in an ERP-type lawn care system (400). Referring to FIG. 4, the screen of the lawn care system (400) may be composed of a confirmation request button (410), a map area (420), and a lawn monitoring area (430).

[0072]

[0073] That is, the device (100) according to one embodiment implemented in the ERP-type lawn management system (400) can visualize and display the movement path of the mobile robot, activate a confirmation request button (410) for the displayed movement path, and provide a movement command for the mobile robot based on an approval input for the confirmation request button (410).

[0074]

[0075] Various embodiments of the present disclosure may be implemented as software comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine (e.g., a display device or a computer). For example, a processor (120) of the machine (e.g., processor (120)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0076] According to one embodiment, the method according to the various embodiments disclosed in the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0077] Although the present invention has been described with reference to the drawings, it is not limited to the disclosed embodiments and drawings, and a person skilled in the art related to the present embodiment will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the above-described description. Therefore, the disclosed methods should be considered from an illustrative rather than a restrictive point of view. Even if the operation and effect according to the configuration of the present invention is not explicitly described and described while describing the embodiment, the effect that can be predicted by the configuration can also be recognized. The scope of the present invention is indicated by the claims rather than the foregoing description, and all differences within the equivalent scope should be interpreted as being included in the present invention.

Claims

1. In a method for providing information related to lawn care by using sensing information collected by a robot, A step of determining a search area to be subjected to lawn care based on map information; A step of determining a movement path for a mobile robot to move within the search area based on the search area and the map information; A step of obtaining sensing information including vision information obtained from a vision sensor included in the mobile robot moving based on the movement path and non-vision information obtained from a sensor other than the vision sensor; A step of processing the vision information and the non-vision information in different network models to obtain grass condition information for the search area; and A method comprising: providing lawn care solution information for lawn care for the search area based on the lawn condition information.

2. In paragraph 1, The step of obtaining the above grass condition information is A step of obtaining grass diagnosis information based on the result of processing the above vision information using the first network model; and A method comprising: a step of obtaining the grass condition information by applying the grass diagnosis information and the non-vision information to a second network model.

3. In paragraph 2, The above lawn care solution information is A method comprising lawn care know-how information and task command information for the mobile robot to perform a task corresponding to the lawn condition information in the search area.

4. In paragraph 2, The step of obtaining the above grass condition information is It further includes a step of obtaining sunlight information based on the result of processing the above vision information using a 3D map processing model, The step of obtaining the grass condition information by applying the above grass diagnosis information and the above non-vision information to the second network model is as follows. A method for obtaining the grass condition information by applying the above grass diagnosis information, the above non-vision information, and the above sunlight amount information to the second network model.

5. In paragraph 2, A method further comprising: a step of updating the first network model and the second network model based on the grass state information.

6. In paragraph 1, A step of obtaining obstacle information on the movement path based on the sensing information; and further comprising a step of updating the movement path based on the above obstacle information; A method wherein the above obstacle information includes obstacle location information on the movement path, slope information on the movement path, and path status information on the movement path.

7. In paragraph 1 A step of visualizing and displaying the above movement path; A step of activating a confirmation request button for the displayed movement path; and A method further comprising: providing a movement command to the mobile robot based on an approval input for the confirmation request button; 8. In paragraph 4 Further comprising a step of obtaining a 3D map representing the search area as at least one voxel using the 3D map processing model; The above sunlight information is A method determined based on GPS coordinate information and sun position information corresponding to each vertex of the above voxel.

9. In paragraph 3 The above work command information is A method comprising at least one of weeding, fertilizer application, pest control, watering, divot repair, and sowing.

10. In a device that provides information related to lawn care using a robot, Determine the search area to be targeted for lawn care based on map information, Based on the above search area and the map information, the mobile robot determines a movement path for moving within the search area, Obtain sensing information including vision information obtained from a vision sensor included in the mobile robot moving based on the movement path and non-vision information obtained from a sensor other than the vision sensor, The above vision information and the above non-vision information are processed in different network models to obtain grass condition information for the search area, A device comprising a processor that provides lawn care solution information for lawn care for the search area based on the lawn condition information.

Images