Control method of self-moving equipment and self-moving equipment
By using binocular vision modules and image processing technology on self-mobile devices, we can identify and cut high grass that is missed, and solve the problems of low mowing efficiency and grass leaf spread, and achieve efficient lawn management.
Patent Information
- Application Number
- CN202310729104.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-19
- Publication Date
- 2024-12-24
AI Technical Summary
Visual self-mobile devices such as monocular visual mowing robots find it difficult to identify obstacles such as hedges or flowers during the mowing process, resulting in low mowing efficiency and inability to deal with the missed mowing grass, which is then detected as an obstacle by ultrasound, leaving and spreading. .
By equipping a binocular vision module on the mobile device, the image of the forward direction is obtained, parallax processing, edge extraction and color segmentation are performed, the number of pixels whose pixel values are greater than the set threshold is determined, and whether there are leaked tall grass, and The control device moves toward the tall grass for cutting.
Timely handling of leaky grass mowing is achieved, preventing grass from growing taller and spreading, reducing manual intervention, improving mowing efficiency and neatness of the lawn in the area.
Smart Images
Figure CN119179323A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of path planning, and in particular to a control method of a self-moving device and a self-moving device. Background Art
[0002] The visual self-moving device does not require wiring and installation teaching, and can be used as soon as it is put in. Taking the visual self-moving device as an example, the monocular vision lawn mowing robot is generally equipped with ultrasonic auxiliary obstacle detection to prevent collisions with obstacles such as hedges or flowers that are not easily identified by the visual module. The visual self-moving device usually adopts a random mowing mode, and the traversal efficiency is not high, so the grass will be missed in some places. But over time, these missed grasses are high enough to be easily identified as obstacles by ultrasonic waves, so they will be left behind and will gradually spread to the surrounding areas. Summary of the invention
[0003] The technical problem to be solved by the present invention is to provide a control method for a self-moving device and a self-moving device, so as to effectively and timely process the grass that is missed from being cut.
[0004] To solve the above technical problems, according to one aspect of the present invention, a control method for a self-moving device is provided, which is used to control the self-moving device to move in a specified area to process a predetermined object in the specified area, comprising:
[0005] During the movement of the self-moving device, acquiring an image of an area in a direction in which the self-moving device moves;
[0006] According to the image, it is determined whether the predetermined objects in the area in the advancing direction of the self-moving device include a special predetermined object to control the moving direction of the self-moving device, wherein at least one characteristic parameter of the special predetermined object and the predetermined object is different.
[0007] According to another aspect of the present invention, there is provided a self-moving device, the self-moving device comprising:
[0008] Vision module;
[0009] at least one processor; and
[0010] a memory communicatively connected to the at least one processor; wherein,
[0011] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the self-moving device described in any embodiment of the present invention.
[0012] Compared with the prior art, the present invention obtains an image of the area in the forward direction of the self-moving device while the self-moving device moves in the designated area; processes the image to determine whether the predetermined objects in the area in the forward direction of the self-moving device include a special predetermined object, and controls the self-moving device to move in the direction including the special predetermined object to process the special predetermined object, such as cutting tall grass, thereby achieving effective and timely processing of missed grass and preventing the missed grass from growing taller and spreading to a wider area and requiring manual processing.
[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0015] Figure 1 is a flow chart of a method for controlling a self-moving device in an embodiment of the present invention;
[0016] Figure 2 is a flowchart of an implementation step of step S2 in an embodiment of the present invention;
[0017] Figure 3 is a flowchart of an implementation step of step S21 in an embodiment of the present invention;
[0018] Figure 4 is a flowchart of an implementation step of step S22 in an embodiment of the present invention;
[0019] Figure 5 is a flow chart of obtaining a thresholded image in an embodiment of the present invention;
[0020] Figure 6 is a flowchart of another implementation step of step S22 in an embodiment of the present invention;
[0021] Figure 7 It is a schematic diagram of the structure of a self-moving device implementing an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] Figure 1 is a flow chart of a control method of a self-moving device in one embodiment of the present invention. This embodiment can be applied to controlling the self-moving device to move in a specified area to process a predetermined object in the specified area, such as Figure 1 As shown, the method specifically comprises the following steps:
[0025] S1. During the movement of the self-moving device, an image of the area in the moving direction of the self-moving device is acquired.
[0026] In this embodiment, the self-moving device is a lawn mowing robot. The designated area is the area where the lawn mowing robot needs to mow. The predetermined object is the grass in the area where the lawn mowing robot needs to mow.
[0027] In this embodiment, images are collected by a visual module on a self-mobile device. The visual module on the self-mobile device includes a first image module and a second image module, that is, a binocular vision module is set on the self-mobile device in this embodiment. During the movement of the self-mobile device, the first image module is used to obtain a first image of the area in the forward direction of the self-mobile device, and the second image module is used to obtain a second image of the area in the forward direction of the self-mobile device. Images have multiple characteristics, such as hue (Hue, H), saturation (Saturation, S), and value (V).
[0028] S2. Determine, based on the image, whether the predetermined objects in the area in the advancing direction of the self-moving device include a special predetermined object, so as to control the moving direction of the self-moving device, wherein at least one characteristic parameter of the special predetermined object and the predetermined object is different.
[0029] In this embodiment, the specially predetermined object is the grass that is missed to be mowed in the area that needs to be mowed by the mowing robot, wherein the height of the missed grass is higher than the height of the surrounding grass.
[0030] Specifically, the first image and the second image acquired as above are first processed to obtain a processed image. The number of pixel points in the first part and the second part of the processed image whose pixel values are greater than a set threshold is counted, and whether the predetermined objects include a special predetermined object is determined according to the statistical result, and finally the mobile device is controlled to move toward the direction of the first part and the second part that includes the special predetermined object.
[0031] The processing may include disparity processing, edge extraction, color segmentation, etc. When it is confirmed that a specific object exists, the mobile device is controlled to move directly toward the specific object for cutting or fixed-point spiral cutting.
[0032] Figure 2 1 is a flowchart of an implementation step of step S2 in an embodiment of the present invention. Figure 2 As shown, the method of determining whether the predetermined objects in the area in the advancing direction of the self-moving device include a special predetermined object according to the image to control the moving direction of the self-moving device may include the following detailed steps:
[0033] S21. Process the image.
[0034] The first image and the second image are processed by performing disparity processing, edge extraction, color segmentation and the like to obtain processed images.
[0035] S22. Determine whether the predetermined objects include a special predetermined object according to predetermined elements respectively included in the first part and the second part of the processed image, wherein the first part and the second part are independent of each other and have equal areas.
[0036] Specifically, the first part and the second part can be divided into two parts with the vertical center line of the processed image as the dividing line. It can be understood that the first part corresponds to the image on the left side of the forward direction of the lawn mower robot, and the second part corresponds to the image on the right side of the forward direction of the lawn mower robot. The predetermined element includes the number of pixels whose pixel values are greater than a set threshold. The feature parameter can be a parameter of a feature such as color or contour. The number of pixels whose pixel values are greater than a set threshold in each of the first part and the second part of the processed image is counted, and whether the predetermined object includes a special predetermined object is determined based on the statistical result.
[0037] S23. Control the mobile device to move toward the direction of the first part or the second part containing the specially predetermined object.
[0038] According to the statistical results, it is determined that the predetermined objects include the special predetermined objects, that is, according to the statistical results, it is determined that there is missed grass in the area that needs to be mowed by the lawn mowing robot, then the mobile device is controlled to move toward the direction of the first part and the second part that includes the special predetermined object, that is, the lawn mowing robot is controlled to move in the direction of the missed grass.
[0039] Figure 3 1 is a flowchart of an implementation step of step S21 in an embodiment of the present invention. Figure 3 As shown, the image processing may include the following detailed steps:
[0040] S211. Obtain an initial disparity map according to the first image and the second image.
[0041] In this step, the technical means used to obtain the initial disparity map according to the first image and the second image are conventional techniques, which will not be described in detail here.
[0042] S212. Obtain a relative disparity map according to the initial disparity map and the reference disparity map.
[0043] Specifically, a reference disparity map is first obtained, and then an absolute value of a difference between pixel values of corresponding pixels in the initial disparity map and the reference disparity map is calculated to generate a relative disparity map.
[0044] The reference disparity map may be obtained by pre-photographing the horizontal ground using the first image module and the second image module of the visual module to obtain a flat ground image, and then processing the flat ground image to obtain the reference disparity map.
[0045] S213. Perform edge extraction on the relative disparity map to obtain an edge image.
[0046] It is important to know that the edge of an image is the most basic feature of an image. An edge refers to a sudden change in the grayscale information of a local characteristic of an image.
[0047] Specifically, edge extraction is performed on the relative disparity map to obtain an edge image.
[0048] Figure 4 Flow chart of an implementation step of step S22 in an embodiment of the present invention. Figure 4 As shown, according to the predetermined elements respectively included in the first part and the second part of the processed image, determining whether the predetermined objects include the special predetermined object may include the following detailed steps:
[0049] S221. Count the number of pixels whose pixel values are greater than 0 in the first part and the second part of the edge image respectively, wherein the first part and the second part are symmetrical about the center of the edge image.
[0050] S222. Determine whether the difference between the number of pixel points with pixel values greater than 0 between the first part and the second part satisfies a preset condition, and if so, determine that the first part and the second part with a larger number of pixel points with pixel values greater than 0 contain the special predetermined object. The preset condition is: the number of pixel points of the first part and the second part with a larger number of pixel points with pixel values greater than 0 is a preset multiple of the number of pixel points of the first part and the second part with a smaller number of pixel points with pixel values greater than 0.
[0051] Specifically, the edge image is the processed image described above, and the number of pixels with pixel values greater than 0 in the first part and the second part of the edge image are counted respectively, wherein the first part and the second part can be the part after the vertical center line of the processed image is used as the dividing line, and the first part and the second part are symmetrical about the center of the edge image. If the number of pixels in the first part and the second part with more pixel values greater than 0 exceeds the number of pixels in the first part and the second part with less pixel values greater than 0 by a preset multiple, it is determined that the first part and the second part with more pixel values greater than 0 contains a special predetermined object. The preset multiple can be 1.5 times.
[0052] Figure 5 is a flow chart of obtaining a threshold image in an embodiment of the present invention; Figure 5 As shown, the image is processed, and the following steps are also included:
[0053] S214. Convert the first image or the second image into an HSV (Hue, Saturation, Value) image.
[0054] In this step, firstly, any one of the first image or the second image is converted into an HSV image.
[0055] S215. Binarize the HSV image using a parameter threshold corresponding to a predetermined color to obtain a thresholded image, wherein the predetermined color is consistent with the color of the predetermined object.
[0056] Specifically, the color of the predetermined object may be the color of lawn grass, which is green. The predetermined color is consistent with the color of the predetermined object, which is also green. A parameter threshold corresponding to the color consistent with the color of the predetermined object is obtained, and then the HSV image is binarized based on the parameter threshold. The value of the pixel point located at the parameter threshold becomes 255, and the value of the other pixel points becomes 0, thereby obtaining a thresholded image.
[0057] Figure 6 FIG. 4 is a flowchart of another implementation step of step S22 in an embodiment of the present invention. Figure 6 As shown, the step of determining whether the predetermined objects include the special predetermined object according to the predetermined elements respectively included in the first part and the second part of the processed image may include the following detailed steps:
[0058] S223. Count the number of pixel points whose pixel values are greater than 0 in the first part and the second part of the edge image respectively, and count the number of pixel points whose pixel values are greater than 0 in the first part and the second part of the thresholded image respectively.
[0059] Specifically, first, both the edge image and the threshold image can be used as the processed image, then each image is divided into two parts according to the vertical center line as the dividing line, and the images on the same side are grouped as a group, and the whole is divided into the first part and the second part.
[0060] The number of pixel points whose pixel values are greater than 0 in the first part and the second part of the edge image is counted, and the number of pixel points whose pixel values are greater than 0 in the first part and the second part of the thresholded image is counted respectively.
[0061] S224. Determine whether the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the edge image meets a preset condition, and determine whether the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the thresholded image meets a preset condition.
[0062] Among them, the preset condition is that the number of pixels in the first part and the second part with more pixel values greater than 0 is a preset multiple of the number of pixels in the first part and the second part with less pixel values greater than 0.
[0063] S225. If the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the edge image meets the preset conditions, and the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the thresholded image meets the preset conditions, then it is determined that the side with more pixel points with pixel values greater than 0 in the first part and the second part contains a special predetermined object.
[0064] If the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the edge image meets the preset conditions, and the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the thresholded image meets the preset conditions, then it is determined that the side with more pixel points with pixel values greater than 0 in the first part and the second part contains a special predetermined object.
[0065] The technical solution of the embodiment of the present invention is to obtain an image of the area in the forward direction of the self-moving device during its movement; determine, based on the image, whether the predetermined objects in the area in the forward direction of the self-moving device include a special predetermined object, so as to control the moving direction of the self-moving device to process the special predetermined object, such as cutting tall grass, thereby achieving effective and timely processing of missed grass and preventing the missed grass from growing taller and spreading to a wider area and requiring manual processing.
[0066] Figure 7 A schematic diagram of a self-moving device 70 that can be used to implement an embodiment of the present invention is shown. The self-moving device 70 is intended to represent various forms of mobile devices, such as a lawn mowing robot. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.
[0067] like Figure 7 As shown, the self-mobile device 70 includes a visual module 71, at least one processor 72, and a memory 73 that is communicatively connected to the at least one processor 72. The visual module 71 can obtain image information in the forward direction of the self-mobile device 70, and the processor 72 can execute the various methods and processes described above, such as the self-mobile device control method:
[0068] During the movement of the self-moving device, an image of the area in the forward direction of the self-moving device is obtained; based on the image, it is determined whether the predetermined objects in the area in the forward direction of the self-moving device include a special predetermined object, so as to control the moving direction of the self-moving device, wherein at least one characteristic parameter of the special predetermined object and the predetermined object is different.
[0069] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0070] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A control method for a self-moving device, used to control the self-moving device to move in a specified area to process a predetermined object in the specified area, characterized in that: include: During the movement of the self-moving device, acquiring an image of an area in a direction in which the self-moving device moves; According to the image, it is determined whether the predetermined objects in the area in the advancing direction of the self-moving device include a special predetermined object to control the moving direction of the self-moving device, wherein at least one characteristic parameter of the special predetermined object and the predetermined object is different.
2. The control method according to claim 1, characterized in that: Determining, according to the image, whether the predetermined objects in the area in the advancing direction of the self-moving device include a special predetermined object, comprises: The image is processed, and according to predetermined elements respectively contained in a first part and a second part of the processed image, it is determined whether the predetermined objects include a special predetermined object, wherein the first part and the second part are independent of each other and have equal areas.
3. The control method according to claim 2, characterized in that: The step of acquiring an image of an area in a moving direction of the self-moving device during the moving of the self-moving device comprises: During the movement of the self-moving device, a first image module is used to acquire a first image of the area in the moving direction of the self-moving device, and a second image module is used to acquire a second image of the area in the moving direction of the self-moving device.
4. The control method according to claim 3, characterized in that: The processing of the image comprises: Obtaining an initial disparity map according to the first image and the second image; Obtaining a relative disparity map according to the initial disparity map and the reference disparity map; Perform edge extraction on the relative disparity map to obtain an edge image.
5. The control method according to claim 4, characterized in that: The determining whether the predetermined objects include a special predetermined object according to the predetermined elements respectively included in the first part and the second part of the processed image includes: Counting the number of pixels having pixel values greater than 0 in the first part and the second part of the edge image respectively, wherein the first part and the second part are symmetrical about the center of the edge image; Determine whether the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part meets a preset condition. If so, determine that the side with more pixel points with pixel values greater than 0 between the first part and the second part contains a special predetermined object.
6. The control method according to claim 4, characterized in that: The processing of the image comprises: Convert the first image or the second image into an HSV image; The HSV image is binarized using a parameter threshold corresponding to a predetermined color to obtain a thresholded image, wherein the predetermined color is consistent with the color of the predetermined object.
7. The control method according to claim 6, characterized in that: The determining whether the predetermined objects include a special predetermined object according to the predetermined elements respectively included in the first part and the second part of the processed image includes: Counting the number of pixel points whose pixel values are greater than 0 in the first part and the second part of the edge image respectively, and counting the number of pixel points whose pixel values are greater than 0 in the first part and the second part of the thresholded image respectively; Determining whether a difference in the number of pixel points having pixel values greater than 0 between the first part and the second part of the edge image satisfies a preset condition, and determining whether a difference in the number of pixel points having pixel values greater than 0 between the first part and the second part of the thresholded image satisfies a preset condition; If the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the edge image meets a preset condition, and the difference in the number of pixel points with pixel values greater than 0 between the first part and the second part of the thresholded image meets a preset condition, then it is determined that the side with more pixel points with pixel values greater than 0 between the first part and the second part contains the special predetermined object.
8. The control method according to claim 5 or 7, characterized in that: The preset condition is: the number of pixels in the first part and the second part with a larger number of pixels having pixel values greater than 0 is a preset multiple of the number of pixels in the first part and the second part with a smaller number of pixels having pixel values greater than 0.
9. The control method according to claim 5 or 7, characterized in that: The controlling the moving direction of the self-moving device includes: The self-moving device is controlled to move toward a direction including a specific predetermined object in the first part and the second part.
10. A self-propelled device, characterized in that: The self-mobile device comprises: Vision module; at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the self-moving device control method according to any one of claims 1 to 9.