Control method, storage medium and autonomous working machine

Abstract

The invention relates to a control method, a storage medium and an autonomous working machine, the control method is applied to the autonomous working machine, the autonomous working machine is provided with a camera, the autonomous working machine has an automatic mapping mode and a remote control mapping mode, the automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of a working area, and the remote control mapping mode is a remote control mapping mode. The remote control mapping mode is a mapping mode in which a user remotely controls the autonomous working machine to move along the boundary; the control method comprises the steps of obtaining an image based on a camera; performing image processing on the image to obtain an image processing result; under the condition that the image processing result represents that the autonomous working machine is located at the safety boundary position, the autonomous working machine is controlled to move based on an automatic mapping mode; and under the condition that the image processing result represents that the autonomous working machine is located at the unsafe boundary position, controlling the autonomous working machine to move based on a remote control mapping mode. The mapping efficiency and safety are improved, user operation is reduced, and meanwhile the accuracy of the map is ensured.

Description

Technical Field

[0001] The present invention relates to a control method, a storage medium and an autonomous working machine. Background Art

[0002] With the gradual popularization of intelligent devices in daily life, autonomous working machines (such as automatic lawn mowers for garden repairs and sweeping robots for home floor cleaning) are also popular among users.

[0003] Autonomous machines typically move and / or operate within a work area. For example, a robotic lawn mower can drive across a user's lawn and perform its mowing operations, significantly reducing labor costs.

[0004] Currently, smart lawn mowers on the market usually require users to create a work area map of the lawn when using it for the first time, and the lawn mower moves according to the work area map during subsequent mowing. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the problem to be solved by the embodiments of the present disclosure is to improve the efficiency and security of mapping, reduce user operations, and ensure the accuracy of the map.

[0006] In some embodiments, a control method for an autonomous working machine is provided, wherein the autonomous working machine has a camera, and the autonomous working machine has an automatic mapping mode and a remote control mapping mode. The automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of a working area, and the remote control mapping mode is a mapping mode in which a user remotely controls the movement of the autonomous working machine along the boundary.

[0007] The control method includes:

[0008] Acquire an image based on the camera;

[0009] performing image processing on the image to obtain an image processing result;

[0010] When the autonomous working machine is in the remote-controlled mapping mode and the image processing result indicates that the autonomous working machine is located at a safety boundary position, switching the mapping mode of the autonomous working machine from the remote-controlled mapping mode to the automatic mapping mode, and controlling the autonomous working machine to move based on the automatic mapping mode;

[0011] When the autonomous working machine is in the automatic mapping mode and the image processing result indicates that the autonomous working machine is located at an unsafe boundary position, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

[0012] In some embodiments, the unsafe boundary includes one or more of a pool boundary, a curved boundary, a pit boundary, and a narrow area boundary.

[0013] In some embodiments, the autonomous working machine is configured to determine visual feature information based on images captured by the camera, and perform simultaneous positioning and map construction based on the visual feature information and the inertial navigation information; the control method further includes:

[0014] When the autonomous working machine is in the automatic mapping mode and the positioning accuracy of the autonomous working machine is less than a first threshold, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

[0015] In some embodiments, the autonomous working machine is configured to determine visual feature information based on images captured by the camera, and perform simultaneous positioning and map construction based on the visual feature information and the inertial navigation information; the control method further includes:

[0016] When the autonomous working machine is in the remote control mapping mode and the positioning accuracy of the autonomous working machine is not less than a first threshold, the mapping mode of the autonomous working machine is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine is controlled to move based on the automatic mapping mode.

[0017] In some embodiments, the autonomous working machine is configured to determine visual feature information based on images captured by the camera, and perform simultaneous positioning and map construction based on the visual feature information and the inertial navigation information; the control method further includes:

[0018] When the autonomous working machine is in the automatic mapping mode and the amount of visual feature information determined based on the image is less than a second threshold, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

[0019] In some embodiments, the autonomous working machine is configured to determine visual feature information based on images captured by the camera, and perform simultaneous positioning and map construction based on the visual feature information and the inertial navigation information; the control method further includes:

[0020] When the autonomous working machine is in the remote control mapping mode and the amount of visual feature information determined based on the image is not less than a second threshold, the mapping mode of the autonomous working machine is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine is controlled to move based on the automatic mapping mode.

[0021] In some embodiments, the control method further includes: when the autonomous working machine is in the automatic mapping mode and a dynamic object is detected, switching the mapping mode of the autonomous working machine from the automatic mapping mode to the remote control mapping mode, and controlling the autonomous working machine to move based on the remote control mapping mode.

[0022] In some embodiments, switching the mapping mode of the autonomous machine from the automatic mapping mode to the remote mapping mode includes:

[0023] controlling the autonomous working machine to issue a reminder message;

[0024] When the autonomous machine receives a remote mapping instruction, the mapping mode of the autonomous machine is switched from the automatic mapping mode to the remote mapping mode.

[0025] In some embodiments, switching the mapping mode of the autonomous machine from the remote mapping mode to the automatic mapping mode includes:

[0026] controlling the autonomous working machine to issue a reminder message;

[0027] When the autonomous machine receives an automatic mapping instruction, the mapping mode of the autonomous machine is switched from the remote control mapping mode to the automatic mapping mode.

[0028] In some embodiments, a control method for an autonomous working machine is provided, wherein the autonomous working machine has a camera and an inertial navigation module, wherein the camera is configured to acquire images, and the inertial navigation module is configured to generate inertial navigation information, and the autonomous working machine has an automatic mapping mode and a remote control mapping mode;

[0029] In the remote control mapping mode or the automatic mapping mode, the autonomous working machine determines visual feature information based on the image, and performs synchronous positioning and map construction based on the visual feature information and the inertial navigation information;

[0030] The automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of the working area, and the remote control mapping mode is a mapping mode in which the user remotely controls the movement of the autonomous working machine along the boundary;

[0031] The control method includes:

[0032] When the autonomous machine is in the automatic mapping mode and the visual feature information determined based on the image does not meet an automatic mapping condition, the mapping mode of the autonomous machine is switched from the automatic mapping mode to the remote control mapping mode.

[0033] In some embodiments, a control method for an autonomous working machine is provided, which is applied to the autonomous working machine, wherein the autonomous working machine has a camera, and the autonomous working machine is configured to generate a work area map based on images captured by the camera during a mapping process; the autonomous working machine has an automatic mapping mode and a remote control mapping mode, wherein the automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of the work area, and the remote control mapping mode is a mapping mode in which a user remotely controls the movement of the autonomous working machine along the boundary;

[0034] The control method includes:

[0035] Get initial instructions;

[0036] In a case where the initial instruction is an automatic mapping instruction, controlling the machine to move based on the automatic mapping mode;

[0037] When a mode switching condition is met, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode. The mode switching condition includes detecting that the autonomous working machine is in a slipping condition, a stuck condition, a bumpy condition, a visual feature missing condition, or an image boundary missing condition.

[0038] In some embodiments, the control method further comprises:

[0039] In a case where the initial instruction is a remote control mapping instruction, the machine is controlled to move based on the remote control mapping mode, and mode switching is prohibited.

[0040] In some embodiments, when the positioning accuracy determined based on the image is less than a first threshold, it is determined that the autonomous working machine is in the visual feature missing working condition.

[0041] In some embodiments, when the amount of visual feature information determined based on the image is less than a second threshold, it is determined that the autonomous working machine is in the visual feature missing working condition.

[0042] In some embodiments, when it is detected that the autonomous working machine is in a bumpy condition, a slipping condition, a stuck condition, or a condition where the image boundary is lost, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to continue mapping from the current position.

[0043] In some embodiments, the autonomous working machine is configured to interact with a user terminal, and the user terminal is configured to display remote control controls during the mapping process of the autonomous working machine;

[0044] When a mode switching condition is met, switching the mapping mode from the automatic mapping mode to the remote control mapping mode includes:

[0045] When the mode switching condition is met, sending a mode switching request to the user terminal, so that the user terminal generates and sends a user remote control instruction according to the user's operation instruction on the remote control control;

[0046] After receiving the user remote control instruction, the machine is controlled to move based on the remote control mapping mode and according to the user remote control instruction.

[0047] In some embodiments, when it is detected that the autonomous working machine is in a visual feature missing condition or an image boundary missing condition, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to re-map.

[0048] In some embodiments, the autonomous working machine is configured to interact with a user terminal;

[0049] Switching the mapping mode from the automatic mapping mode to the remote control mapping mode includes:

[0050] sending a mode suggestion instruction to the user terminal so that the user terminal displays mode suggestion information, where the mode suggestion information is used to suggest the user to switch the mapping mode to the remote control mapping mode;

[0051] After receiving the user remote control instruction sent by the user terminal, the machine is controlled to move based on the remote control mapping mode and according to the user remote control instruction.

[0052] In some embodiments, controlling the autonomous working machine to remap includes:

[0053] Controlling the autonomous working machine to return to the starting position of the current mapping;

[0054] The autonomous working machine is controlled to start mapping from the mapping starting position.

[0055] In some embodiments, controlling the autonomous working machine to remap includes:

[0056] Determine target locations on the boundary;

[0057] The autonomous working machine is controlled to start mapping from the target position.

[0058] In some embodiments, when it is detected that the autonomous working machine is in the image boundary loss state for a preset time period, the autonomous working machine is controlled to search for a boundary, and after the boundary is found, the autonomous working machine is controlled to re-build the image;

[0059] If the number of times the autonomous working machine is in the image boundary loss condition is greater than a preset number, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to re-map.

[0060] In some embodiments, a control method for an autonomous working machine is provided, the method being applied to the autonomous working machine, the autonomous working machine having a camera, the control method comprising:

[0061] After the autonomous working machine is powered on for the first time, collecting images based on the camera;

[0062] determining visual feature information based on the image;

[0063] In the absence of obtaining a work area map, the autonomous working machine is controlled to move and work in the work area according to the visual feature information.

[0064] In some embodiments, the work area map is generated according to the visual feature information while the autonomous working machine is moving and working.

[0065] In some embodiments, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above method.

[0066] In some embodiments, an autonomous working machine is provided, comprising:

[0067] processor;

[0068] a memory for storing instructions executable by the processor;

[0069] The processor is used to execute the above method.

[0070] The embodiments of the present disclosure can improve map building efficiency and security, reduce user operations, and ensure map accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The above-mentioned objectives, technical solutions and beneficial effects of the present invention can be clearly obtained through the following detailed description of specific embodiments that can implement the present invention, combined with the description of the accompanying drawings.

[0072] The same reference numerals and symbols are used in the drawings and the description to designate the same or equivalent elements.

[0073] Figure 1 A schematic diagram of a lawn mower moving within a working area provided for some exemplary embodiments of the present application;

[0074] Figure 2 A schematic diagram of the structure of an autonomous working machine provided in some embodiments of the present application;

[0075] Figure 3 A flowchart of a control method provided in some embodiments of the present application;

[0076] Figure 4 A schematic diagram of an environment of a user's work area;

[0077] Figure 5 A schematic diagram of an autonomous working machine mapping a working area provided for some exemplary embodiments of the present application;

[0078] Figure 6 A flowchart of a control method provided in some other embodiments of the present application. DETAILED DESCRIPTION

[0079] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the accompanying drawings. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure. The embodiments provided in this specification may be combined with each other.

[0080] In the present invention, unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0081] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0082] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0083] The embodiments provide a control method, a storage medium, and an autonomous working machine. The autonomous working machine may be an autonomously moving smart device, such as an autonomous lawn mower, an autonomous vacuum cleaner, an autonomous mop, or an autonomous snow blower. The autonomous machine can automatically move within a specified working area and perform corresponding tasks, or it can return to a docking station along the boundary of the working area for docking or charging.

[0084] In some embodiments, an autonomous working machine is provided, such as Figure 1 and Figure 2 As shown, the autonomous working machine 30 includes a body 100 , an imaging sensor 200 , a position sensor 500 , and a control circuit 600 .

[0085] Specifically, the autonomous working machine 30 includes a drive device 700 mounted on the machine body 100. The drive device 700 is configured to drive the machine body 100 on a work surface in response to received drive commands. The drive device 700 typically includes rollers and a motor that drives the rollers. The rollers may include a driving wheel and a driven wheel. The rollers may be located on both sides of the machine body 100, with one or two rollers on each side.

[0086] The autonomous working machine 30 also includes a working module, which is used to perform specific work tasks. For example, if the autonomous working machine 30 is an automatic lawn mower, the working module includes mowing blades, a cutting motor, and other components. It may also include auxiliary components such as a mowing height adjustment mechanism to optimize or adjust the mowing effect. For example, if the autonomous working machine 30 is an automatic vacuum cleaner, the working module includes a vacuum motor, a vacuum port, a vacuum tube, a vacuum chamber, a dust collection device, and other working components used to perform the vacuuming task.

[0087] The autonomous working machine 30 may also include an energy module, which is used to provide energy for various operations of the autonomous working machine 30. The energy module may include a rechargeable battery and a charging connection structure, wherein the charging connection structure is usually a charging electrode sheet, which can be used in conjunction with a charging electrode sheet set at the docking station to charge the autonomous working machine 30.

[0088] The autonomous working machine 30 further includes a memory 400 for storing data generated by sensors or control circuits, or pre-storing data for use by the control circuits.

[0089] The autonomous working machine 30 further includes a position sensor 500 , which may include an IMU (inertial sensor) or an ODO (odometer) installed on the driving device 700 , etc., for obtaining a relative position according to the movement of the body 100 .

[0090] In addition to the above modules, the autonomous working machine 30 may also include a shell for accommodating and installing each module, a control panel for user operation, etc. It may also include various environmental sensors, such as humidity sensors, temperature sensors, acceleration sensors, light sensors, etc. The above sensors can assist the autonomous working machine 30 in judging the working environment to execute the corresponding program.

[0091] The control circuit 600 is the core component of the autonomous working machine 30, which is used to control the automatic movement and operation of the autonomous working machine 30. The functions it performs include controlling the working module to start or stop, controlling the movement of the drive device 700, judging the power of the energy module and timely controlling the autonomous working machine 30 to return to the docking station for automatic docking and charging, and executing corresponding programs in combination with the data from the environmental sensor.

[0092] Reference Figure 1 and Figure 2 The autonomous working machine 30 includes an imaging sensor 200 connected to the body 100 and configured to capture images in the forward direction of the body 100. The images are at least partially images of the working surface in the forward direction. The captured images are within a field of view 210 of the imaging sensor 200. The imaging sensor 200 may be a camera or a laser radar commonly used in the industry.

[0093] Typically, the imaging sensor 200 is mounted near the front upper portion of the body 100, preferably centered, with its viewing angle facing forward and downward to capture images of the work surface. Its field of view 210 can be adjusted based on actual needs. A larger field of view 210 captures more images in the forward direction of the body 100, and vice versa. The forward direction of the body 100 can be various, such as normal forward movement, backward movement, and turning. In this embodiment, the forward direction of the body 100 refers to the normal forward direction, i.e., the direction of the central axis of the body 100.

[0094] To address the technical challenges of improving mapping efficiency and safety, reducing user effort, and ensuring map accuracy, this embodiment provides a control method for an autonomous working machine 30. The autonomous working machine 30 is equipped with a camera, which can be located on the front of the machine body and can capture images of the working area. The autonomous working machine 30 is configured to move and / or operate within the working area, as well as at the boundaries of the working area.

[0095] After purchasing the autonomous machine 30, users will need to configure it accordingly, which includes creating a map of the user's home work area. The map includes at least the coordinates of multiple locations corresponding to the boundaries of the work area. The map can be stored on the machine or in a cloud server, or the user can access it via a mobile phone or other electronic device. The map is used to define the operating range of the autonomous machine 30, allowing the autonomous machine 30 to move or operate within, on, and / or near these boundaries according to the map.

[0096] In some embodiments, the mapping method includes: the autonomous working machine 30 moves along the boundary, collecting coordinate information during the movement, and obtaining a boundary map of the work area after the autonomous working machine 30 moves along the boundary once. The coordinate information / positioning information can be collected based on positioning technologies such as GPS, RTK, IMU, or VSLAM.

[0097] Taking a lawn mower as an example, different objects are often set at the boundaries of the working area (lawn), such as walls, trees, shrubs, swimming pools, fences, steps, decorative signs, stones, pits, roads, sidewalks, stepping stones, flowers, dynamic objects, etc., which makes the boundary situation very complicated and causes trouble for the movement of the autonomous working machine 30.

[0098] like Figure 4 As shown, the work area 1 of the user's home includes a boundary 10 and an area enclosed by the boundary 10. Near the boundary 10 of the work area 1 are arranged trees 11, a fence 12, stepping stones 13, and a swimming pool 14. The boundary 10 includes a relatively straight boundary section and a very curved and complex boundary section 15. Figure 4 In the example, the autonomous working machine 30 is docked at a charging station 31, which is located outside the working area 1 (it can also be located inside or on the boundary). The complex and diverse boundary conditions pose many difficulties for the autonomous working machine 30 in the mapping process, making it difficult for the autonomous working machine 30 to accurately map the area fully automatically.

[0099] In some embodiments, the autonomous machine 30 is equipped with a camera and has both automatic and remote mapping modes. Automatic mapping mode allows the autonomous machine 30 to automatically move along the boundaries of its work area. In automatic mapping mode, the autonomous machine 30 automatically identifies the boundaries of its work area and automatically moves along them, recording its coordinates during movement. Remote mapping mode allows the autonomous machine 30 to be moved along the boundaries by a user. The user operates a remote control device to control the direction of movement of the autonomous machine 30, causing it to move along the boundaries. The autonomous machine 30 moves based on the instructions of the remote control device, recording its coordinates during movement. Mapping can be performed in either automatic or remote mapping mode, or switching between different mapping modes can be performed during a single mapping process. The autonomous machine 30 is configured to determine visual feature information based on images captured by the camera, and to perform simultaneous positioning and map construction based on this visual feature information and inertial navigation information.

[0100] Combine Figure 3 As shown, in some embodiments, the control method of the autonomous working machine 30 includes:

[0101] S101: Acquire an image based on a camera.

[0102] S102: Perform image processing on the image to obtain an image processing result.

[0103] In some embodiments, there is one camera, which is located at the front of the machine. The camera can provide the machine with a forward field of view and take images, which can be two-dimensional images. After obtaining the image, the image is processed, and the image processing includes semantic segmentation, image recognition, image depth information estimation, post-processing analysis, etc., so as to obtain the image processing results. The image processing results include: the position of grass in the image, the position of non-grass, the boundary position between grass and non-grass, the type of non-grass, the depth information of each object in the image, the relative position relationship between non-grass and the machine, etc.; further, the mapping mode of the autonomous working machine 30 is determined according to the image processing results.

[0104] In some embodiments, the control method of the autonomous working machine 30 further includes:

[0105] S103: When the autonomous working machine 30 is in the remote mapping mode and the image processing result indicates that the autonomous working machine 30 is located at a safety boundary, switching the mapping mode of the autonomous working machine 30 from the remote mapping mode to the automatic mapping mode, and controlling the autonomous working machine 30 to move based on the automatic mapping mode;

[0106] S104: When the autonomous working machine 30 is in the automatic mapping mode and the image processing result indicates that the autonomous working machine 30 is located at an unsafe boundary position, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine 30 is controlled to move based on the remote control mapping mode.

[0107] If the autonomous working machine 30 moves near the safety boundary position, the possibility of damage, failure, shutdown and other problems of the machine is small. Therefore, at this time, the autonomous working machine 30 can move along the boundary based on the automatic mapping mode and collect positioning information during the movement.

[0108] An unsafe boundary location refers to a boundary that can cause or is highly likely to cause problems such as a machine collision, fall, jam, stop, slip, run out of bounds, or become entangled. If the autonomous machine 30 moves near an unsafe boundary location, non-grass objects there may cause the machine to collide, fall, get stuck, stop, slip, run out of bounds, or become entangled. In other words, if there are non-grass objects near the boundary that may cause the machine to collide, fall, get stuck, stop, slip, run out of bounds, or become entangled, this location is an unsafe boundary location, and the autonomous machine 30 should be avoided from moving automatically there. If the image processing results indicate that the autonomous machine 30 is currently located near an unsafe boundary location (e.g., the unsafe boundary is located to the side of the autonomous machine 30 or in the forward direction), the autonomous machine 30 automatically switches to remote mapping mode and maintains this mode, or the autonomous machine 30 generates a reminder message, prompting the user to manually switch to remote mapping mode and maintain this mode until leaving the unsafe boundary location. In the remote control mapping mode, the user controls the movement of the autonomous working machine 30 via the remote control device, so that the autonomous working machine 30 avoids unsafe boundary positions during the movement and collects positioning information during the movement.

[0109] The control method provided in the embodiment of this specification is applied to the autonomous working machine 30. The autonomous working machine 30 moves based on the automatic mapping mode at the safe boundary position, which can simplify user operations, improve mapping efficiency, and reflect the intelligence of the machine; it moves based on the remote control mapping mode at the unsafe boundary position, which can avoid damage, failure, out-of-bounds and other problems of the autonomous working machine 30 at the unsafe boundary position, thereby improving the accuracy and success rate of mapping; it avoids the trouble of users restarting multiple times due to failures in mapping, and avoids the situation where the autonomous working machine 30 gets into trouble in a potentially dangerous environment and needs to be rescued by the user.

[0110] When the machine moves from a safe boundary to an unsafe boundary, it automatically switches to remote mapping mode or issues a reminder message to remind the user to switch the machine to remote mapping mode. When the machine moves from an unsafe boundary to a safe boundary, it automatically switches to automatic mapping mode or issues a reminder message to remind the user to switch the machine to automatic mapping mode.

[0111] In some embodiments, the unsafe boundary includes one or more of a pool boundary, a curved boundary, a pit boundary, and a narrow area boundary. A curved boundary refers to a boundary with a relatively large curvature or a relatively complex shape.

[0112] If the autonomous working machine 30 moves automatically at the edge of a pool, it may fall into the pool. If it moves automatically at the edge of a pit, it may easily fall into the pit. If it moves at a curved edge, it may easily cause the machine to repeatedly turn or go out of bounds. If it moves at the edge of a narrow area, it may easily cause the machine to get stuck.

[0113] Combine Figure 4 As shown, boundary 10 includes a pool boundary 14, a curved boundary 15, a fence boundary, and a tree boundary. Autonomous machine 30 moving autonomously near pool boundary 14 could potentially cause it to fall into the pool. Moving autonomously near curved boundary 15 could potentially cause it to go out of bounds. Moving autonomously near a suspended fence could potentially cause it to become stuck under suspended fence 12. Moving autonomously near tree 11 could potentially cause a collision. Autonomous machine 30 moving autonomously near stepping stone 13 would not cause damage or other problems.

[0114] Combine Figure 5 As shown, based on the control method provided in this embodiment, the autonomous machine 30 moves within the unsafe boundary using a remote control mapping mode, with a movement trajectory 60 shown as the dashed line in the figure. As can be seen, under user control, the movement trajectory of the autonomous machine 30 near the unsafe boundary adapts to the terrain and environment. A map is generated based on the positioning information obtained by the autonomous machine 30 along the movement trajectory 60. Controlling the movement and / or operation of the autonomous machine 30 based on the map can prevent damage, collisions, out-of-bounds operations, and shutdowns near the unsafe boundary.

[0115] In some embodiments, the image is a two-dimensional image;

[0116] Perform image processing on the image to obtain image processing results, including:

[0117] Perform semantic segmentation on the image to determine the grass area information and non-grass area information;

[0118] determining depth information based on the image;

[0119] The grass area information, non-grass area information and depth information are post-processed to obtain an image processing result.

[0120] Based on the semantic segmentation model, the image is segmented to divide the image into grass and non-grass areas, determine the type of the non-grass area, and locate the corresponding boundary position in the image. Grass area information includes one or more of the following: location, area, length, and width; non-grass area information includes one or more of the following: type, location, area, length, and width. Depth information is estimated for the image to obtain the image depth information. The semantic segmentation results are fused and filtered with the depth information to obtain the image processing result.

[0121] In some embodiments, the mapping mode of the autonomous machine 30 is determined based on the image processing results. After obtaining the image processing results, if the autonomous machine 30 determines based on the image results that it is near an unsafe boundary, the autonomous machine 30 is controlled to move based on the remote mapping mode. If the autonomous machine 30 determines based on the image results that it is near a safe boundary, the autonomous machine 30 is controlled to move based on the automatic mapping mode.

[0122] Combine Figure 4 and Figure 5 As shown, near the pool 14, the autonomous working machine 30 can identify the non-grass area as a pool based on the image and obtain the depth information of the pool, thereby determining that the boundary of this location is an unsafe boundary; similarly, the same is true for locations such as trees, fences, and pits. Near the curved boundary 15, the boundary position can be determined based on the image, and fitting processing is performed to determine that the curvature of the boundary segment is large / unstraight / unreliable, and the boundary of this location is determined to be an unsafe boundary. Near the stepping stone 13, the autonomous working machine 30 can identify the non-grass area as a stepping stone based on the image and obtain the depth information of the stepping stone, thereby determining that the boundary of this location is a safe boundary; wherein, the stepping stone is generally a flat stone slab embedded in the lawn, and the autonomous working machine 30 can move safely and automatically on the stepping stone.

[0123] In some embodiments, the autonomous machine 30 also includes an inertial navigation module configured to generate inertial navigation information. The control method further includes, in remote mapping mode or automatic mapping mode, acquiring positioning information and building a map using VSLAM (Visual Simultaneous Localization and Mapping) technology. During the mapping process, the autonomous machine 30 determines visual feature information from the image and performs simultaneous positioning and mapping based on this visual feature information and inertial navigation information.

[0124] The camera serves as a perception module of the autonomous working machine 30 . The autonomous working machine 30 also has a vslam module. The vslam module is configured to run a vslam algorithm program. The vslam module is integrated in a control chip, and the control chip is set inside the autonomous working machine 30 .

[0125] In some embodiments, the inertial navigation module, perception module, and vslam module operate simultaneously. If the perception module detects an obstacle, the autonomous machine 30 can output an image outline of the obstacle, and the vslam module can output a point cloud (depth information) of the obstacle, allowing the autonomous machine 30 to determine whether it can safely escape the obstacle or bypass it. Simultaneously, the perception module segments the boundaries between grass and non-grass areas and determines whether to use remote mapping mode based on the complexity of the boundaries. The perception module and vslam module work together to classify and determine whether to switch mapping modes based on the image.

[0126] In some embodiments, the autonomous working machine 30 is configured to determine visual feature information based on images captured by a camera, and perform simultaneous positioning and map construction based on the visual feature information and inertial navigation information; the control method further includes:

[0127] When the autonomous working machine 30 is in the automatic mapping mode and the positioning accuracy of the autonomous working machine 30 is less than the first threshold, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine 30 is controlled to move based on the remote control mapping module.

[0128] Low positioning accuracy will cause the machine to be unable to automatically determine the direction of movement. In this case, the user needs to remotely build the map to improve the accuracy of the map.

[0129] In some embodiments, the autonomous working machine 30 is configured to determine visual feature information based on images captured by a camera, and perform simultaneous positioning and map construction based on the visual feature information and inertial navigation information; the control method further includes:

[0130] When the autonomous working machine 30 is in the remote control mapping mode and the positioning accuracy of the autonomous working machine 30 is not less than the first threshold, the mapping mode of the autonomous working machine 30 is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine 30 is controlled to move based on the automatic mapping mode.

[0131] Automatic mapping with good positioning accuracy can simplify user operations.

[0132] In some embodiments, the autonomous working machine 30 is configured to determine visual feature information based on images captured by a camera, and perform simultaneous positioning and map construction based on the visual feature information and inertial navigation information; the control method further includes:

[0133] When the autonomous working machine 30 is in the automatic mapping mode and the amount of visual feature information determined based on the image is less than a second threshold, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine 30 is controlled to move based on the remote control mapping mode.

[0134] In some embodiments, the autonomous working machine 30 is configured to determine visual feature information based on images captured by a camera, and perform simultaneous positioning and map construction based on the visual feature information and inertial navigation information; the control method further includes:

[0135] When the autonomous working machine 30 is in the remote control mapping mode and the amount of visual feature information determined based on the image is not less than the second threshold, the mapping mode of the autonomous working machine 30 is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine 30 is controlled to move based on the automatic mapping mode.

[0136] Vslam technology is based on visual features for positioning. If the visual feature information in the image is small, it will affect the positioning accuracy of Vslam.

[0137] Combine Figure 4 and Figure 5 As shown, the work area 1 of the user's home is adjacent to the neighbor's lawn 2, which is surrounded by a boundary 20. The neighbor's lawn 2 and the work area 1 of the user's home are divided by a boundary section 16. The boundary section 16 is usually surrounded by grass. In practice, the position of the boundary line is often not obvious, difficult to identify, and difficult to distinguish. For example, only a thin rope is set between adjacent lawns to divide lawns of different ownership.

[0138] When the autonomous machine 30 moves near the boundary 16, the proportion of grass in the image will increase significantly, while the proportion of other objects will decrease. This will result in a reduction in the effective visual features in the image, making it impossible to effectively extract and stably match visual features, reducing positioning accuracy, and making it impossible for the machine to determine which direction to move (losing the boundary), thereby deviating from the actual boundary or going out of bounds, reducing mapping accuracy.

[0139] Similarly, if the proportion of pure-color glass / curtain wall in the image captured by the camera is too high, it will also lead to insufficient effective visual features in the image.

[0140] When the autonomous working machine 30 detects that the amount of visual feature information determined based on the image is small, it moves and collects positioning information based on the remote control mapping model.

[0141] If there are rich texture features near the boundaries of the image, it is conducive to visual feature extraction and stable matching.

[0142] Light changes can also affect visual feature extraction and stable matching. Stable and uniform light is conducive to visual feature extraction and stable matching. If the light during mapping is too dim, the autonomous machine 30 switches to remote mapping mode.

[0143] In some embodiments, when the image processing result indicates that the autonomous working machine 30 is located at an unsafe boundary position, controlling the autonomous working machine 30 to move based on the remote control mapping mode includes:

[0144] When the autonomous working machine 30 is in the automatic mapping mode and the boundary information indicates that the autonomous working machine 30 is located at an unsafe boundary position, controlling the autonomous working machine 30 to issue a warning message;

[0145] When the autonomous working machine 30 receives the remote control mapping instruction, the autonomous working machine 30 is controlled to move based on the remote control mapping mode.

[0146] In some embodiments, a reminder message is issued, such as a control indicator light turning on, a buzzer alarm, a pop-up window appearing in an APP, etc.

[0147] Users can send remote control mapping instructions to the machine through the remote control device.

[0148] In some embodiments, if the positioning accuracy of the autonomous working machine 30 is low or the location is at an unsafe boundary, the remote control mapping mode is switched.

[0149] In some embodiments, if the autonomous machine 30 is in automatic mapping mode and detects a dynamic object, such as a person or small animal, the mapping mode of the autonomous machine 30 is switched from automatic mapping mode to remote mapping mode, and the autonomous machine 30 is controlled to move based on the remote mapping mode. Using remote mapping mode when a dynamic object is detected can prevent harm to people or animals and improve safety.

[0150] In some embodiments, switching the mapping mode of the autonomous working machine 30 from the automatic mapping mode to the remote mapping mode includes:

[0151] Controlling the autonomous working machine 30 to issue a reminder message;

[0152] When the autonomous working machine 30 receives the remote mapping instruction, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote mapping mode.

[0153] When the autonomous working machine 30 determines that the mode needs to be switched to the remote control mapping mode, the autonomous working machine 30 can issue a reminder message, for example, it can send a reminder message to the user terminal. After the user receives it, the user can operate on the user terminal, and the user terminal sends a remote control mapping instruction to the autonomous working machine 30. When the autonomous working machine 30 receives the remote control mapping instruction, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode.

[0154] In some embodiments, switching the mapping mode of the autonomous working machine 30 from the automatic mapping mode to the remote mapping mode includes:

[0155] Controlling the autonomous working machine 30 to issue a reminder message;

[0156] When the autonomous working machine 30 receives the automatic mapping instruction, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode.

[0157] When the autonomous working machine 30 determines that the mode needs to be switched to the automatic mapping mode, the autonomous working machine 30 can issue a reminder message, for example, it can send a reminder message to the user terminal. After the user receives it, the user can operate on the user terminal, and the user terminal sends an automatic mapping instruction to the autonomous working machine 30. When the autonomous working machine 30 receives the automatic mapping instruction, the mapping mode of the autonomous working machine 30 is switched from the remote control mapping mode to the automatic mapping mode.

[0158] In some embodiments, the number of cameras is multiple; or the camera is a 360-degree camera; or the camera is a TOF camera.

[0159] In some embodiments, the image may be an image with depth information.

[0160] In some embodiments, if any position of the boundary of the work area to be mapped is safe and the positioning accuracy of the machine at any position of the boundary is high, the entire process of mapping along the edge can use the automatic mapping mode without the need for user participation in mapping.

[0161] In some embodiments, if any position of the boundary of the working area to be mapped is safe and the positioning accuracy of the machine at any position of the boundary is high, the user can choose not to map and choose not to map on the control terminal of the autonomous working machine 30 (such as a mobile phone) or the control panel of the autonomous working machine 30. The autonomous working machine 30 then skips the mapping step and directly starts working (if it is a lawn mower, the work is mowing) or maps during the work process. The user does not need to pay attention to the operation of the machine, which simplifies user operations.

[0162] In some embodiments, if any position of the boundary of the working area to be mapped is safe and the positioning accuracy of the machine at any position of the boundary is high, the user can choose not to map, and choose not to map on the control terminal of the autonomous working machine 30 (such as a mobile phone) or the control panel of the autonomous working machine 30. The autonomous working machine 30 then starts to move and work automatically. During the automatic movement process, a map can be automatically created and stored without informing the user, and the user does not need to pay attention to the operation of the machine, thereby simplifying user operations.

[0163] This specification also provides a control method for an autonomous working machine 30, wherein the autonomous working machine 30 has a camera and an inertial navigation module, wherein the camera is configured to acquire an image, and the inertial navigation module is configured to generate inertial navigation information, and the autonomous working machine 30 has an automatic mapping mode and a remote control mapping mode;

[0164] In the remote control mapping mode or the automatic mapping mode, the autonomous working machine 30 determines visual feature information based on the image, and performs simultaneous positioning and map construction based on the visual feature information and inertial navigation information;

[0165] The automatic mapping mode is a mapping mode in which the autonomous working machine 30 automatically moves along the boundary of the working area, and the remote mapping mode is a mapping mode in which the user remotely controls the autonomous working machine 30 to move along the boundary;

[0166] Control methods include:

[0167] When the autonomous working machine 30 is in the automatic mapping mode and the visual feature information determined based on the image does not meet the automatic mapping conditions, the mapping mode of the autonomous working machine 30 is switched from the automatic mapping mode to the remote control mapping mode.

[0168] Automatic mapping conditions include high positioning accuracy or a large amount of visual feature information. If the automatic mapping conditions are not met, it means that the positioning accuracy is low or the visual feature information is insufficient. In this case, the autonomous working machine 30 moves based on the remote control mapping mode.

[0169] In addition, an embodiment is provided, which is a control device for an autonomous working machine 30. The autonomous working machine 30 has a camera and has an automatic mapping mode and a remote mapping mode. The automatic mapping mode is a mapping mode in which the autonomous working machine 30 automatically moves along the boundary of a working area. The remote mapping mode is a mapping mode in which a user remotely controls the movement of the autonomous working machine 30 along the boundary.

[0170] The control device includes:

[0171] An acquisition module is configured to acquire an image based on a camera;

[0172] An image processing module is configured to perform image processing on the image to obtain an image processing result;

[0173] A first control module is configured to control the autonomous working machine 30 to move based on the automatic mapping mode when the image processing result indicates that the autonomous working machine 30 is located at a safety boundary position;

[0174] The second control module is configured to control the autonomous working machine 30 to move based on the remote control mapping mode when the image processing result indicates that the autonomous working machine 30 is located at an unsafe boundary position.

[0175] The control device is integrated into a control chip, which is provided inside the autonomous working machine 30 .

[0176] In addition, an embodiment is provided, which is a control device for an autonomous working machine 30. The autonomous working machine 30 has a camera and an inertial navigation module. The camera is configured to acquire images, and the inertial navigation module is configured to generate inertial navigation information. The autonomous working machine 30 has an automatic mapping mode and a remote control mapping mode.

[0177] In the remote control mapping mode or the automatic mapping mode, the autonomous working machine 30 determines visual feature information based on the image, and performs simultaneous positioning and map construction based on the visual feature information and inertial navigation information;

[0178] The automatic mapping mode is a mapping mode in which the autonomous working machine 30 automatically moves along the boundary of the working area, and the remote mapping mode is a mapping mode in which the user remotely controls the autonomous working machine 30 to move along the boundary;

[0179] The control device includes:

[0180] The third control module is configured to switch the mapping mode of the autonomous working machine 30 to the remote control mapping mode when the autonomous working machine 30 is in the automatic mapping mode and the visual feature information determined based on the image does not meet the automatic mapping conditions.

[0181] The control device is integrated into a control chip, which is provided inside the autonomous working machine 30 .

[0182] Combine Figure 6 As shown, in some embodiments, a control method for an autonomous working machine 30 is provided, which is applied to the autonomous working machine 30. The autonomous working machine 30 has a camera, and the autonomous working machine 30 is configured to generate a work area map based on images collected by the camera during a mapping process; the autonomous working machine 30 has an automatic mapping mode and a remote mapping mode. The automatic mapping mode is a mapping mode in which the autonomous working machine 30 automatically moves along the boundary of the work area, and the remote mapping mode is a mapping mode in which a user remotely controls the autonomous working machine 30 to move along the boundary.

[0183] Control methods include:

[0184] Step S201: Obtaining initial instructions;

[0185] Step S202: When the initial instruction is an automatic mapping instruction, controlling the machine to move based on the automatic mapping mode;

[0186] Step S203: If a mode switching condition is met, the mapping mode is switched from the automatic mapping mode to the remote mapping mode. The mode switching condition includes detecting that the autonomous working machine 30 is in a slipping condition, a stuck condition, a bumpy condition, a visual feature missing condition, or an image boundary missing condition.

[0187] In some embodiments, before starting to build a map, the machine needs to first determine a starting location for building a map, and start building the map from the starting location. For example, the starting location for building a map can be the location of the charging station 31.

[0188] In some embodiments, when the machine is at the starting position for mapping, the user can provide initial instructions to the machine. The initial instructions represent the mapping mode selected by the user at the beginning of the mapping process. The user can issue initial instructions on the machine's control panel, use voice control, or send initial instructions to the machine through a user terminal, such as a remote control, mobile phone, or computer.

[0189] In some embodiments, if the initial instruction received by the machine is an automatic mapping instruction, the machine is controlled to automatically move along the boundary starting from the mapping starting position, and at the same time, a map of the work area boundary is created based on VSLAM technology. During the process of automatic movement along the edge and mapping, if a special situation (meeting the mode switching conditions) causes the automatic mapping to be impossible, it is necessary to switch the mapping mode and switch the mapping mode to remote control mapping mode. Special situations include abnormal working conditions, such as machine slipping, getting stuck, and bumping; they also include mapping failure conditions, such as the machine detecting that the current visual features are missing and the boundary cannot be detected based on the image.

[0190] In some embodiments, in remote mapping mode, the user can control the movement of the autonomous working machine 30 and overcome or avoid special situations, so that the machine can successfully move along the boundary until the movement path forms a closed loop and the machine creates a boundary map of a single area.

[0191] In some embodiments, when the machine detects based on inertial navigation information or images that the machine wheels are rotating but the machine's position remains unchanged, it can be determined that the machine is in a slipping condition. When the machine detects based on inertial navigation information that the wheels are not rotating as instructed, it can be determined that the machine is in a stuck condition. When the machine detects based on images that the position of the object in front has suddenly changed, it can be determined that the machine is in a bumpy condition. When the machine detects based on images that the number of visual features is small, it can be determined that the machine is in a visual feature missing condition. When the machine detects discontinuous boundaries based on images, it can be determined that the machine is in an image boundary loss condition. Reasons for the image boundary loss condition include that the boundaries in reality are relatively blurred, the lawn at the boundary is sparse grass, etc. In other embodiments, other conventional technical methods can also be used to determine the working condition of the machine.

[0192] Switching the mapping mode from automatic to manual when the mode switching conditions are met can improve mapping efficiency, ensure map accuracy, and enhance the smoothness of subsequent map-based movement of the machine.

[0193] Combine Figure 4 As shown, the boundary 10 includes a boundary 14 near a pond, a curved boundary 15, a boundary near a fence, and a boundary near a tree. Autonomous machine 30 may slip if it moves autonomously near pond boundary 14, may go out of bounds if it moves autonomously near curved boundary 15, may get stuck under the suspended fence 12 if it moves autonomously near a suspended fence, and may collide or get stuck if it moves autonomously near a tree 11.

[0194] Combine Figure 5 As shown, based on the control method provided in this embodiment, after the autonomous machine 30 switches to remote mapping mode, it moves according to the remote mapping mode, with the movement trajectory 60 shown as the dotted line in the figure. It can be seen that under the user's remote control, the movement trajectory of the autonomous machine 30 adapts to the terrain and environment. A map is generated based on the positioning information obtained by the autonomous machine 30 along the movement trajectory 60. Controlling the movement and / or operation of the autonomous machine 30 based on the map can avoid machine damage, collisions, out-of-bounds operations, shutdowns, slippage, jamming, bumps, loss of visual features, loss of boundaries, and other issues, thereby improving mapping efficiency.

[0195] In some embodiments, the control method further includes:

[0196] Step S204: When the initial instruction is a remote control mapping instruction, the machine is controlled to move based on the remote control mapping mode, and mode switching is prohibited.

[0197] If the user instructs the machine to move based on the remote mapping mode at the beginning of the map, the mode switch is prohibited when creating a boundary map of a certain area in the work area; the machine moves at least one circle around the boundary of a certain area in the remote mapping mode to obtain a more accurate map.

[0198] Users can independently determine the mapping mode based on their lawn conditions. For example, if the lawn area is relatively regular and in good condition, users can choose automatic mapping mode at the beginning of mapping, saving manpower and improving comfort. If the lawn area is more complex and the grass is too sparse, users can directly choose remote mapping mode at the beginning of mapping.

[0199] In some embodiments, when the positioning accuracy determined based on the image is less than a first threshold, it is determined that the autonomous working machine 30 is in a visual feature missing condition.

[0200] The first threshold value can be set according to actual needs. If the positioning accuracy is detected to be less than the first threshold value, it can be determined that the machine is in a visual feature missing working condition.

[0201] For an autonomous working machine 30 that builds maps based on VSLAM technology, if the visual positioning accuracy is low, the positioning will be inaccurate and the positioning deviation will be very large, and the map cannot be created automatically. Therefore, if the machine encounters a situation of low positioning accuracy when moving based on the automatic mapping mode, it is necessary to switch to the remote control mapping mode.

[0202] In some embodiments, when the amount of visual feature information determined based on the image is less than a second threshold, it is determined that the autonomous working machine 30 is in a visual feature missing condition.

[0203] The second threshold value can be set according to actual needs. If the amount of visual feature information detected is less than the second threshold value, it can be determined that the machine is in a visual feature missing working state.

[0204] For an autonomous working machine 30 that builds maps based on VSLAM technology, if the amount of visual feature information is small, the positioning will be inaccurate and the positioning deviation will be very large, and the map cannot be created automatically. Therefore, if the machine encounters a situation where the amount of visual feature information is small when moving based on the automatic mapping mode, it is necessary to switch to the remote control mapping mode.

[0205] In some embodiments, when it is detected that the autonomous working machine 30 is in a bumpy condition, a slipping condition, a stuck condition, or a condition where the image boundary is lost, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine 30 is controlled to continue mapping from the current position.

[0206] If the machine encounters bumps, slippage, jams, or image boundary loss while moving in automatic mapping mode, it is necessary to switch to remote mapping mode. In remote mapping mode, mapping can continue based on the map created by automatic mapping, starting from the machine's location when the mode was switched, which improves mapping efficiency.

[0207] When encountering bumps, boundary loss, or obstacles, the machine can slow down and move to provide machine safety.

[0208] In some embodiments, the autonomous working machine 30 is configured to interact with a user terminal, and the user terminal is configured to display remote control controls during the mapping process of the autonomous working machine 30 ;

[0209] When the mode switching conditions are met, the mapping mode is switched from automatic mapping mode to remote mapping mode, including:

[0210] When the mode switching condition is met, a mode switching request is sent to the user terminal, so that the user terminal generates and sends a user remote control instruction according to the user's operation instruction on the remote control control;

[0211] After receiving the user's remote control command, the control machine moves based on the remote control mapping mode and according to the user's remote control command.

[0212] The user terminal can be a device such as a remote controller, mobile phone, or computer. The user can operate the user terminal to send instructions to the autonomous working machine 30. During the mapping process, regardless of the mapping mode, the user terminal can always display the remote control, which is used to control the movement direction or speed of the autonomous working machine 30 during mapping.

[0213] In the automatic mapping mode, the user terminal displays the remote control control. At this time, since the machine automatically builds the map, even if the user clicks or operates the remote control, the user terminal may not respond to the user's operation, thereby preventing the user from interfering with the automatic mapping process.

[0214] In some embodiments, when the mode switching conditions are met, the autonomous working machine 30 sends a mode switching request to the user terminal, the user terminal receives the mode switching request, and displays a warning message. The warning message is used to prompt or suggest the user to switch the mapping mode to the remote control mapping mode, and remind the user that if the remote control mapping mode is switched, the mapping mode cannot be switched to the automatic mapping mode during this mapping process.

[0215] In some embodiments, if the user operates on the user terminal to determine that the mapping mode is switched to the remote control mapping mode, the remote control control is activated, and the user can further click or operate the remote control control. The user terminal responds to the user's operation instructions, and generates and sends user remote control instructions according to the user operation instructions. After the autonomous working machine 30 receives the user remote control instructions, it can control the autonomous working machine 30 to move along the boundary according to the user remote control instructions, thereby enabling the user to use the user terminal to remotely control the autonomous working machine 30 to move along the boundary and build a map.

[0216] In some embodiments, the remote control can be displayed on the user terminal as a thumbnail. When the user clicks the thumbnail, the user terminal automatically enlarges the remote control to display the remote control buttons. Alternatively, during the mapping process, the user terminal always displays the entire remote control, that is, all buttons of the remote control.

[0217] In some embodiments, when it is detected that the autonomous working machine 30 is in a visual feature missing condition or an image boundary missing condition, the mapping mode is switched from the automatic mapping mode to the remote mapping mode, and the autonomous working machine 30 is controlled to re-map.

[0218] If the machine encounters a missing visual feature or image boundary condition while moving in automatic mapping mode, it is necessary to switch to remote mapping mode. Because missing visual features or image boundary conditions can cause large positioning deviations, loss of visual-related information, or large cumulative errors, switching to remote mapping mode requires re-mapping to ensure map accuracy. Re-mapping discards the previously created map and creates a new one.

[0219] In some embodiments, the autonomous working machine 30 is configured to interact with a user terminal;

[0220] Switch the mapping mode from automatic mapping mode to remote mapping mode, including:

[0221] Sending a mode suggestion instruction to the user terminal so that the user terminal displays mode suggestion information, where the mode suggestion information is used to suggest the user to switch the mapping mode to the remote control mapping mode;

[0222] After receiving the user remote control instruction sent by the user terminal, the control machine moves based on the remote control mapping mode and according to the user remote control instruction.

[0223] If the autonomous working machine 30 determines that the mapping mode should be switched to the remote control mapping mode, the autonomous working machine 30 needs to send a mode suggestion instruction to the user terminal, and the user terminal displays mode suggestion information according to the mode suggestion instruction. The mode suggestion information is used to suggest the user to switch the mapping mode to the remote control mapping mode.

[0224] If the user accepts the suggestion, the user will operate the user terminal, and the user terminal will generate a user remote control instruction based on the user operation instruction. After receiving the user remote control instruction sent by the user terminal, the autonomous working machine 30 will move along the boundary and build a map based on the remote control mapping mode and according to the user remote control instruction.

[0225] In some embodiments, controlling the autonomous working machine 30 to remap includes:

[0226] Control the autonomous working machine 30 to return to the starting position of the current mapping;

[0227] The autonomous working machine 30 is controlled to start mapping from a mapping start position.

[0228] If the autonomous working machine 30 encounters a visual feature missing condition during the automatic mapping process, it will cause a large positioning deviation. Therefore, when re-mapping, it is necessary to return to the previous mapping starting position to start mapping. The mapping starting position can be the charging station 31.

[0229] The autonomous machine 30 may send information to the user terminal so that the user terminal prompts the user that the autonomous machine 30 needs to return to the charging station 31 to re-establish a map.

[0230] In some embodiments, controlling the autonomous working machine 30 to remap includes:

[0231] Determine target locations on the boundary;

[0232] The autonomous working machine 30 is controlled to start mapping from a target position.

[0233] Re-mapping can also be done by taking any position on the boundary as the target position and starting to re-map from that position.

[0234] For example, if the autonomous working machine 30 encounters a boundary loss condition during the automatic mapping process, the machine re-starts mapping from the target location.

[0235] In some embodiments, when it is detected that the autonomous working machine 30 is in an image boundary loss state for a preset time period, the autonomous working machine 30 is controlled to find a boundary, and after finding the boundary, the autonomous working machine 30 is controlled to re-build the image;

[0236] If the number of times the autonomous working machine 30 is in the image boundary loss condition is greater than a preset number, the mapping mode is switched from the automatic mapping mode to the remote mapping mode, and the autonomous working machine 30 is controlled to re-map.

[0237] If the autonomous machine 30 encounters a boundary loss condition for the first time during the automatic mapping process, it can determine whether this condition has persisted for a preset period of time. If so, the autonomous machine 30 can automatically move to find the boundary. Once the boundary is found, it re-maps the area using a point on the boundary as the target location, moving and building the map in automatic mapping mode. This process is imperceptible to the user, reducing user interaction and improving user comfort.

[0238] If the autonomous working machine 30 encounters boundary loss conditions multiple times and attempts to re-map multiple times, when the number of times exceeds the preset number, the autonomous working machine 30 determines that the mapping mode needs to be switched from automatic mapping mode to remote mapping mode. The autonomous working machine 30 requests the user to switch modes and re-map.

[0239] In some embodiments, a control method for an autonomous working machine 30 is provided. The method is applied to the autonomous working machine 30 , wherein the autonomous working machine 30 has a camera. The control method includes:

[0240] After the autonomous working machine 30 is powered on for the first time, an image is collected based on the camera;

[0241] determining visual feature information based on the image;

[0242] Without acquiring a work area map, the autonomous working machine 30 is controlled to move and work in the work area based on the visual feature information.

[0243] In some embodiments, while the autonomous working machine 30 is moving and working, a work area map is generated based on visual feature information.

[0244] In some embodiments, if any position of the boundary of the working area to be mapped is safe and the positioning accuracy of the machine at any position of the boundary is high, the recognizability is good, and the boundary is clear, the user can choose not to map. If the user chooses not to map on the user terminal of the autonomous working machine 30 (such as a mobile phone) or the control panel of the autonomous working machine 30, the autonomous working machine 30 will skip the mapping step and directly start working (if it is a lawn mower, the work is mowing) or map during the work process. The user does not need to pay attention to the operation of the machine, which simplifies the user operation.

[0245] In some embodiments, if any position of the boundary of the working area to be mapped is safe and the positioning accuracy of the machine at any position of the boundary is high, the user can choose not to map, and choose not to map on the control terminal of the autonomous working machine 30 (such as a mobile phone) or the control panel of the autonomous working machine 30. The autonomous working machine 30 then starts to move and work automatically. During the automatic movement process, a map can be automatically created and stored without informing the user, and the user does not need to pay attention to the operation of the machine, thereby simplifying user operations.

[0246] After the machine is turned on for the first time, it automatically finds boundaries based on images, moves within and on them, and operates without acquiring a map. During this process, a map is generated in real time based on visual feature information. This entire process is user-intuitive and requires no user interaction, improving convenience and comfort.

[0247] A computer-readable storage medium is also provided. The storage medium stores a computer program, and the computer program is used to execute the above control method.

[0248] In addition, an embodiment is provided, which is an autonomous working machine 30, comprising: a processor; a memory for storing processor-executable instructions; and a processor for executing the above-mentioned control method.

[0249] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0250] The above embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

[0251] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0252] The above embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A control method for an autonomous working machine, wherein the autonomous working machine has a camera, characterized in that: The autonomous working machine has an automatic mapping mode and a remote control mapping mode, wherein the automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of the working area, and the remote control mapping mode is a mapping mode in which the user remotely controls the movement of the autonomous working machine along the boundary; The control method includes: Acquire an image based on the camera; performing image processing on the image to obtain an image processing result; When the autonomous working machine is in the remote-controlled mapping mode and the image processing result indicates that the autonomous working machine is located at a safety boundary position, switching the mapping mode of the autonomous working machine from the remote-controlled mapping mode to the automatic mapping mode, and controlling the autonomous working machine to move based on the automatic mapping mode; When the autonomous working machine is in the automatic mapping mode and the image processing result indicates that the autonomous working machine is located at an unsafe boundary position, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

2. The control method of the autonomous working machine according to claim 1, characterized in that: The unsafe boundary includes one or more of a pool boundary, a curved boundary, a pit boundary, and a narrow area boundary.

3. The control method of the autonomous working machine according to claim 1, wherein: The autonomous working machine is configured to determine visual feature information based on the image captured by the camera, and perform synchronous positioning and map construction based on the visual feature information and the inertial navigation information; The control method further includes: When the autonomous working machine is in the automatic mapping mode and the positioning accuracy of the autonomous working machine is less than a first threshold, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

4. The control method of the autonomous working machine according to claim 1, wherein: The autonomous working machine is configured to determine visual feature information based on the image captured by the camera, and perform synchronous positioning and map construction based on the visual feature information and the inertial navigation information; The control method further includes: When the autonomous working machine is in the remote control mapping mode and the positioning accuracy of the autonomous working machine is not less than a first threshold, the mapping mode of the autonomous working machine is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine is controlled to move based on the automatic mapping mode.

5. The control method of the autonomous working machine according to claim 1, characterized in that: The autonomous working machine is configured to determine visual feature information based on the image captured by the camera, and perform synchronous positioning and map construction based on the visual feature information and the inertial navigation information; The control method further includes: When the autonomous working machine is in the automatic mapping mode and the amount of visual feature information determined based on the image is less than a second threshold, the mapping mode of the autonomous working machine is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to move based on the remote control mapping mode.

6. The control method of the autonomous working machine according to claim 1, wherein: The autonomous working machine is configured to determine visual feature information based on the image captured by the camera, and perform synchronous positioning and map construction based on the visual feature information and the inertial navigation information; The control method further includes: When the autonomous working machine is in the remote control mapping mode and the amount of visual feature information determined based on the image is not less than a second threshold, the mapping mode of the autonomous working machine is switched from the remote control mapping mode to the automatic mapping mode, and the autonomous working machine is controlled to move based on the automatic mapping mode.

7. The control method of the autonomous working machine according to claim 1, wherein: The control method further includes: when the autonomous working machine is in the automatic mapping mode and a dynamic object is detected, switching the mapping mode of the autonomous working machine from the automatic mapping mode to the remote control mapping mode, and controlling the autonomous working machine to move based on the remote control mapping mode.

8. The control method of an autonomous working machine according to any one of claims 1 to 7, characterized in that: Switching the mapping mode of the autonomous working machine from the automatic mapping mode to the remote control mapping mode includes: controlling the autonomous working machine to issue a reminder message; When the autonomous machine receives a remote mapping instruction, the mapping mode of the autonomous machine is switched from the automatic mapping mode to the remote mapping mode.

9. The control method of an autonomous working machine according to any one of claims 1 to 7, characterized in that: Switching the mapping mode of the autonomous working machine from the remote control mapping mode to the automatic mapping mode includes: controlling the autonomous working machine to issue a reminder message; When the autonomous machine receives an automatic mapping instruction, the mapping mode of the autonomous machine is switched from the remote control mapping mode to the automatic mapping mode.

10. A control method for an autonomous working machine, the autonomous working machine having a camera and an inertial navigation module, wherein the camera is configured to acquire an image, and the inertial navigation module is configured to generate inertial navigation information, wherein: The autonomous working machine has an automatic mapping mode and a remote control mapping mode; In the remote control mapping mode or the automatic mapping mode, the autonomous working machine determines visual feature information based on the image, and performs synchronous positioning and map construction based on the visual feature information and the inertial navigation information; The automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of the working area, and the remote control mapping mode is a mapping mode in which the user remotely controls the movement of the autonomous working machine along the boundary; The control method includes: When the autonomous machine is in the automatic mapping mode and the visual feature information determined based on the image does not meet an automatic mapping condition, the mapping mode of the autonomous machine is switched from the automatic mapping mode to the remote control mapping mode.

11. A control method for an autonomous working machine, applied to the autonomous working machine, wherein the autonomous working machine has a camera, characterized in that: The autonomous working machine is configured to generate a work area map based on images captured by the camera during a mapping process; the autonomous working machine has an automatic mapping mode and a remote mapping mode, wherein the automatic mapping mode is a mapping mode in which the autonomous working machine automatically moves along the boundary of the work area, and the remote mapping mode is a mapping mode in which a user remotely controls the movement of the autonomous working machine along the boundary; The control method includes: Get initial instructions; In a case where the initial instruction is an automatic mapping instruction, controlling the machine to move based on the automatic mapping mode; When a mode switching condition is met, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode. The mode switching condition includes detecting that the autonomous working machine is in a slipping condition, a stuck condition, a bumpy condition, a visual feature missing condition, or an image boundary missing condition.

12. The control method of the autonomous working machine according to claim 11, characterized in that: The control method further includes: In a case where the initial instruction is a remote control mapping instruction, the machine is controlled to move based on the remote control mapping mode, and mode switching is prohibited.

13. The control method of the autonomous working machine according to claim 11, characterized in that: When the positioning accuracy determined based on the image is less than a first threshold, it is determined that the autonomous working machine is in the visual feature missing working condition.

14. The control method of the autonomous working machine according to claim 11, characterized in that: When the amount of visual feature information determined based on the image is less than a second threshold, it is determined that the autonomous working machine is in the visual feature missing operating condition.

15. The control method of the autonomous working machine according to claim 11, wherein: When it is detected that the autonomous working machine is in a bumpy working condition, a slipping working condition, a stuck working condition, or an image boundary loss working condition, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to continue mapping from the current position.

16. The control method of the autonomous working machine according to claim 11, wherein: The autonomous working machine is configured to interact with a user terminal, and the user terminal is configured to display remote control controls during a mapping process of the autonomous working machine; When a mode switching condition is met, switching the mapping mode from the automatic mapping mode to the remote control mapping mode includes: When the mode switching condition is met, sending a mode switching request to the user terminal, so that the user terminal generates and sends a user remote control instruction according to the user's operation instruction on the remote control control; After receiving the user remote control instruction, the machine is controlled to move based on the remote control mapping mode and according to the user remote control instruction.

17. The control method of the autonomous working machine according to claim 11, wherein: When it is detected that the autonomous working machine is in a visual feature missing condition or an image boundary missing condition, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to re-map.

18. The control method of the autonomous working machine according to claim 17, characterized in that: The autonomous working machine is configured to interact with a user terminal; Switching the mapping mode from the automatic mapping mode to the remote control mapping mode includes: sending a mode suggestion instruction to the user terminal so that the user terminal displays mode suggestion information, where the mode suggestion information is used to suggest the user to switch the mapping mode to the remote control mapping mode; After receiving the user remote control instruction sent by the user terminal, the machine is controlled to move based on the remote control mapping mode and according to the user remote control instruction.

19. The control method of the autonomous working machine according to claim 17, wherein: Controlling the autonomous working machine to re-map includes: Controlling the autonomous working machine to return to the starting position of the current mapping; The autonomous working machine is controlled to start mapping from the mapping starting position.

20. The control method of the autonomous working machine according to claim 17, wherein: Controlling the autonomous working machine to re-map includes: Determine target locations on the boundary; The autonomous working machine is controlled to start mapping from the target position.

21. The control method of the autonomous working machine according to claim 11, characterized in that: When it is detected that the autonomous working machine is in the image boundary loss condition for a preset time period, controlling the autonomous working machine to find the boundary, and controlling the autonomous working machine to re-build the image after finding the boundary; If the number of times the autonomous working machine is in the image boundary loss condition is greater than a preset number, the mapping mode is switched from the automatic mapping mode to the remote control mapping mode, and the autonomous working machine is controlled to re-map.

22. A control method for an autonomous working machine, applied to the autonomous working machine, wherein the autonomous working machine has a camera, characterized in that: The control method includes: After the autonomous working machine is powered on for the first time, collecting images based on the camera; determining visual feature information based on the image; In the absence of obtaining a work area map, the autonomous working machine is controlled to move and work in the work area according to the visual feature information.

23. The control method of the autonomous working machine according to claim 22, characterized in that: During the process of the autonomous working machine moving and working, the working area map is generated according to the visual feature information.

24. A computer-readable storage medium, characterized in that The storage medium stores a computer program for executing the method according to any one of claims 1 to 23 .

25. An autonomous working machine, characterized in that include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method according to any one of claims 1 to 23.

Images