Control method, storage medium, and autonomous working machine

By using imaging sensors to generate real-time maps in autonomous working machines, the problems of mobility and work efficiency and safety balance in complex environments are solved, efficient obstacle avoidance and path planning are achieved, and sensor system complexity is reduced.

WO2024227434A1PCT designated stage expired Publication Date: 2024-11-07POSITEC TECH CHINA CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/090726
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-08
Filing Date
2024-04-30
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

When existing autonomous working machines move and work in complex environments, it is difficult to efficiently avoid collisions and obstacles, and the sensor system is highly complex, making it difficult to balance movement efficiency and safety.

Method used

Imaging sensors are used to collect work area images, generate real-time dynamic maps, and control machine movement and work based on the map. Through image fusion, coordinate transformation and rasterization processing, obstacles and boundaries are determined and the movement path is optimized.

Benefits of technology

It improves the movement and work efficiency of autonomous working machines, can efficiently detect obstacles and plan obstacle avoidance paths, reduces the risk of collisions, and reduces the complexity of the sensor system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024090726_07112024_PF_FP_ABST
    Figure CN2024090726_07112024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed in the present invention are a control method, a storage medium, and an autonomous working machine. The control method is applied to the autonomous working machine. The autonomous working machine is configured to move and / or work in a working area. The autonomous working machine is provided with an imaging sensor. The method comprises: acquiring a working area image collected by the imaging sensor; generating a real-time map according to the working area image, the real-time map being a dynamically updated map; and on the basis of the real-time map, controlling the autonomous working machine to move and / or work. The present invention can improve the moving efficiency and working efficiency of autonomous working machines, and avoids abnormal conditions such as collision.
Need to check novelty before this filing date? Find Prior Art

Description

A control method, storage medium and autonomous working machine Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and in particular to a control method, a storage medium and an autonomous working machine. Background Art

[0002] As the pace of human life accelerates, autonomous machines are increasingly used around the world, such as automated vacuum cleaners and mops for house cleaning and automated lawn mowers for gardening. Because they operate autonomously without the need for constant user control, they free people from arduous household or work tasks, saving significant energy and time.

[0003] Autonomous machines typically operate within a predefined area. They can typically move randomly within the area using non-localized methods such as boundary line and grass recognition, or plan their movements within the area using methods such as satellite positioning. Autonomous machines also require several sensors to detect obstacles, such as ultrasonic sensors and visual sensors. However, there needs to be a balance between the complexity of autonomous machines and their relative capabilities. Therefore, it is necessary to provide autonomous machines that can operate effectively without overly complex systems and sensors.

[0004] Summary of the Invention

[0005] In order to overcome the defects of the prior art, the problem to be solved by the embodiments of the present disclosure is how to improve the mobility and working efficiency of the autonomous working machine and avoid abnormal situations such as collisions.

[0006] In some embodiments, a control method is provided for an autonomous working machine, the autonomous working machine being configured to move and / or work in a working area, the autonomous working machine being provided with an imaging sensor, the method comprising:

[0007] Acquiring an image of the working area captured by the imaging sensor;

[0008] Generate a real-time map based on the work area image, wherein the real-time map is a dynamically updated map;

[0009] The autonomous working machine is controlled to move and / or work based on the real-time map.

[0010] In some embodiments, the working area image is an image captured when the autonomous working machine moves within a target range;

[0011] The target range is a range centered on the first position or the second position and having a preset size. The first position is the power-on position of the autonomous working machine or the position when the working mode is switched to the target mode. The second position satisfies the preset relative position relationship with the first position, and the adjacent second positions satisfy the preset relative position relationship. The target mode is an obstacle avoidance mode, an escape mode or a narrow channel movement mode.

[0012] In some embodiments, the origin of the coordinate system of the real-time map is a first position or a second position, the first position is the power-on position of the autonomous working machine (101) or the position when the working mode is switched to the target mode, the second position satisfies a preset relative position relationship with the first position, and adjacent second positions satisfy the preset relative position relationship, and the target mode is an obstacle avoidance mode, an escape mode, or a narrow channel movement mode.

[0013] In some embodiments, generating a real-time map based on the work area image includes:

[0014] Acquire non-image sensor information;

[0015] determining heading information corresponding to the working area image based on the non-image sensing information;

[0016] The working area image is fused according to the heading information to generate the real-time map.

[0017] In some embodiments, the non-image sensory information includes inertial navigation information and / or mileage information.

[0018] In some embodiments, performing fusion processing on the work area image according to the heading information to generate the real-time map includes:

[0019] performing coordinate conversion processing on the working area image according to the heading information to generate an image to be combined;

[0020] The images to be combined are combined to generate the real-time map.

[0021] In some embodiments, combining the images to be combined to generate the real-time map includes:

[0022] Determining the coincident coordinate points between the images to be combined;

[0023] Determine the target attributes of the coincident coordinate points;

[0024] The real-time map is generated according to the coincident coordinate points and the target attributes of the coincident coordinate points.

[0025] In some embodiments, performing coordinate conversion processing on the working area image according to the heading information to generate the image to be combined includes:

[0026] Determine the target coordinate system;

[0027] determining a coordinate conversion relationship between a coordinate system of the working area image and the target coordinate system according to the target coordinate system and the heading information;

[0028] According to the coordinate transformation relationship, coordinate transformation processing is performed on the working area image to generate the image to be combined.

[0029] In some embodiments, determining a target coordinate system includes:

[0030] determining a latest image among the work area images;

[0031] Using the coordinate system corresponding to the latest image as the target coordinate system;

[0032] Alternatively, determining a preset coordinate system corresponding to the preset position and the preset direction;

[0033] The preset coordinate system is used as the target coordinate system.

[0034] In some embodiments, further comprising:

[0035] If, after the combination processing, there are map images in the real-time map that are beyond the preset map range, the map images that are beyond the preset map range are deleted.

[0036] In some embodiments, the size of the real-time map is determined according to time, the location of the autonomous working machine, the working mode of the autonomous working machine, and / or the surrounding environment of the autonomous working machine.

[0037] In some embodiments, before performing coordinate conversion processing on the working area image based on the heading information to generate the image to be combined, the method further includes:

[0038] performing image segmentation processing on the working area image to determine a grass area and a non-grass area in the working area image, wherein the non-grass area has at least one type;

[0039] Determine the grid confidence based on the correspondence between the grid and the image pixels and the number of image pixels;

[0040] According to the grid confidence, the working area image after image segmentation is subjected to rasterization processing to generate a rasterized working area image.

[0041] In some embodiments, the grid confidence is equal to a ratio of a first number to a second number, where the first number is the number of non-grass pixels in the image pixels corresponding to the grid, and the second number is the total number of image pixels corresponding to the grid.

[0042] In some embodiments, before performing coordinate conversion processing on the working area image based on the heading information to generate the image to be combined, the method further includes:

[0043] The gridded working area image is corrected according to the grid confidence and / or grid position to generate a corrected working area image.

[0044] In some embodiments, before performing image segmentation processing on the working area image to determine the grass area and the non-grass area in the working area image, the method further includes:

[0045] The distorted region image in the working region image is eliminated, where the distorted region image is a region image whose image distortion amount is greater than a preset distortion value.

[0046] In some embodiments, the real-time map is a map that is dynamically updated according to time, the location of the autonomous working machine, and / or the surrounding environment of the autonomous working machine.

[0047] In some implementations, the real-time map is a map of a portion of the work area.

[0048] In some embodiments, the origin of the coordinate system of the real-time map corresponds to the origin of the coordinate system corresponding to the current image;

[0049] Alternatively, the origin of the coordinate system of the real-time map corresponds to a preset position;

[0050] Alternatively, the origin of the coordinate system of the real-time map corresponds to the current position of the autonomous working machine.

[0051] In some embodiments, the real-time map includes at least one map area, and an area type of the at least one map area includes one or more of a drivable area, a non-drivable area, a boundary area, and an obstacle area.

[0052] In some embodiments, the number of the work area images is determined based on a work mode and / or a movement state of the autonomous working machine.

[0053] In some embodiments, the real-time map is deleted when preset conditions are met, and the preset conditions include at least one of the following conditions: the posture change between consecutive images is greater than a change threshold, the heading accuracy of the autonomous working machine is less than a preset accuracy, the autonomous working machine detects a collision, the autonomous working machine detects an uphill slope, and the autonomous working machine detects a downhill slope.

[0054] In some embodiments, the working area image is an image captured by the autonomous working machine within a target time period;

[0055] The target time period is an adjustable time period, and the real-time map is updated based on the adjustment of the target time period.

[0056] In some embodiments, the working area image includes a current image and a historical image, and the acquisition time of the current image is later than the acquisition time of any other image acquired by the imaging sensor.

[0057] In some embodiments, a control device is provided for an autonomous working machine, wherein the autonomous working machine is configured to move and / or work in a working area, and the autonomous working machine is provided with an imaging sensor. The control device includes:

[0058] an acquisition module, configured to acquire an image of the working area captured by the imaging sensor;

[0059] A map generation module is configured to generate a real-time map based on the work area image, wherein the real-time map is a dynamically updated map;

[0060] A control module is configured to control the autonomous working machine to move and / or work based on the real-time map.

[0061] In some embodiments, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above method.

[0062] In some embodiments, an autonomous working machine is provided, comprising:

[0063] processor;

[0064] a memory for storing instructions executable by the processor;

[0065] The processor is configured to execute the above method.

[0066] The embodiments of the present disclosure can improve the mobility and working efficiency of autonomous working machines.

[0067] The autonomous machine can move and / or operate based on a real-time map. For example, it can detect obstacles and plan obstacle avoidance paths based on the real-time map, detect boundaries and plan paths along them based on the real-time map, and detect people or animals based on the real-time map. Thus, the autonomous machine in this embodiment can efficiently move and operate based on a local map, improving movement and operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0069] FIG1 is a schematic structural diagram of an autonomous working machine provided by some embodiments of the present application;

[0070] FIG2 is a module diagram of some embodiments of the present application;

[0071] FIG3 is a schematic diagram of an autonomous working machine according to some embodiments of the present application when encountering an obstacle;

[0072] FIG4 a is a schematic diagram illustrating the relationship between an autonomous working machine and a range represented by a real-time map according to some embodiments of the present application;

[0073] FIG4 b is a schematic diagram showing the relationship between an autonomous working machine and a range represented by a real-time map, provided in some other embodiments of the present application;

[0074] FIG5 is a flow chart of a control method provided in some embodiments of the present application;

[0075] FIG6 is a schematic diagram of a process for generating a real-time image based on a work area image in a control method provided in some embodiments of the present application;

[0076] [Corrected 16.05.2024 in accordance with Rule 91] Figure 7 is a schematic diagram of a control device provided in some embodiments of the present application; [0076.1][Corrected 16.05.2024 according to Rule 91] Figure 8 is a schematic diagram of the connection relationship between the memory and the processor in the autonomous working machine provided in some embodiments of the present application. DETAILED DESCRIPTION

[0077] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the accompanying drawings. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure. The embodiments provided in this specification may be combined with each other.

[0078] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0079] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0081] The embodiments provide an autonomous working machine, a control method, and a computer-readable storage medium. The autonomous working machine 101 may be an autonomously moving intelligent device, such as an autonomous lawn mower, an autonomous vacuum cleaner, an autonomous mop, or an autonomous snow blower. The autonomous machine can automatically move within a specified working area and perform corresponding tasks, or it can return to a docking station 102 along the boundary of the working area for docking or charging.

[0082] This embodiment provides an autonomous working machine 101 . As shown in FIG. 1 and FIG. 2 , the autonomous working machine 101 includes a body 100 , an imaging sensor 200 , a position sensor 300 , and a control circuit 600 .

[0083] Specifically, the body 100 includes a drive device 700, which is used to drive the body 100 on the work surface according to received drive commands. The drive device 700 generally includes rollers and a motor that drives the rollers. The rollers may include a driving wheel and a driven wheel. The rollers may be distributed on both sides of the body 100, with one or two rollers on each side.

[0084] The body 100 also includes a working module, which is used to perform specific work tasks. For example, if the autonomous working machine 101 is an automatic lawn mower, the working module includes mowing blades, a cutting motor, and other components. It may also include auxiliary components such as a mowing height adjustment mechanism to optimize or adjust the mowing effect. For example, if the autonomous working machine 101 is an automatic vacuum cleaner, the working module includes a vacuum motor, a vacuum port, a vacuum tube, a vacuum chamber, a dust collection device, and other working components used to perform the vacuuming task.

[0085] The body 100 may also include an energy module, which is used to provide energy for various operations of the autonomous working machine 101. The energy module may include a rechargeable battery and a charging connection structure, wherein the charging connection structure is usually a charging electrode sheet, which can be used in conjunction with a charging electrode sheet set at the docking station to charge the autonomous working machine 101.

[0086] The body 100 further includes a memory 400 , which is used to store data generated by sensors or control circuits, or to pre-store data for use by the control circuits.

[0087] The body 100 further includes a position sensor 500 , which may include an inertial sensor IMU or an odometer ODO installed on the driving device 700 , etc., for obtaining a relative position according to the movement of the body 100 .

[0088] In addition to the above-mentioned modules, the body 100 may also include a shell for accommodating and installing each module, a control panel for user operation, etc. It may also include various environmental sensors, such as humidity sensors, temperature sensors, acceleration sensors, light sensors, etc. The above-mentioned sensors can assist the autonomous working machine 101 in judging the working environment in order to execute the corresponding program.

[0089] The control circuit 600 is the core component of the autonomous working machine 101, which is used to control the automatic movement and work of the autonomous working machine 101. The functions it performs include controlling the working module to start or stop working, generating a moving path and controlling the drive device 700 to move according to the path, judging the power of the energy module and timely controlling the autonomous working machine 101 to return to the docking station for automatic docking and charging, and executing corresponding programs in combination with the data from the environmental sensor.

[0090] 2 and 3 , the autonomous working machine 101 includes an imaging sensor 200 , which is connected to the machine body 100 and is used to capture images in the forward direction of the machine body 100. These images, at least in part, capture images of the working surface in the forward direction. The captured images are within the field of view 210 of the imaging sensor 200 . The imaging sensor 200 can be a commonly used camera or lidar.

[0091] Typically, the imaging sensor 200 is mounted near the front upper portion of the fuselage 100, preferably centered, with its viewing angle facing forward and downward to capture images of the work surface. Its field of view 210 can be adjusted based on actual needs. A larger field of view 210 captures more images in the forward direction of the fuselage 100, and vice versa. The forward direction of the fuselage 100 can be various, such as normal forward movement, backward movement, and turning. In this embodiment, the forward direction of the fuselage refers to the normal forward direction, i.e., the direction of the fuselage's central axis.

[0092] To address the technical problem of improving the movement and work efficiency of an autonomous working machine 101, this embodiment provides a control method applied to the autonomous working machine 101. The autonomous working machine 101 is configured to move and / or work within a work area. The autonomous working machine 101 is equipped with an imaging sensor 200, which may be a camera located on the front of the machine body. The camera can capture images of the work area. The autonomous working machine 101 can move and / or work within or along the boundaries of the work area.

[0093] With reference to FIG5 , in some embodiments, the control method includes:

[0094] Step S101: Acquire an image of the working area captured by the imaging sensor 200;

[0095] Step S102: Generate a real-time map based on the work area image, where the real-time map is a dynamically updated map;

[0096] Step S103 : Controlling the autonomous working machine 101 to move and / or work based on the real-time map.

[0097] In some embodiments, the imaging sensor 200 may capture images while the autonomous working machine 101 is moving. The work area images may be a portion of all images captured by the imaging sensor 200. For example, the work area images are images captured when the autonomous working machine 101 is located at a certain position, or images captured at a certain point in time or during a certain period of time, or images with a certain correlation.

[0098] In some embodiments, the work area images may have certain correlations. For example, the work area images may be captured at similar times, at similar locations, at even intervals, with a total data volume less than a specific value, or with the corresponding pose changes meeting certain conditions.

[0099] In some embodiments, starting with the latest captured image, a portion of the historical images is filtered from all historical images, and the latest image and the filtered portion of the historical images are used as the work area images. The filtering method can be based on timestamps, shooting locations, pose change amplitude, etc. The time intervals between adjacent timestamps can be uniform or set according to a certain pattern.

[0100] In some embodiments, the screening method can be based on the magnitude of the posture change between images. For example, the latest current image and the first historical image are obtained, the timestamp of the first historical image is earlier than the timestamp of the current image, and the posture change of the first historical image compared to the current image is determined; if the posture change is less than a first threshold, it means that the posture change of the machine when shooting the current image and the first historical image is too small, or even remains stationary, and the first historical image cannot be used as the working area image; if the posture change is greater than the first threshold, the first historical image can be used as the working area image, and further, a second historical image can be obtained, the timestamp of the second historical image is earlier than the timestamp of the first historical image, and the posture change of the second historical image compared to the current image is determined. If the posture change is greater than the first threshold and less than or equal to the second threshold, the second historical image can be used as the working area image; if the posture change of the second historical image compared to the current image is greater than the second threshold, it means that the posture change is too large, and the second historical image cannot be used as the working area image.

[0101] In some embodiments, since the work area image can be continuously updated with the movement position of the machine, the posture of the machine, the change of time or the change of the environment, the real-time map can be dynamically updated accordingly.

[0102] In some embodiments, the real-time map is a map of a portion of the work area.

[0103] A real-time map can represent an area within a certain range, including grass, non-grass, boundaries, or obstacles within a certain range around the autonomous working machine 101. The range and resolution of the real-time map can be adjusted according to actual needs. For example, the range of the real-time map can be 5 meters by 5 meters, with a resolution of 5 cm; or the range can be 4 meters by 4 meters; or the range can be 20 meters by 20 meters. The real-time map can clearly and completely display the characteristic information of the area, such as one or more of the following: location, coordinates, shape, area, and type.

[0104] In the representation range of the real-time map, the autonomous working machine 101 may be located at the center of the range or at another location within the range.

[0105] In some embodiments, the autonomous working machine 101 can move and / or operate based on a real-time map. For example, it can detect obstacles and plan obstacle avoidance paths based on the real-time map, detect boundaries and plan paths along the edges based on the real-time map, and detect people or animals based on the real-time map. Thus, the autonomous working machine 101 in this embodiment can efficiently move and operate based on a local map, thereby improving movement and operation efficiency.

[0106] As shown in FIG3 , the autonomous working machine 101 can collect images in real time during movement, and when encountering obstacles a1 and a2, it can detect them based on the images; when the autonomous working machine 101 is in posture B1, it can collect image c1, and the edges e1 and e2 of obstacle a1 appear in image c1; the autonomous working machine 101 moves from posture B1 to posture B2; the autonomous working machine 101 is in posture B2, and it can collect image c2, and the edges e2 and e3 of obstacle a1 and the edge e4 of obstacle a2 appear in image c2; it can be seen that when the autonomous working machine 101 is in posture B2, the edge e1 cannot be detected in the image; and in this embodiment, the autonomous working machine 101 1 can fuse images c1 and c2 to produce a real-time map, which includes the location information of edges e1, e2, e3, and e4. Thus, even when the autonomous machine 101 is in position B2, relevant information about edge e1 is not lost. The autonomous machine 101 can plan its movement path based on more comprehensive information about its surroundings, enabling advance and efficient planning, improving movement efficiency and accuracy, and avoiding collisions. For example, when the autonomous machine 101 is in position B2, the real-time map can be used to determine how to most effectively avoid obstacles. By comparing different obstacle avoidance schemes (obstacle avoidance from the right side of e1 and obstacle avoidance from the left side of e4), the more efficient obstacle avoidance scheme can be implemented. The fusion of two images in Figure 3 is for illustrative purposes only. In actual fusion processing, the number of images and the corresponding positional changes of the different images to be fused can be adjusted according to actual needs.

[0107] In some embodiments, the control method also includes: deleting the real-time map when preset conditions are met, the preset conditions including at least one of the following conditions: the posture change between consecutive images is greater than the change threshold, the heading accuracy of the autonomous working machine 101 is less than the preset accuracy, the autonomous working machine 101 detects a collision, the autonomous working machine 101 detects an uphill slope, and the autonomous working machine 101 detects a downhill slope.

[0108] For example, when the autonomous working machine 101 encounters bumpy road conditions, it will cause a large change in posture between the two frames of images taken by the autonomous working machine 101, or will cause the heading accuracy to decrease. At this time, the accuracy of the real-time map is greatly reduced. Therefore, the real-time map can be deleted and re-established when the autonomous working machine 101 can move stably.

[0109] When the autonomous working machine 101 collides or is in the process of going up or down a slope, the accuracy of the real-time map will be greatly reduced. The real-time map can be deleted and re-established when the autonomous working machine 101 can move stably.

[0110] In some embodiments, the size of the real-time map is determined based on the time, the location of the autonomous working machine 101, the working mode of the autonomous working machine 101 and / or the surrounding environment of the autonomous working machine 101. The size of the real-time map can be adjusted according to actual needs. For example, the vision is clearer during the day and unclear at night, so the image is more accurate during the day and a real-time map with a larger range can be obtained, while the range of the real-time map at night is smaller. For another example, if the environment where the machine is located is more complex and there are more obstacles, or the machine enters the obstacle avoidance mode, a larger real-time map needs to be used for the safety of the machine or obstacle avoidance; if the environment where the machine is located is relatively safe, the size of the real-time map can be smaller.

[0111] In some embodiments, the autonomous working machine 101 can generate two real-time maps of different sizes at the same time, and call different real-time maps based on different trigger conditions; in some cases, different real-time maps can be called at the same time, and after judging and planning the path based on the different real-time maps, the movement and work of the autonomous working machine 101 are more intelligent; in some cases, only one real-time map is used, which can reduce the use of computing power and improve processing speed. For example, when the autonomous working machine 101 moves based on normal mode, a smaller real-time map (such as 4 meters × 4 meters) is called, and when the autonomous working machine 101 moves based on escape mode / obstacle avoidance mode / narrow channel movement mode, a smaller real-time map and a larger real-time map (such as 20 meters × 20 meters) are called at the same time. Normal mode, escape mode, obstacle avoidance mode, and narrow channel movement mode can be automatically switched by the autonomous working machine 101 without user intervention.

[0112] In some embodiments, the work area image is an image captured by the autonomous work machine 101 within a target time period.

[0113] The imaging sensor 200 can capture images while the autonomous working machine 101 is moving. Each image is timestamped, indicating the time when the image was captured. The target time period can be a period of time prior to the current time point, where the current time point is the latest real-time point. Based on the image timestamp information, images captured within the target time period (i.e., images of the working area) can be filtered from all images captured by the imaging sensor 200.

[0114] In some embodiments, the target time period is an adjustable time period, and the real-time map is updated based on the adjustment of the target time period. The target time period is adjustable. For example, the time period parameter can be set by the user or automatically adjusted by the autonomous working machine 101 based on movement and working conditions.

[0115] In some embodiments, the work area image includes a current image and historical images, and the acquisition time of the current image is later than the acquisition time of any other image acquired by the imaging sensor 200. The current image is an image acquired at the latest time point.

[0116] For example, when generating a smaller real-time map (e.g., 4 meters x 4 meters), first obtain images collected at the current time point and within the previous minute. From these images, one image is taken at evenly spaced intervals (e.g., one image every 5 seconds) and used as the work area image. This approach ensures that the real-time map accurately represents the environment surrounding the autonomous working machine 101, ensures a faster update frequency, and improves the map's accuracy and real-time performance.

[0117] As shown in Figure 4a, a working area 10 is formed by enclosing a boundary, and the autonomous working machine 101 can move and / or work within the working area, and can also move and / or work along the boundary; when the autonomous working machine 101 is located at position D1, the representation range of the real-time map can be as shown in range 61; when the autonomous working machine 101 is located at position D2, the representation range of the real-time map can be as shown in range 62; when the autonomous working machine 101 is located at position D3, the representation range of the real-time map can be as shown in range 63; that is, the real-time map can be dynamically updated according to the position of the autonomous working machine 101.

[0118] In some embodiments, the origin of the coordinate system of the real-time map corresponds to the origin of the coordinate system corresponding to the current image. The current image is the latest image and can reflect the actual location of the autonomous working machine 101.

[0119] The origin of the coordinate system of a smaller-sized real-time map (such as 4 meters × 4 meters) is the origin of the coordinate system corresponding to the current image.

[0120] In some embodiments, the working area image is an image captured when the autonomous working machine 101 moves within the target range; the target range is a range centered on the first position or the second position and having a preset size, the first position is the power-on position of the autonomous working machine 101 or the position when the working mode is switched to the target mode, a preset relative position relationship is satisfied between the second position and the first position, and a preset relative position relationship is satisfied between adjacent second positions, and the target mode is an obstacle avoidance mode, an escape mode, or a narrow channel movement mode.

[0121] Exemplarily, the preset relative position relationship is that the distance between the two positions is greater than a distance threshold.

[0122] In target mode, a larger map size is required because a larger map size has more and more comprehensive environmental information, which facilitates the autonomous working machine 101 to plan more accurate movement strategies when avoiding obstacles, getting out of trouble, or in narrow passages.

[0123] In target mode, in order to improve the intelligence of the autonomous working machine 101 and better avoid obstacles, escape from trouble or quickly pass through narrow passages, the real-time map should not only reflect the small environment around the autonomous working machine 101, but also the environment within the range where the autonomous working machine 101 has moved.

[0124] For example, a larger real-time map (such as 20 meters × 20 meters) is created when the machine is turned on. The autonomous working machine 101 moves forward 10 meters from the position at the time of power-on. The images taken within this range can be used as working area images and used to generate a real-time map, or multiple images can be selected from these images as working area images and used to generate a real-time map; after the distance between the position of the autonomous working machine 101 and the power-on position exceeds 10 meters, the images taken within a range of 20 meters × 20 meters centered on the current position are used as working area images and used to generate a real-time map; after the autonomous working machine 101 continues to move forward more than 10 meters, the target range is re-determined, and so on.

[0125] It can be seen that in this way, the environmental range represented by the real-time map will only change when the conditions are met. If the autonomous working machine 101 is trapped in an area, using this method can avoid losing valuable environmental information, making it easier for the autonomous working machine 101 to plan more accurate movement strategies when avoiding obstacles, escaping or in narrow passages.

[0126] The larger-sized real-time map is created when the machine is turned on, and the larger-sized real-time map is called after the autonomous machine 101 enters the target mode; or the larger-sized real-time map is created only when the autonomous machine 101 enters the target mode.

[0127] As shown in Figure 4b, in the working area 10 formed by the boundary, the autonomous working machine 101 can move and / or work within the working area, and can also move and / or work along the boundary; when the autonomous working machine 101 is located at position D4 and position D5, the representation range of the real-time map can be as shown in range 64; when the autonomous working machine 101 is located at position D6 and position D7, the representation range of the real-time map can be as shown in range 65.

[0128] In some embodiments, the origin of the coordinate system of the real-time map is the first position or the second position, the first position is the power-on position of the autonomous working machine 101 or the position when the working mode is switched to the target mode, a second position and the first position satisfy a preset relative position relationship, adjacent second positions satisfy a preset relative position relationship, and the target mode is an obstacle avoidance mode, an escape mode, or a narrow channel movement mode.

[0129] In order to avoid the process of creating a real-time map too frequently, the coordinate system used by the autonomous working machine 101 as it moves within the target range may remain unchanged.

[0130] As shown in FIG6 , in some embodiments, step S102 includes:

[0131] Step S1021: Acquire non-image sensing information;

[0132] Step S1022: determining heading information corresponding to the working area image based on the non-image sensing information;

[0133] Step S1023: Based on the heading information, the working area image is fused and processed to generate a real-time map.

[0134] In some embodiments, the autonomous machine 101 is equipped with a high-precision heading sensor. Based on this high-precision heading sensor, the machine's heading, position change amplitude, and other parameters can be determined more accurately and efficiently. For example, the heading sensor can include one or more of an inertial measurement unit (IMU), an odograph (ODO), a high-precision kinematic positioner (RTK), and a ranging radar.

[0135] In some embodiments, non-image sensor information includes inertial navigation information and / or mileage information. Heading information corresponding to each work area image can be determined based on the inertial navigation information and / or mileage information. The heading information may include the position, posture, and / or movement direction of the machine when the image was captured.

[0136] In some embodiments, based on the heading information corresponding to each work area image, the work area images are effectively spliced ​​and fused to obtain a real-time map.

[0137] Due to the use of high-precision heading sensors, the heading information is more accurate, and more accurate posture change information and real-time maps can be obtained to ensure that the autonomous working machine 101 moves and / or works accurately; and the data processing volume is small and the calculation is fast.

[0138] In some embodiments, step S1023 includes:

[0139] According to the heading information, coordinate transformation is performed on the working area image to generate the image to be combined;

[0140] The images to be combined are combined and processed to generate a real-time map.

[0141] Based on the image acquisition module's internal and external parameters, distortion parameters, and other information, the work area image can be analyzed to determine the relative position of each object and location in the image relative to the image acquisition module (camera). This allows the coordinate information of each point in the image to be obtained in a coordinate system established based on the camera's pose during capture. Because the camera's pose is different when capturing each work area image, the coordinate system corresponding to each work area image is different.

[0142] In some embodiments, the work area image can be subjected to coordinate conversion processing, and the work area image can be converted into the same coordinate system to obtain the image to be combined. That is to say, the image coordinate systems corresponding to the images to be combined are the same, which facilitates the subsequent combination steps of the images and facilitates the generation of accurate real-time maps.

[0143] Based on the heading information of each work area image, the relative position relationship between any two work area images can be determined. Consequently, the transformation relationship between the coordinate systems of any two work area images can be determined based on the heading information. The transformation relationship between the coordinate system of any work area image relative to a specific position coordinate system can also be determined based on the heading information. Due to the high accuracy of the heading information, the resulting transformation relationship is more precise, facilitating the generation of high-precision real-time maps.

[0144] In some embodiments, performing coordinate conversion processing on the working area image to generate the image to be combined includes:

[0145] Determine the target coordinate system;

[0146] Determine the coordinate transformation relationship between the coordinate system of the working area image and the target coordinate system according to the target coordinate system and heading information;

[0147] According to the coordinate transformation relationship, the working area image is subjected to coordinate transformation processing to generate the image to be combined.

[0148] In some embodiments, determining a target coordinate system includes:

[0149] Determine the latest image in the work area image;

[0150] The coordinate system corresponding to the latest image is used as the target coordinate system.

[0151] In some embodiments, the target coordinate system can be the coordinate system information corresponding to any image in the work area image, for example, the coordinate system information corresponding to the current latest image. Since the coordinate system information of the current latest image is related to the current position of the machine, using the coordinate system of the current latest image as the coordinate system of the real-time map can make the coordinate system of the real-time map match the current position of the machine, more accurately reflecting the conditions of the area around the machine. Since the origin of the map coordinate system is related to the current position of the machine, it is more convenient and accurate when performing path planning, without the need for other complex coordinate conversions, and facilitates path planning based on the real-time map.

[0152] As shown in Figure 3, when fusing image c1 and image c2, the coordinate system S2 corresponding to image c2 can be used as the target coordinate system; the heading information of the machine when it is in posture B1 and the heading information of the machine when it is in posture B2 are determined based on the heading sensor, and then the relative posture relationship between posture B1 and posture B2 is determined; since the coordinate system S1 of image c1 is established based on posture B1, and the coordinate system S2 of image c2 is established based on posture B2, the coordinate transformation relationship when converting image c1 to the coordinate system S2 of image c2 can be determined based on the relative posture relationship between posture B1 and posture B2, and the image c1 is converted to the coordinate system S2 of image c2 based on the coordinate transformation relationship, thereby placing image c1 and image c2 in the same coordinate system (S2), so as to facilitate the fusion of the coordinate information of image c1 and the coordinate information of image c2. For example, the relative posture relationship is determined based on IMU and ODO. Since the information accuracy of IMU and ODO is high, the relative posture relationship is more accurate, and the obtained coordinate transformation relationship is also more accurate, making the real-time map more accurate, complete, and clear, which is convenient for path planning; and the calculation is easy and the cost is low.

[0153] In some embodiments, determining a target coordinate system includes:

[0154] Determine a preset coordinate system corresponding to a preset position and a preset direction;

[0155] Use the preset coordinate system as the target coordinate system.

[0156] In some embodiments, the target coordinate system can be the coordinate system information corresponding to a specific position and a specific direction in the working area. For example, the target coordinate system can be established with the charging station position as the origin, the north as the Y-axis, and the east as the X-axis; or, the cross-region transition position, the corner position in the working area, the starting position of the work, etc. can be used as the origin.

[0157] In order to ensure the accuracy of the real-time map, the amplitude of the change between the working area images should generally not be too large. For example, two working area images can capture the same obstacle. The difference is that the two images are taken based on different camera postures. Therefore, in the two working area images, the same obstacle appears in different postures. However, in the real-time map, the same obstacle or the same location point can only appear once. For example, after the coordinate system of the working area image is converted to the same coordinate system, coordinate A appears repeatedly in multiple images. Due to errors, distortions and other reasons, coordinate A may represent different objects. For example, in the first image, coordinate A represents grass, and in the second image, coordinate A represents non-grass. However, in the real-time map, coordinate A can only represent one of grass and non-grass. Therefore, it is necessary to solve the problem of how to determine the attributes of overlapping coordinate points.

[0158] In some embodiments, combining the images to be combined to generate a real-time map includes:

[0159] Determine the coincident coordinate points between the images to be combined;

[0160] Determine the target attributes of the coincident coordinate points;

[0161] A real-time map is generated based on the coincident coordinate points and the target attributes of the coincident coordinate points.

[0162] In some embodiments, for overlapping coordinate points that appear repeatedly in multiple images to be combined, it is necessary to determine their attribute probabilities based on the attributes of the overlapping coordinate points in multiple images, determine the target attributes based on the attribute probabilities, and use the target attributes as the attributes of the overlapping coordinate points in the real-time map.

[0163] In some embodiments, the number of images in which the overlapping coordinate point has the attribute of non-grass can be determined based on the attributes of the overlapping coordinate point in each image. If the number is greater than a preset value, it means that the overlapping coordinate point is marked as non-grass in more images, and the probability that the overlapping coordinate point is actually non-grass is relatively high. In this case, the target attribute of the overlapping coordinate point can be determined as non-grass, and the attribute of the coordinate point can be marked as non-grass in the real-time map.

[0164] In some embodiments, if the target attribute of the coincident coordinate point is not non-grass, the attribute of the coordinate point may be marked as grass in the real-time map.

[0165] In some embodiments, for the coordinate points in the representation range of the real-time map, after determining the attributes of the coordinate points one by one and marking them, a complete real-time map can be obtained, and the real-time map can reflect the grass and non-grass in the representation range.

[0166] In some embodiments, for non-overlapping coordinate points, i.e., those that appear in only one image, these non-overlapping coordinate points can be retained and their attributes in the image can be trusted. Because generating a real-time map requires a large number of work area images and the pose variations between images are small, typically only a very small number of coordinate points appear in only one image, and their impact on the accuracy of the real-time map is minimal.

[0167] In some embodiments, for non-coincident coordinate points, the coordinate points may also be discarded.

[0168] In some embodiments, if there is a map image beyond the preset map range after the combination process, the map image beyond the preset map range is deleted.

[0169] For example, if a real-time map can represent an area within a range of 5 meters * 5 meters, and the origin of the coordinate system is located at the center of the map, then the real-time map can only reflect objects within the range of [-2.5 meters, 2.5 meters] in the horizontal direction at most; during the combination process, if the coordinate point in the image to be combined is (3 meters, 1 meter), then the (3 meters, 1 meter) coordinate point exceeds the representation range of the real-time map, and the (3 meters, 1 meter) coordinate point can be discarded and does not need to be reflected on the real-time map.

[0170] In some embodiments, before performing coordinate conversion processing on the working area image based on the heading information to generate the image to be combined, the method further includes:

[0171] Performing image segmentation processing on the working area image to determine grass areas and non-grass areas in the working area image, wherein the number of types of the non-grass areas is at least one;

[0172] Determine the grid confidence based on the correspondence between the grid and the image pixels and the number of image pixels;

[0173] According to the grid confidence, the working area image after image segmentation is rasterized to generate a rasterized working area image.

[0174] Image recognition can be performed on the work area image to determine multiple types, which are mainly divided into grass and non-grass. Non-grass includes one or more of stone slabs, soil, fallen leaves, fallen flowers, ditches, fences, tall grass, sprinkler heads, manhole covers and obstacles. Non-grass can also include other types, but this embodiment is not limited to this.

[0175] In some embodiments, the working area image may be subjected to image segmentation processing to determine the grass area and the non-grass area; and the working area image after the image segmentation processing may be subjected to rasterization processing.

[0176] The rasterization process may include: obtaining an initialized raster image, where the initialized raster image is a raster image in which the grids are not marked; marking the grids in the initialized raster image one by one according to the working area image after image segmentation processing, and obtaining a rasterized working area image after marking is completed.

[0177] The grids in the initialization grid map can be marked one by one based on the grid confidence.

[0178] The grid confidence represents the probability that the location corresponding to the grid is not grass.

[0179] In some embodiments, the grid confidence is equal to a ratio of a first number to a second number, where the first number is the number of non-grass pixels in the image pixels corresponding to the grid, and the second number is the total number of image pixels corresponding to the grid.

[0180] Based on the grid confidence, grids in the initialization grid image can be marked one by one. For example, the number of non-grass pixels (a first number) when the grid position is mapped in the working area image is determined, the total number of image pixels (a second number) when the grid position is mapped in the working area image is determined, the ratio of the first number to the second number is determined, and the grid confidence is determined based on the ratio.

[0181] If the grid confidence is high, the grid can be marked as a non-grass grid; if the grid confidence is low, the grid can be marked as a grass grid.

[0182] In some embodiments, before performing coordinate conversion processing on the working area image based on the heading information to generate the image to be combined, the method further includes:

[0183] The rasterized working area image is corrected according to the grid confidence and / or grid position to generate a corrected working area image.

[0184] In some embodiments, if a non-grass grid is located at an edge of a non-grass grid area, the non-grass grid at the edge may be corrected and marked as a grass grid.

[0185] In some embodiments, if the position of a non-grass grid is at the edge of a non-grass grid area, the grid can be corrected based on the grid confidence of the non-grass grid located at the edge. For example, if the grid confidence is high, no correction is performed; if the grid confidence is low, the grid is corrected and marked as a grass grid.

[0186] Correcting the non-grass grid located at the edge of the non-grass grid area can more accurately reflect the boundary between grass and non-grass, ensure that the boundary position in the grid map is more consistent with the actual boundary position, and thus make the boundary between grass and non-grass in the real-time map more consistent with the actual boundary, avoid the moving path of the autonomous working machine 101 from being too far away from the boundary, avoid missed cutting, and improve the cutting coverage rate.

[0187] In some embodiments, if the non-grass grid is surrounded by grass grids, that is, the non-grass grid is an isolated non-grass grid, the grid is likely to be incorrectly marked, and the non-grass grid is corrected and marked as a grass grid.

[0188] In some embodiments, for the rasterized working area image, it is necessary to perform a connected domain judgment on the obstacle grid. If several obstacle grids are connected to each other, these obstacle grids will be merged into an obstacle connected domain. Then determine how many such obstacle connected domains there are in a single raster image. After determining the connected domain, a loop is performed to determine whether each connected domain is close to the edge of the raster image. If it is close, it is a boundary, otherwise it is an obstacle. The purpose of distinguishing the connected domain into obstacles and boundaries is to execute different filtering strategies for obstacles and boundaries when processing a single raster image, and to filter out obstacles and boundaries that do not meet the conditions. Specific filtering conditions can be set according to actual conditions. For example, the connected domain marked as a boundary is corrected and marked as grass, and the connected domain marked as an obstacle maintains its mark.

[0189] In some embodiments, before performing image segmentation processing on the work area image and determining the grass area and non-grass area in the work area image, the method further includes:

[0190] The distorted area image in the working area image is eliminated, where the distorted area image is an area image where the image distortion amount is greater than a preset distortion value.

[0191] Camera images inevitably contain distortion. Distorted areas are difficult to process and can adversely affect real-time map integration, reducing map accuracy. Removing distorted areas from each work area image before image segmentation can improve real-time map accuracy. The preset distortion value can be set based on actual needs.

[0192] Due to the better performance of some cameras, this step can be ignored when processing images and skipped directly. Alternatively, this step is only applied in some special scenarios, such as when the field of view is inaccurate at night and the distorted area of ​​the image at night has a greater impact on the accuracy of the real-time map than during the day, so the distorted area can be removed.

[0193] In some embodiments, a real-time map is a map that is dynamically updated over time. For example, during the day, the imaging sensor 200 has a better field of view and can clearly capture distant objects, so the real-time map can represent a larger range. At night, however, due to the poorer field of view of the imaging sensor 200, the accuracy of the distant objects captured is less accurate, and the real-time map can only be generated based on the captured images of closer objects, resulting in a smaller representation range.

[0194] In some embodiments, the real-time map is a map that is dynamically updated based on the location of the autonomous working machine 101. For example, the representation range of the real-time map is the area around the autonomous working machine 101. When the autonomous working machine 101 moves, the representation range of the real-time map also moves accordingly.

[0195] As shown in Figure 4a, a working area 10 is formed by enclosing a boundary, and the autonomous working machine 101 can move and / or work within the working area, and can also move and / or work along the boundary; when the autonomous working machine 101 is located at position D1, the representation range of the real-time map can be as shown in range 61; when the autonomous working machine 101 is located at position D2, the representation range of the real-time map can be as shown in range 62; when the autonomous working machine 101 is located at position D3, the representation range of the real-time map can be as shown in range 63; that is, the real-time map can be dynamically updated according to the position of the autonomous working machine 101.

[0196] In some embodiments, the real-time map is a map that is dynamically updated based on the surrounding environment of the autonomous working machine 101. For example, if the surrounding environment of the autonomous working machine 101 suddenly changes, such as when a person or animal suddenly enters the camera's field of view, the surrounding environment of the autonomous working machine 101 may undergo a sudden change. In this case, the real-time map may be dynamically updated based on the changed image of the working area.

[0197] In some embodiments, the real-time map is dynamically updated based on the operating mode of the autonomous machine 101. For example, a 4m x 4m real-time map may be used when the autonomous machine 101 is operating in normal mode, while a 20m x 20m real-time map may be used when the autonomous machine 101 is operating in escape mode, obstacle avoidance mode, or narrow passage mode. The autonomous machine 101 can automatically switch between normal mode, escape mode, obstacle avoidance mode, and narrow passage mode without requiring user interaction.

[0198] In some embodiments, the origin of the coordinate system of the real-time map corresponds to the origin of the coordinate system corresponding to the current image. The current image is the latest image and can reflect the actual location of the autonomous working machine 101.

[0199] In some embodiments, the origin of the coordinate system of the real-time map corresponds to a preset location. For example, the preset location can be a charging station, a transfer location, a work start location, etc.

[0200] In some embodiments, the origin of the coordinate system of the real-time map corresponds to the current location of the autonomous working machine 101 .

[0201] In some embodiments, the real-time map includes at least one map area, and the area type of the at least one map area includes one or more of a drivable area, a non-drivable area, a boundary area, and an obstacle area.

[0202] Exemplarily, the map area can be determined based on the grass area and non-grass area in the working area image, the grass area in the working area image corresponds to the drivable area of ​​the real-time map, and the non-grass area in the working area image corresponds to the non-drivable area / boundary area / obstacle area of ​​the real-time map.

[0203] Exemplarily, the real-time map only includes drivable areas and non-drivable areas, without further dividing the non-drivable areas into boundary areas and obstacle areas.

[0204] In some embodiments, the number of work area images is determined based on the work mode and / or movement status of the autonomous work machine 101 .

[0205] For example, when the autonomous machine 101 is operating during the day, the field of view is good and the images are clear, so the image processing accuracy is high. Even with a small number of work area images, a relatively accurate real-time map can be generated. When the autonomous machine 101 is operating at night, the field of view is poor and the images are unclear, so the image processing accuracy is reduced, and a larger number of work area images are required to generate a real-time map.

[0206] For example, when the movement and / or operation of the autonomous working machine 101 is abnormal, or when the autonomous working machine 101 is in a special scene, location, or working condition, and a higher accuracy requirement is placed on the real-time map, the accuracy of the real-time map can be improved by increasing the number of images of the working area.

[0207] In some embodiments, the work area image is an image captured by the autonomous work machine 101 within a target time period.

[0208] The imaging sensor 200 can capture images while the autonomous working machine 101 is moving. Each image is timestamped, indicating the time when the image was captured. The target time period can be a period before the current time point, where the current time point is the latest real-time time point, or a period between two historical time points, where the historical time point is later than the current time point. Based on the image timestamp information, images captured within the target time period (i.e., images of the work area) can be filtered from all images captured by the imaging sensor 200.

[0209] In some embodiments, the target time period is an adjustable time period, and the real-time map is updated based on the adjustment of the target time period. The target time period is adjustable. For example, the time period parameter can be set by the user or automatically adjusted by the autonomous working machine 101 based on movement and working conditions.

[0210] In some embodiments, the work area image includes a current image and historical images, and the acquisition time of the current image is later than the acquisition time of any other image acquired by the imaging sensor 200. The current image is an image acquired at the latest time point.

[0211] In addition, in conjunction with FIG7 , an embodiment is also provided, which is a control device 180 applied to an autonomous working machine 101. The autonomous working machine 101 is configured to move and / or work in a working area. The autonomous working machine 101 is provided with an imaging sensor. The control device 180 includes:

[0212] An acquisition module 1810 is configured to acquire an image of the working area captured by an imaging sensor;

[0213] The map generation module 1820 is configured to generate a real-time map based on the work area image, where the real-time map is a dynamically updated map;

[0214] The control module 1830 is configured to control the autonomous working machine 101 to move and / or work based on the real-time map.

[0215] In some embodiments, the acquisition module 1810 , the map generation module 1820 , and the control module 1830 may be integrated into a controller, or may be separately disposed in different controllers.

[0216] Based on the same inventive concept as the aforementioned embodiment, an embodiment of the present invention provides an autonomous working machine 101. As shown in FIG8 , the autonomous working machine 101 includes: a processor 310 and a memory 400 storing a computer program. The processor 310 illustrated in FIG8 does not indicate that there is one processor 310, but is merely used to indicate the positional relationship of the processor 310 relative to other components. In actual applications, there may be one or more processors 310. Similarly, the memory 400 illustrated in FIG8 has the same meaning, that is, it is merely used to indicate the positional relationship of the memory 400 relative to other components. In actual applications, there may be one or more memories 400. When the processor 310 runs the computer program, the control method of the aforementioned embodiment is implemented.

[0217] The autonomous machine 101 may also include at least one network interface 312. The various components within the autonomous machine 101 are coupled together via a bus system 313. It will be appreciated that the bus system 313 is used to enable communication between these components. In addition to a data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in FIG8 , all of these buses are labeled as the bus system 313.

[0218] The memory 400 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a magnetic disk or a magnetic tape. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 400 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memories.

[0219] The memory 400 in the embodiment of the present invention is used to store various types of data to support the operation of the autonomous working machine 101. Examples of such data include: any computer program for operating on the autonomous working machine 101, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program can include various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention can be included in the application program.

[0220] Based on the same inventive concept as the aforementioned embodiment, this embodiment further provides a computer storage medium storing a computer program. The computer storage medium may be a memory such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface mount device, an optical disc, or a compact disc read-only memory (CD-ROM). The computer storage medium may also be various devices including one or any combination of the aforementioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer storage medium is executed by a processor, the control method applied to the autonomous working machine 101 is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to the description of the control method embodiment shown above, which will not be repeated here.

[0221] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion of elements other than the listed elements and may also include additional elements not specifically listed.

[0223] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A control method, applied to an autonomous working machine (101), the autonomous working machine being configured to move and / or work in a working area (10), the autonomous working machine being provided with an imaging sensor (200), characterized in that: The method comprises: Acquiring an image of the working area captured by the imaging sensor (200); Generate a real-time map based on the work area image, wherein the real-time map is a dynamically updated map; The autonomous working machine (101) is controlled to move and / or work based on the real-time map.

2. The control method according to claim 1, characterized in that: It also includes deleting the real-time map when a preset condition is met, and the preset condition includes at least one of the following conditions: the change in posture between consecutive images is greater than a change threshold, the heading accuracy of the autonomous working machine (101) is less than a preset accuracy, the autonomous working machine (101) detects a collision, the autonomous working machine (101) detects an uphill slope, and the autonomous working machine (101) detects a downhill slope.

3. The control method according to claim 1 or 2, characterized in that: The working area image is an image collected when the autonomous working machine (101) moves within a target range; The target range is a range with a preset size centered on the first position or the second position, the first position is the power-on position of the autonomous working machine (101) or the position when the working mode is switched to the target mode, the second position satisfies the preset relative position relationship with the first position, and adjacent second positions satisfy the preset relative position relationship, and the target mode is an obstacle avoidance mode, an escape mode or a narrow channel movement mode.

4. The control method according to any one of claims 1 to 3, characterized in that: The origin of the coordinate system of the real-time map is a first position or a second position, the first position is the power-on position of the autonomous working machine (101) or the position when the working mode is switched to a target mode, the second position satisfies a preset relative position relationship with the first position, and adjacent second positions satisfy the preset relative position relationship, and the target mode is an obstacle avoidance mode, an escape mode or a narrow channel movement mode.

5. The control method according to any one of claims 1 to 4, characterized in that: The real-time map is a map that is dynamically updated according to time, the location of the autonomous working machine (101), the working mode of the autonomous working machine (101) and / or the surrounding environment of the autonomous working machine (101).

6. The control method according to any one of claims 1 to 5, characterized in that: The origin of the coordinate system of the real-time map corresponds to the origin of the coordinate system corresponding to the current image; Alternatively, the origin of the coordinate system of the real-time map corresponds to a preset position; Alternatively, the origin of the coordinate system of the real-time map corresponds to the current position of the autonomous working machine (101).

7. The control method according to any one of claims 1 to 6, characterized in that: The real-time map includes at least one map area, and the area type of the at least one map area includes one or more of a drivable area, a non-drivable area, a boundary area, and an obstacle area.

8. The control method according to any one of claims 1 to 7, characterized in that: The number of the work area images is determined based on a work mode and / or a movement state of the autonomous working machine (101).

9. The control method according to any one of claims 1 to 8, characterized in that: The working area image is an image collected by the autonomous working machine (101) within a target time period; The target time period is an adjustable time period, and the real-time map is updated based on the adjustment of the target time period.

10. The control method according to any one of claims 1 to 9, characterized in that: The working area image comprises a current image and a historical image, and the acquisition time of the current image is later than the acquisition time of any other image acquired by the imaging sensor (200).

11. The control method according to any one of claims 1 to 10, characterized in that: Generating a real-time map according to the work area image, including: Acquire non-image sensor information; Determining heading information corresponding to the working area image according to the non-image sensing information; The working area image is fused according to the heading information to generate the real-time map.

12. The control method according to claim 11, characterized in that: According to the heading information, the working area image is fused to generate the real-time map, including: According to the heading information, coordinate conversion processing is performed on the working area image to generate an image to be combined; The images to be combined are combined to generate the real-time map.

13. The control method according to claim 12, characterized in that: The images to be combined are combined to generate the real-time map, including: Determining the coincident coordinate points between the images to be combined; Determine the target attributes of the coincident coordinate points; The real-time map is generated according to the coincident coordinate points and the target attributes of the coincident coordinate points.

14. The control method according to claim 12 or 13, characterized in that: According to the heading information, coordinate conversion processing is performed on the working area image to generate an image to be combined, including: Determine the target coordinate system; Determining a coordinate conversion relationship between a coordinate system of the working area image and the target coordinate system according to the target coordinate system and the heading information; According to the coordinate transformation relationship, coordinate transformation processing is performed on the working area image to generate the image to be combined.

15. The control method according to claim 14, characterized in that: Determine the target coordinate system, including: Determining the latest image among the working area images; Using the coordinate system corresponding to the latest image as the target coordinate system; Alternatively, determining a preset coordinate system corresponding to the preset position and the preset direction; The preset coordinate system is used as the target coordinate system.

16. The control method according to any one of claims 12 to 15, characterized in that: Also includes: If after the combination processing, there are map images in the real-time map that exceed the preset range of the map, the map images that exceed the preset range of the map are deleted.

17. The control method according to any one of claims 1 to 16, characterized in that: The size of the real-time map is determined according to time, the location of the autonomous working machine (101), the working mode of the autonomous working machine (101) and / or the surrounding environment of the autonomous working machine (101).

18. The control method according to any one of claims 12 to 16, characterized in that: According to the heading information, before performing coordinate transformation processing on the working area image to generate the image to be combined, the method further includes: Performing image segmentation processing on the working area image to determine a grass area and a non-grass area in the working area image, wherein the number of types of the non-grass area is at least one; The grid confidence is determined according to the correspondence between the grid and the image pixels and the number of image pixels; According to the grid confidence, the working area image after image segmentation is subjected to rasterization processing to generate a rasterized working area image.

19. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 18.

20. An autonomous working machine (101), characterized in that include: Processor (310); A memory (400) for storing instructions executable by the processor; The processor (310) is configured to execute the method according to any one of claims 1 to 18.

Citation Information

Patent Citations

  • Mobile robot three-dimensional mapping and obstacle avoidance method based on space bag of words model

    CN105843223A

  • SLAM-based robot navigation method and robot

    CN109933056A

  • Robot navigation positioning system and method

    CN109959377A

  • Autonomous mobile robot indoor and outdoor positioning method and device based on multi-sensor fusion

    CN113607166A

  • Outdoor automatic work control system, method and equipment based on machine vision

    CN113885495A