Control method for autonomous working machine, and autonomous working machine

Abstract

Provided in the present disclosure are a control method for an autonomous working machine, and an autonomous working machine. The control method comprises: acquiring a first map of a first working area, wherein the first map is generated by means of a first image of the first working area, and the first map is based on a first coordinate system; and establishing a passage map, comprising: in response to a passage start position setting instruction, determining a passage start position of a passage area on the basis of the first map; in response to a movement instruction, controlling a machine to move from the passage start position to a passage end position; during the movement, controlling the machine to collect a passage image at each first target position; matching image features of each passage image with image features of the first image, so as to determine the pose of the machine under the first coordinate system; and storing the image features of each passage image and the corresponding pose, so as to obtain the passage map. The present application can establish a map for passages between working areas, thereby helping the autonomous working machine move from one working area to another working area.

Description

Method for controlling an autonomous working machine and autonomous working machine TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer application, and in particular, relates to a method for controlling an autonomous working machine and an autonomous working machine. BACKGROUND

[0002] With the continuous development of automation technology, machines can be used to replace humans to perform various tasks in more and more fields, thereby liberating humans from tedious, laborious, and repetitive work and greatly improving efficiency. These machines can be, for example, a floor cleaning robot for home floor cleaning, an automatic mowing device for garden repair, and the like.

[0003] Autonomous working machines usually move based on an established map to complete a task. At present, there are relatively many ways to map a working area, but in some scenarios, it is necessary to switch between multiple working areas, that is, to move from one working area to another working area, and there is currently no good method to map a passageway. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a method for controlling an autonomous working machine and an autonomous working machine to facilitate mapping a passageway between working areas, thereby helping the autonomous working machine to move from one working area to another working area.

[0005] In a first aspect, a method for controlling an autonomous working machine is provided, and the method comprises:

[0006] obtaining a first map of the first working area, the first map being generated from a first image of the first working area, wherein the first map is based on a first coordinate system;

[0007] mapping a passageway, wherein the passageway corresponds to a passageway area connected to the first working area, comprising: in response to a passageway start position setting instruction, determining a passageway start position of the passageway area based on the first map; in response to a moving instruction, controlling the autonomous working machine to move from the passageway start position to a passageway end position; in the process of moving, controlling the autonomous working machine to collect passageway images at each first target position; matching image features of the passageway images with image features of the first image to determine a pose of the autonomous working machine in the first coordinate system; storing the image features of the passageway images and the corresponding pose to obtain the passageway map.

[0008] According to an implementable manner in the embodiments of the present disclosure, the passageway map and the first map belong to the same map.

[0009] According to an implementable manner in the embodiments of the present disclosure, before the passage map is established, the control method further includes:

[0010] evaluating a quality index of the first map;

[0011] when the quality index does not meet the passage mapping requirement, prohibiting the execution of the passage mapping operation.

[0012] According to an implementable manner in the embodiments of the present disclosure, before the passage map is established, the control method further includes:

[0013] obtaining a second map of the second working area, including: controlling the autonomous working machine to collect a second image in the second working area to generate the second map;

[0014] Correspondingly, the image features of the passage image are matched with the image features of the first image to determine the pose of the autonomous working machine in the first coordinate system, including:

[0015] The image features of the passage image are matched with the image features derived from the first image and the second image to determine the pose of the autonomous working machine in the first coordinate system.

[0016] According to an implementable manner in the embodiments of the present disclosure, the control method further includes:

[0017] controlling the autonomous working machine to move in the second working area from the passage end position;

[0018] controlling the autonomous working machine to collect a second image in the process of moving to generate a second map of the second working area.

[0019] According to an implementable manner in the embodiments of the present disclosure, the control method further includes:

[0020] determining that the passage start position is in the first map; and,

[0021] determining that the passage end position is in the second map.

[0022] According to an implementable manner in the embodiments of the present disclosure, in response to a passage start position setting instruction, the passage start position of the passage area is determined based on the first map, including:

[0023] in response to a passage start position setting instruction, controlling the autonomous working machine to collect a passage image at a current position;

[0024] matching the image features of the passageway image with image features of the first image to determine a position confidence of the autonomous working machine;

[0025] in response to the position confidence being greater than or equal to a confidence threshold, setting the current position as the passageway start position.

[0026] According to an implementable manner in the embodiments of the present disclosure, the control method further comprises:

[0027] in response to the position confidence being less than the confidence threshold, controlling the autonomous working machine to send information of a passageway start position setting failure to a user terminal to prompt the user to re-set the passageway start position.

[0028] According to an implementable manner in the embodiments of the present disclosure, after determining the passageway start position of the passageway region, the control method further comprises:

[0029] controlling the autonomous working machine to collect passageway images at different directions at the passageway start position and extract image features;

[0030] adding the image features to the passageway map to perfect the map information of the passageway start position.

[0031] According to an implementable manner in the embodiments of the present disclosure, controlling the autonomous working machine to collect passageway images at different directions at the passageway start position comprises:

[0032] controlling the autonomous working machine to rotate at the passageway start position and collecting passageway images by using a camera of the autonomous working machine in the process of rotation, the passageway images comprising multiple images at different directions; or,

[0033] controlling the camera to rotate relative to a body of the autonomous working machine and collecting passageway images by using the camera in the process of camera rotation, the passageway images comprising multiple images at different directions; or,

[0034] controlling a panoramic camera of the autonomous working machine to collect a panoramic image, the passageway image being the panoramic image.

[0035] According to an implementable manner in the embodiments of the present disclosure, controlling the autonomous working machine to rotate at the passageway start position and collecting passageway images by using a camera of the autonomous working machine in the process of rotation comprises:

[0036] controlling the autonomous working machine to rotate at the passageway start position and collecting passageway images by using a camera of the autonomous working machine at a preset angle interval in the process of rotation, the preset angle interval being greater than or equal to 30 degrees and less than or equal to 60 degrees.

[0037] According to an implementable manner in the embodiments of the present disclosure, the autonomous working machine is controlled to collect a passageway image at each first target position in the process of movement, comprising:

[0038] The autonomous working machine is controlled to collect the passageway image in different directions at the passageway end position, and an image feature is extracted;

[0039] The image feature is added to the passageway map to perfect the map information of the passageway end position.

[0040] In a second aspect, a control method of an autonomous working machine is provided, which is applied to a user terminal to establish a passageway map between a first working area and a second working area, and the control method comprises:

[0041] A mapping interface is displayed, and the mapping interface at least comprises the first working area;

[0042] In response to a passageway start position setting element on the mapping interface being triggered, a passageway start position setting instruction is sent to an autonomous working machine, and the passageway start position setting instruction is used to determine a passageway start position;

[0043] In a remote control mode, a movement instruction is sent to the autonomous working machine, and the movement instruction is used to control the autonomous working machine to move from the passageway start position to a passageway end position.

[0044] According to an implementable manner in the embodiments of the present disclosure, the control method further comprises:

[0045] In the remote control mode, in response to a position of the autonomous working machine satisfying a passageway start position setting condition, a triggerable passageway start position setting element is displayed on the mapping interface.

[0046] According to an implementable manner in the embodiments of the present disclosure, in response to the position of the autonomous working machine not satisfying the passageway start position setting condition, at least one of the following page information is displayed on the mapping interface: the passageway start position setting element that is prohibited to be triggered, information of a start position setting failure, and information prompting the user to re-perform the passageway start position setting.

[0047] According to an implementable manner in the embodiments of the present disclosure, the passageway start position setting condition comprises at least one of:

[0048] When the passageway start position setting instruction is received, the autonomous working machine is in the first working area; or,

[0049] The position confidence of the autonomous working machine is greater than or equal to a confidence threshold when the passage start position setting instruction is received.

[0050] In a third aspect, an autonomous working machine is provided, comprising:

[0051] a processor;

[0052] a memory for storing the processor-executable instructions;

[0053] the processor configured to perform the method of any one of the first aspect and the second aspect.

[0054] In a fourth aspect, a computer-readable storage medium is provided, the storage medium storing a computer program configured to perform the method of any one of the first aspect and the second aspect.

[0055] As can be seen from the above technical solutions, after the first map of the first working area is acquired, the passage start position is determined through the passage start position setting instruction, and the autonomous working machine is caused to move from the passage start position to the passage end position in response to the movement instruction, the autonomous working machine is controlled to collect passage images at the first target positions in the movement process, and the passage map is established according to the image features of the passage images and the corresponding poses. In this way, the map establishment of the passage area is realized, so as to help the autonomous working machine to move from one working area to another working area based on the working map.

[0056] In the process of establishing the passage map provided by the embodiments of the present disclosure, it is not necessary to arrange auxiliary devices such as magnetic strips between the working area boundaries and the working areas, and the process is more simple and has lower cost. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0058] FIG. 1 shows an exemplary system architecture diagram to which the embodiments of the present disclosure can be applied;

[0059] FIG. 2 is a schematic structural diagram of an autonomous mobile machine provided by an exemplary embodiment of the present disclosure;

[0060] FIG. 3 is another schematic structural diagram of an autonomous working machine provided by an embodiment of the present disclosure;

[0061] FIG. 4 is a flowchart of a control method applied to an autonomous working machine according to an embodiment of the present disclosure;

[0062] FIG. 5 is a schematic diagram of determining a starting position of a pass according to an embodiment of the present disclosure;

[0063] FIG. 6 is a schematic diagram of establishing a pass map according to an embodiment of the present disclosure;

[0064] FIG. 7 is a flowchart of a control method applied to a user terminal according to an embodiment of the present disclosure;

[0065] FIG. 8 is a schematic diagram of a mapping interface according to an embodiment of the present disclosure;

[0066] FIG. 9 is another schematic diagram of a mapping interface according to an embodiment of the present disclosure;

[0067] FIG. 10 is a schematic block diagram of an autonomous working machine according to an embodiment of the present disclosure;

[0068] FIG. 11 is another flowchart of a control method applied to an autonomous working machine according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0070] The terms used in the embodiments of the present disclosure are merely for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms "a", "an" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0071] It should be understood that the term "and / or" used herein is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0072] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if a stated condition or event is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting" or "in response to detecting a stated condition or event."

[0073] In order to solve the problem that the transition between multiple work areas cannot be realized, the following methods exist. For example, a transition is realized by laying auxiliary equipment such as a magnetic strip between the work area boundary and the work area, but this method is complex to implement and high in cost. Some methods, such as using an RTK method to obtain channel information, are prone to losing positioning in a shadow area (i.e., an area with obstruction) and thus failing to transition, and are high in cost. Other methods, such as setting a label at the starting position of the channel, the label containing channel direction information, and controlling the machine to move according to the channel direction information when the label is recognized. In this case, the label must be set artificially, and the machine must travel in a certain direction to recognize the label, which is a cumbersome and unintelligent process.

[0074] Therefore, the embodiments of the present disclosure provide a brand-new method. In order to facilitate the understanding of the embodiments of the present disclosure, the system architecture to which the embodiments of the present disclosure are applied is first described.

[0075] FIG. 1 shows an exemplary system architecture diagram to which the embodiments of the present disclosure can be applied. As shown in FIG. 1, the system can include an autonomous working machine, a server, and a user terminal.

[0076] The autonomous working machine refers to an electronic device with autonomous movement and working capability. For example, a sweeping robot, a mowing device, an automatic vacuum cleaner, an automatic mop, an automatic snow sweeper, and the like, wherein the mowing device can include an automatic mower, a riding mower, and the like. The autonomous working machine can obtain user instructions through its own buttons, screens, and the like, and perform corresponding work according to the user instructions. It can also interact with the server through the network, send data to the server, or receive data or instructions sent by the server.

[0077] The server provides application services for managing and controlling the autonomous working machine. On the one hand, it can interact with the autonomous working machine through the network. On the other hand, it can interact with the user terminal through the network, send data or instructions to the user terminal, and receive data or instructions sent by the user terminal. On the other hand, the server can act as a bridge between the autonomous working machine and the user terminal, forwarding data from the autonomous working machine to the user terminal, or forwarding instructions from the user terminal to the autonomous working machine.

[0078] The service end can be a single server, a server group composed of multiple servers, or a cloud server. The cloud server, also known as a cloud computing server or a cloud host, is a host product in a cloud computing service system, and is used to solve the defects of large management difficulty and weak service scalability in traditional physical hosts and virtual private server (VPS, Virtual Private Server) services.

[0079] The user terminal can run a client, an applet, or a Web application running through a browser, etc. The user can send data, instructions, etc. to the service end through the user terminal, and then realize management and control of the autonomous working machine. In addition, the user terminal can also display data from the service end. The user can log in to the application service provided by the service end through the user terminal, that is, become a login user. One login user can bind one or more autonomous working machines.

[0080] The above-mentioned user terminal can include but is not limited to, such as: a remote controller, a smart mobile terminal, a wearable device, a PC (Personal Computer), etc. The smart mobile device can include, such as a mobile phone, a tablet computer, a notebook computer, a PDA (Personal Digital Assistant), an Internet car, etc. The wearable device can include, such as a smart watch, smart glasses, a smart bracelet, a VR (Virtual Reality) device, an AR (Augmented Reality) device, a mixed reality device (i.e. a device that can support virtual reality and augmented reality), etc.

[0081] It should be understood that the number of autonomous working machines, service ends and user terminals in FIG. 1 is only illustrative. According to the needs of implementation, there can be any number of autonomous working machines, service ends and user terminals.

[0082] As one of the implementable modes, the control device in the autonomous working machine can construct a working map including the working area by using an inertial navigation module, an odometer, an imaging sensor (for example, a camera in the figure), etc. It can also only use an imaging sensor to construct a working map including the working area. It can also use an inertial navigation module, an odometer, an imaging sensor (for example, a camera in the figure), satellite sensor fusion positioning to construct a working map including the working area. In the process of constructing the working map, the first map of the first working area, the second map of the second working area, and the channel map corresponding to the channel area between the first area and the second working area can be generated in the manner provided by the embodiments of the present disclosure. The specific construction process of these maps will be described in detail in subsequent embodiments.

[0083] The autonomous working machine can automatically move and perform corresponding work in a specified working area, and can also return to a parking station along the boundary of the working area to park or charge.

[0084] The embodiment of the present disclosure provides an autonomous working machine, as shown in FIG. 2 and FIG. 3, the autonomous working machine 1 comprises a machine body 100, an imaging sensor 200, a position sensor 500 and a controller 600.

[0085] Specifically, the machine body 100 comprises a driving device 700, which is used to drive the machine body 100 to move on the working surface according to the received driving instructions, and generally comprises a roller and a motor driving the roller to rotate. The roller can include a driving wheel and a driven wheel. The rollers can be distributed on both sides of the machine body 100, and the number of rollers on each side can be one or two.

[0086] The machine body 100 further comprises a working module for performing specific work tasks. For example, if the autonomous working machine is an automatic mower, the working module comprises a mower blade, a cutting motor, etc., and can also comprise a mower height adjusting mechanism and other auxiliary components for optimizing or adjusting the mowing effect; for example, if the autonomous working machine is an automatic vacuum cleaner, the working module comprises a suction motor, a suction port, a suction pipe, a vacuum chamber, a dust collecting device and other working components for performing the suction task.

[0087] The machine body 100 can further comprise an energy module for providing energy for various works of the autonomous working machine, which can include a rechargeable battery and a charging connection structure, wherein the charging connection structure is generally a charging electrode sheet which can be used with a charging electrode sheet arranged at the parking station to charge the autonomous working machine.

[0088] The machine body 100 further comprises a memory 400 for storing data generated by the sensor or the controller 600, or pre-storing data for use by the controller 600.

[0089] The machine body 100 further comprises a position sensor 500, which can include an IMU (inertial sensor) or an ODO (odometer) installed on the driving device 700, for obtaining the relative position according to the movement of the machine body 100.

[0090] The machine body 100 further comprises a satellite positioning sensor such as GPS, RTK, etc.

[0091] In addition to the above-mentioned modules, the machine body 100 can further comprise a shell accommodating and installing various modules, a control panel for user operation, etc., and can further comprise various environmental sensors such as humidity sensor, temperature sensor, acceleration sensor, light sensor, etc., which can assist the autonomous working machine to judge the working environment to perform corresponding programs.

[0092] The controller 600 is a core component of the autonomous working machine, used to control the autonomous working machine to automatically move and work, and the functions executed by the controller 600 include controlling the working module to start working or stop, controlling the driving device 700 to move, judging the power of the energy module and timely controlling the autonomous working machine to return to the docking station to automatically dock and charge, executing corresponding programs in combination with the data of the environmental sensor, etc.

[0093] Referring to FIGS. 2 and 3, the autonomous working machine comprises an imaging sensor 200 connected to the machine body 100, used to collect images in the advancing direction of the machine body 100, which are at least partially images of the work surface in the advancing direction. The collected images are located within the field of view range 210 of the imaging sensor 200. The imaging sensor 200 can be a camera or a laser radar commonly used in the industry, etc.

[0094] Generally, the imaging sensor 200 is installed at an upper position of the front of the machine body 100, preferably centrally arranged, with the visual angle directed downward to collect images of the work surface. The size of the field of view range 210 can be adjusted according to actual needs, and the larger the field of view range 210, the more images in the advancing direction of the machine body 100 are collected, and vice versa. The advancing direction of the machine body 100 can be various, such as normal forward movement, backward movement, turning, etc. In the present embodiment, the advancing direction of the machine body refers to the direction of normal forward movement, i.e. the direction along the central axis of the machine body.

[0095] To realize the establishment of a map for the passage region between work areas, the present embodiment provides a control method applied to an autonomous working machine, the autonomous working machine being provided with a camera, the camera can be located at the front side of the machine body, and the camera can collect images. FIG. 4 is a flowchart of the method provided by the present embodiment, applied to an autonomous working machine, for example, can be executed by a controller in the autonomous working machine. As shown in FIG. 4, the method can comprise the following steps:

[0096] Step 401: obtaining a first map of a first work area. The first map is generated from a first image of the first work area, wherein the first map is based on a first coordinate system.

[0097] Step 402: establishing a passage map. The passage region corresponding to the passage map is connected with the first work area. As shown in FIG. 11, the establishment of the passage map can comprise the following steps:

[0098] S1301: in response to a passage start position setting instruction, determining a passage start position of the passage region based on the first map;

[0099] S1302: in response to a movement instruction, controlling the autonomous working machine to move from the passage start position to a passage end position;

[0100] S1303: control the autonomous working machine to collect a channel image at each first target position during the moving process;

[0101] S1304: match the image features of the channel image with the image features of the first image to determine the pose of the autonomous working machine in the first coordinate system;

[0102] S1305: store the image features of the channel image and the corresponding pose to obtain a channel map.

[0103] As can be seen from the above process, after obtaining the first map of the first working area, the channel starting position is determined through the channel starting position setting instruction, and the autonomous working machine is moved from the channel starting position to the channel ending position in response to the moving instruction. The autonomous working machine is controlled to collect a channel image at each first target position during the moving process, and a channel map is established according to the image features of the channel image and the corresponding pose. In this way, the map establishment of the channel area is realized to help the autonomous working machine to move from one working area to another working area based on the working map.

[0104] The above process provided by the present disclosure will be described in detail in combination with embodiments. It should be noted that the "first", "second", etc. in the present disclosure do not have the limitations of size, order and quantity, and are only used to distinguish in name. For example, "first working area" and "second working area" are only used to distinguish two working areas in name. And so on.

[0105] First, the above step 401, i.e. "obtaining a first map", will be described in detail in combination with embodiments.

[0106] Before establishing the channel map, the creation of the map containing the first working area, referred to as the first map in the embodiments of the present application, is usually completed. The first working area and the second working area involved in the embodiments of the present application are independent closed areas. The channel area can be connected with the first working area or the second working area, or can be connected with both the first working area and the second working area. The channel area can be only a path segment or a region. In the disclosed embodiments, the way of obtaining the first map can include but is not limited to the following several ways:

[0107] The first way: control the autonomous working machine to move along the boundary of the first working area and perform mapping to obtain the first map.

[0108] The above mapping process can include: obtaining inertial navigation information of an inertial navigation module in the autonomous working machine, obtaining wheel speed information of an odometer, and obtaining first images collected by a camera of the autonomous working machine in different directions; using the inertial navigation information, the wheel speed information and the feature information extracted from the first images, a first map is generated.

[0109] For example, the autonomous working machine can move along the boundary of the working area under the control of the user, or the autonomous working machine can automatically identify the boundary of the first working area based on the image collected by the camera using AI (Artificial Intelligence), and then control the autonomous working machine to move along the boundary of the first working area. In the moving process, the inertial navigation information of the inertial navigation module of the autonomous working machine, the wheel speed information of the odometer, and the first image collected by the camera of the autonomous working machine are obtained.

[0110] The inertial navigation module (INS) is an autonomous navigation system that does not rely on external information and is not susceptible to interference. It is based on Newton's laws of mechanics, and by measuring the acceleration and angular acceleration of the carrier in the inertial reference frame, the velocity and angular velocity of the moving carrier are obtained by integrating the data once with respect to time, and then the position information of the moving carrier is obtained by integrating twice. Then transform these data to the navigation coordinate system, so as to obtain the inertial navigation information such as velocity, yaw angle and position in the navigation coordinate system. The inertial navigation module is mainly realized by inertial devices, which include gyroscopes and accelerometers. The gyroscope is mainly used to measure angular acceleration, and the accelerometer is used to measure the acceleration of the object motion. After the initial conditions are given, the inertial navigation module can determine the current position, yaw angle (mainly representing the direction) and velocity without external reference, which mainly represents the pose of the autonomous working machine.

[0111] The odometer is generally a wheel speed encoder that calculates the pose of the autonomous working machine according to the rotation amount of the wheel.

[0112] After obtaining the first image, visual features (also referred to as image features) can be extracted from the first image. The extracted visual features are information extracted from the image that has uniqueness and discriminability, and are used to describe the local structure or specific target features in the image. The autonomous working machine moves while collecting images, extracts visual features from the images, matches the visual features with the visual features of the plurality of first images that have been collected, and deduces the position and attitude (also referred to as "pose") of the autonomous working machine according to the pose corresponding to the first image that matches successfully. For example: if a certain first image is matched, the pose of the autonomous working machine is determined using the pose corresponding to the first image. The visual features extracted from the images in the embodiments of the present disclosure can include but are not limited to one or any combination of feature points, descriptor information, edges, textures, histograms, etc.

[0113] The feature points can be local regions with significant features in the image, such as corner points, edge points, etc. of the image, or can be some special categories of points, such as large trees, garbage cans, and other non-grass category points. The feature points can be a pixel point or a pixel region composed of multiple pixels, and can be represented by a pixel position. The descriptor information is usually a local feature representation of the pixels around the feature points, and is usually a vector or a matrix, used to describe gray-scale and other information. The edge is a region with obvious gray-scale change in the image. Texture describes the repeated structures or patterns in the image, and is used to describe the surface features of the object. The histogram is a statistical distribution of pixel values in the image, which can describe the color, texture, and other features of the image.

[0114] After obtaining the inertial navigation information, the odometry information, and the first image, visual features (also referred to as image features at other locations) can be extracted from the first image, and the inertial navigation information, the odometry information, and the visual features can be used to calculate a plurality of poses of the autonomous working machine from a starting position of the entire mapping work to an ending position of the entire mapping work. For example, the mapping work can be to control the autonomous working machine to start from a charging station and move along the boundary of a first working area. First, the pose of the autonomous working machine (or the camera of the autonomous working machine) at the charging station is determined according to the information collected by the plurality of sensors; second, the first pose of the autonomous working machine when moving at each position along the boundary of the first working area is iteratively derived according to the pose of the autonomous working machine at the charging station. The first map of the first working area is determined according to the image features of the first image and the corresponding pose. The map is based on the first coordinate system, that is, the parameters of the map adopt the same standard.

[0115] The second way is to receive the first map from the server or the user terminal.

[0116] This way is actually that the autonomous working machine obtains the first map established from the server or the user terminal. The first map can be established by the autonomous working machine before, or can be established by other autonomous working machines and sent to the server or the user terminal for storage.

[0117] The content of the first map is the same as that in the first way described above, and is not repeated.

[0118] The third way is to receive a map download instruction from the server or the user terminal, and download the first map according to the map download instruction.

[0119] The first map can be established by the autonomous working machine or other autonomous working machines.

[0120] The content of the first map is the same as the first mode, and will not be repeated.

[0121] The fourth mode is to control the autonomous working machine to collect the first image in the first working area to generate the first map.

[0122] The fourth mode can be to control the autonomous working machine to move in the first working area, and in the process of moving, to obtain the inertial navigation information of the autonomous working machine, the odometer information, and the first image collected by the camera. Different from the first mode, the fourth mode can include controlling the autonomous working machine to move along the boundary of the first working area, and can also include controlling the autonomous working machine to move in the first working area. The specific control mode is similar to the first mode, and can be moved under the control of the user; or based on the image collected by the camera, AI (Artificial Intelligence) is used to automatically identify the boundary of the first working area and obstacles, and then the autonomous working machine is controlled to move along the boundary of the first working area or in the first working area.

[0123] According to the inertial navigation information, the odometer information, and the image features extracted from the first image, the first map of the first working area is generated. The first map can be a map of the boundary of the first working area, or a map of the inside of the first working area. Similarly, the map is based on the first coordinate system, that is, the parameters of the map are the same standard.

[0124] In some embodiments of the present application, before establishing the passage map, the quality index of the first map can be evaluated; when the quality index does not meet the passage mapping requirement, the passage mapping operation is prohibited. The quality index includes that the amount of data constituting the first map is sufficient, or the image features constituting the first map are sufficient, so that the data used for subsequent establishment of the passage map can be looped with the data of the first map. That is, only when the quality index of the first map meets the passage mapping requirement, the passage mapping operation can be performed.

[0125] In some embodiments of the present application, the second map of the second working area can be acquired before the passage map is established. That is, only when the first map of the first working area is acquired and the second map of the second working area is acquired, the passage map can be established. In this way, the continuity of the maps can be ensured, and whether the passage starting position is located in the first map and whether the passage ending position is located in the second map can be detected in time.

[0126] The following steps S1301 to S1305 are described in detail for the above step 402, i.e., “establishing a passage map”.

[0127] In some embodiments of the present application, in response to the passage starting position setting instruction, the passage starting position of the passage region is determined based on the first map; in response to the movement instruction, the autonomous working machine is controlled to move from the passage starting position to the passage ending position; in the process of moving, the autonomous working machine is controlled to collect passage images at each first target position; the image features of the passage images are matched with the image features of the first images to determine the pose of the autonomous working machine in the first coordinate system; the image features of the passage images and the corresponding poses are stored to obtain the passage map. The passage images refer to the images collected by the autonomous working machine in the passage region, including the images collected at the passage starting position, the images collected at the passage ending position, and the images collected in the process of moving from the passage starting position to the passage ending position. The autonomous working machine is controlled to match the image features of the passage images with the stored historical image features to find the historical images matched with the current image features; the pose corresponding to the current image features is derived according to the matched historical image features and the pose of the historical images. The pose and the first map use the same coordinate system, i.e., also based on the first coordinate system. According to the image features of the passage images and the corresponding pose, the position of the autonomous working machine when the passage images are collected is determined, so as to establish the passage map.

[0128] In some embodiments of the present application, the first map and the passage map can belong to the same map. The first map and the passage map are maps using the same coordinate system.

[0129] In some embodiments of the present application, the first map and the passage map can be two independent maps. That is, the image features and the poses in the first map and the passage map are marked with attributes, the image features and the poses of the working area are extracted to obtain the first map, and the image features and the poses of the passage region are extracted to obtain the passage map. By extracting the image features and the poses with the same attribute, the user interface can adaptively display the first map corresponding to the working area or display the passage map corresponding to the passage region.

[0130] In some embodiments of the present application, when the positioning is lost or the positioning error is large during the process of establishing the channel map, the autonomous working machine can detect loop back and realize repositioning of the autonomous working machine when the loop back is established.

[0131] It should be noted that the meaning of "repositioning" involved in the embodiments of the present disclosure at least includes: meeting the visual loop condition, and can further include being able to determine the current pose of the autonomous working machine according to the visual features of the image. The visual loop condition refers to that the autonomous working machine recognizes that the current field of view is similar to the past field of view, that is, the machine has seen similar scenery, for example: a garbage can. When determining whether a position meets the visual loop condition, an image can be collected at the position, and the collected current image is matched with the first image stored in the machine (for example, the similarity is greater than or equal to a preset similarity threshold), that is, the image features of the two are matched. If the autonomous working machine returns to the visited position and there is a constraint relationship between the robot pose and the previous historical pose on the basis of the feature matching and the geometric consistency verification, it is considered that the visual loop condition is met. Further, repositioning includes: finding the historical image matched with the current image feature under the condition that the visual loop condition is met; and deriving the pose corresponding to the current image feature according to the matched historical image feature and the pose of the historical image.

[0132] In some embodiments, the channel start position setting condition can be: determining that the channel start position is in the first map. That is: when the channel start position setting instruction is received, the autonomous working machine is in the first working area. By controlling the channel start position in the first working area, the continuity of the mapping data between the channel map and the first map is ensured, and the subsequent autonomous working machine can also realize continuous work when working using the map.

[0133] In some embodiments, when the channel start position setting instruction is received, the positioning confidence of the autonomous working machine is greater than or equal to a confidence threshold. In this way, when the positioning is lost or the positioning error is large, the autonomous working machine can realize repositioning at the position, ensure the accuracy of the positioning, and realize fast transition. It should be noted that the positioning confidence can be the positioning confidence of the image feature when positioning is performed using an imaging sensor. The positioning confidence can also be the positioning confidence of the fusion positioning when fusion positioning is performed using an inertial navigation module, an odometer, an imaging sensor (for example, a camera in the figure), and a satellite sensor. When the positioning confidence of the fusion positioning is less than the confidence threshold, the autonomous working machine is controlled to move to an open area to restore the positioning confidence. After the positioning confidence is restored, the channel start position is reconfirmed. The restoration method of the positioning confidence of the image feature when positioning is performed using an imaging sensor is described in the subsequent part of the present disclosure.

[0134] In some embodiments, the autonomous working machine is able to identify a position of the non-grass feature as the channel start position. In other embodiments, the channel start position is set near the boundary of the first working area. That is, the distance between the determined channel start position and the boundary of the first working area is less than or equal to a first threshold value. The first threshold value can be an empirical or experimental value, for example, 2 meters. These two ways of setting the channel start position can ensure that the autonomous working machine can reposition at the position and achieve a fast transition.

[0135] The above four ways can be used together to determine the channel start position, can be partially combined, or can be used alone to determine the channel start position.

[0136] For example, the user can control the autonomous working machine to move near the boundary of the first working area, and the autonomous working machine collects images during the movement and continuously determines the position confidence determined by the images. If the confidence is greater than or equal to the confidence threshold, a prompt message can be issued. The prompt message can be issued by the autonomous working machine through its own screen, loudspeaker, etc., or can be sent to the user terminal and displayed on the screen or loudspeaker of the user terminal.

[0137] For example, the screen of the user terminal can display a mapping interface as shown in FIG. 8, which displays the position of the autonomous working machine and a channel start position setting element (such as a start mapping button). The autonomous working machine moves into the first working area (which includes the boundary of the first working area), or the position confidence determined by the channel image is less than the confidence threshold before the channel start position setting element is in an untriggerable state. If the channel start position setting condition is met, such as the position confidence determined by the channel image being greater than or equal to the confidence threshold, or the autonomous working machine moving into the first working area, the first page element can be changed to a triggerable state to prompt the user to determine the position as the channel start position. When the user triggers the channel start position setting element, the user terminal sends a start position setting instruction to the autonomous working machine. The channel start position setting element in the triggerable state and the untriggerable state can be distinguished by different colors, styles, etc. The way of identifying that the autonomous working machine moves into the first working area can include determining whether the position of the autonomous working machine determined by the channel image is in the first map. Alternatively, other sensors such as satellite positioning sensors can be used to determine whether the position of the autonomous working machine is in the first map.

[0138] As an implementation form, in response to the position of the autonomous working machine not satisfying the channel start position setting condition, at least one of the following page information can be displayed on the mapping interface: a channel start position setting element that is prohibited from being triggered, information that the channel start position setting fails, and information that prompts the user to re-perform the channel start position setting.

[0139] As an implementation form, the user can send a control instruction to the autonomous working machine to control the autonomous working machine to first move into the first working area and then control the autonomous working machine to move in the first working area to find a suitable position as the channel start position. The position determination confidence of the image collected by the autonomous working machine at the channel start position is greater than a preset threshold. Generally, when the automatic mower collects a non-grass feature or a fence, a stone road edge, it can be determined that the position determination confidence of the collected image is greater than the preset threshold. Collecting a non-grass feature can be that there are unique feature points such as garbage cans, tree trunks, flowerpots, etc. in the image.

[0140] As shown in FIG. 5, in response to the control instruction of the user, the autonomous working machine controls the autonomous working machine to move to a first position (i.e., position a in FIG. 5), and the autonomous working machine collects an image at the first position. The position determination confidence determined by the image is less than the confidence threshold. Secondly, the autonomous working machine is controlled to move a distance along the boundary of the first working area (see the black dashed arrow) from the first position and then move to a channel start position (i.e., position b in FIG. 5). The position determination confidence determined by the image collected at position b is greater than or equal to the confidence threshold). During the movement, images are collected for repositioning. The movement direction is consistent with the movement direction of the autonomous working machine during the establishment of the first map, which ensures that the collected image is as consistent as possible with the first image, thereby more easily matching the image features of the first image. The image collected at position b is taken as the channel image of the channel start position, and the corresponding pose is obtained according to the channel image.

[0141] In this way, it can be ensured that the position determination confidence of the channel image collected by the autonomous working machine when reaching the channel start position is high, that is, when the autonomous working machine reaches the channel start position and identifies low position determination confidence, this way can be used to restore the position determination confidence of the image features.

[0142] It should be noted that the position determination confidence of the channel image described above is obtained by repositioning using the channel image, which represents the confidence degree of repositioning using the channel image, and also represents the reliability of the boundary position.

[0143] In the process of finding a suitable position as the passage starting position, the autonomous working machine collects passage images and repositions. The meaning of repositioning can be found in the above embodiments. In the process of repositioning, the passage images are essentially matched by using the visual features extracted from the passage images to determine the pose of the autonomous working machine. In the matching process, the confidence of the passage images can be obtained according to the matching results (for example, the matching degree).

[0144] In the above repositioning process, boundary recognition can be further combined, for example, image segmentation can be performed on the collected passage images to obtain image segmentation results, and then the boundaries of the working area are determined by using the image segmentation results. The image segmentation method can be, but is not limited to, an edge detection-based segmentation method, a pixel feature clustering-based segmentation method, a neural network-based segmentation method, and the like. After the image segmentation of the passage images, the category information of each region, that is, the semantic information, can be obtained. For example, the lawn and the non-lawn (the non-lawn can be further divided into categories such as the stone road and the soil). The boundary of the region classified as the lawn is the boundary of the first working area. Since the image segmentation technology is a relatively mature technology at present, the image segmentation technology will not be described in further detail. Based on the results of the boundary recognition, the visual features extracted from the passage images are further matched to determine the pose of the autonomous working machine.

[0145] It should be noted that before determining the passage starting position, the autonomous working machine also collects images, but the collected images are mainly used for repositioning to determine whether they can be used as the passage starting position, so the collected images can not be saved. However, if the storage space of the autonomous working machine is sufficient, these images can also be saved.

[0146] The above steps of “in response to the movement instruction, controlling the autonomous working machine to move from the passage starting position to the passage ending position; in the process of moving, controlling the autonomous working machine to collect passage images at each first target position; matching the image features of the passage images with the image features of the first image to determine the pose of the autonomous working machine in the first coordinate system; storing the image features of the passage images and the corresponding pose to obtain the passage map” will be described in detail in combination with the embodiments. This step actually describes the process of how the autonomous working machine maps and positions in the passage.

[0147] In some embodiments of the present application, in response to the movement instruction, the control autonomous working machine moves from the passage starting position to the passage ending position. At least two cases can be included. Case one: the movement instruction is the user manipulation instruction. At this time, after the autonomous working machine reaches the passage starting position, the user can send the manipulation instruction to the autonomous working machine through the remote controller, the button or the controller on the autonomous working machine, the user terminal bound with the autonomous working machine, etc., to control the moving direction of the autonomous working machine, or the user can control the moving direction of the autonomous working machine through the detachable handrail mounted on the autonomous working machine or riding the autonomous working machine, etc., so that the autonomous working machine moves between the first working area and the second working area from the passage starting position. Case two: the movement instruction is issued by the controller of the autonomous working machine. At this time, the autonomous working machine adopts AI to automatically identify the passage area, and controls the autonomous working machine to move along the boundary of the passage area or within the passage area.

[0148] In the process of moving of the autonomous working machine from the passage starting position, the real-time position and the moving path of the autonomous working machine are acquired, and the real-time position and the moving path are sent to the user terminal bound with the autonomous working machine, so as to display the real-time position and the moving path in the mapping interface of the user terminal. In FIG. 6, the established map is shown by a black solid line, and the second working area which has not been established is shown by a dashed line. The second working area shown by the dashed line can not be displayed in the mapping interface. In FIG. 6, only the relationship between the passage and the second working area is shown, and the same is true in subsequent FIG. 8 and FIG. 9.

[0149] The passage ending position is located in the second working area. As one of the implementable ways, when the autonomous working machine moves to the second working area, the user can send a second instruction to the autonomous working machine. In response to the second instruction, the current position of the autonomous working machine is determined as the passage ending position.

[0150] As one of the implementable ways, the user can send the above-mentioned second instruction through the user terminal bound with the autonomous working machine, and the second instruction is sent to the autonomous working machine via the server end.

[0151] In addition, in addition to the judgment of the user on whether the autonomous working machine reaches the second working area, the autonomous working machine can also automatically identify whether it reaches the second working area through the image collected by the visual sensor, and can issue a prompt after identifying that it reaches the second working area. The prompt can be displayed through the autonomous working machine, such as through the screen display of the autonomous working machine, the flash of the autonomous working machine, the sound prompt of the autonomous working machine, etc. The prompt can be sent to the user terminal via the server end, and can be displayed on the user terminal, and can be prompted through text, animation, sound, etc. After the user obtains the prompt, the sending of the above-mentioned second instruction is triggered.

[0152] In the above movement, the autonomous working machine is controlled to collect a passageway image at each first target position. The first target position can include a passageway start position and a passageway end position. That is, the autonomous working machine is controlled to collect a passageway image at the passageway start position and / or the passageway end position in different directions.

[0153] That is, in one embodiment, after the passageway start position of the passageway region is determined, the autonomous working machine can be further controlled to collect a passageway image at the passageway start position in different directions, and extract image features; and the image features are added to the passageway map to perfect the map information of the passageway start position. The passageway start position is provided with richer passageway images, so that the positioning result of the passageway start position is more accurate, and more historical images can be provided to generate a loop with other positions of the passageway region, thereby improving the positioning accuracy of the other positions.

[0154] In one embodiment, the autonomous working machine can be controlled to collect a passageway image at the passageway end position in different directions, and extract image features; and the image features are added to the passageway map to perfect the map information of the passageway start position. The passageway start position is provided with richer passageway images, so that the positioning result of the passageway start position is more accurate, and more historical images can be provided to generate a loop with other positions of the second working region, thereby improving the positioning accuracy of the other positions.

[0155] If the passageway is short, passageway images can be collected at the passageway start position and the passageway end position in different directions, respectively. However, if the passageway is long, the autonomous working machine can be further controlled to collect passageway images at every preset time interval or every preset distance (for example, every 3 meters) between the passageway start position and the passageway end position, until the autonomous working machine reaches the passageway end position. Thereby, the positioning accuracy of the other positions is improved.

[0156] It should be noted that the autonomous working machine can be controlled to collect passageway images at the passageway start position in different directions in the following ways, but not limited thereto:

[0157] The first way: the autonomous working machine is controlled to rotate at the passageway start position, and passageway images are collected by the camera of the autonomous working machine during the rotation. The passageway images can include at least two images in different directions.

[0158] For example, the autonomous mower rotates at the current position every certain distance during movement, and the camera remains stationary, so as to realize image collection in multiple directions at the position. Taking a camera field angle of 120 degrees as an example, it takes 10 seconds to rotate a circle, and 100 images are collected as passageway images.

[0159] For example, the automatic mower is controlled to rotate at the passage starting position, and the camera of the automatic mower is used to capture passage images at preset angle intervals during the rotation, the preset angle intervals being greater than or equal to 30 degrees and less than or equal to 60 degrees. In this way, when the preset angle interval is 30 degrees, the automatic mower rotates one circle and about 12 pictures are taken.

[0160] The second way: control the camera to rotate relative to the body of the autonomous working machine, and capture passage images based on the camera of the autonomous working machine during the rotation of the camera, the passage images including at least two images in different directions.

[0161] For example, the camera is controlled to rotate every time the automatic mower moves a certain distance, and the automatic mower remains stationary, thereby realizing image capture in multiple directions.

[0162] The third way: control the panoramic camera of the autonomous working machine to capture panoramic images, and the passage images are panoramic images.

[0163] For example, a panoramic camera is provided on the automatic mower, and the panoramic camera captures panoramic images every time the automatic mower moves a certain distance, thereby realizing image capture in multiple directions.

[0164] In addition to the above three ways, other ways can also be used to realize, which are not listed here.

[0165] After multiple passage images are captured at the passage starting position, the position of the autonomous working machine can be determined using the passage images and the first image, that is, the passage starting position is repositioned according to the multiple passage images. If it is determined that the position of the autonomous working machine is located in the first working area (here, the first working area includes the first map), for example, inside the first working area or the boundary of the first working area, it can be considered that the passage starting position is appropriate, and the establishment of the passage map can continue, that is, the autonomous working machine is controlled to capture passage images at each first target position during movement. However, if it is determined that the position of the autonomous working machine is not located in the first working area, it can be considered that the passage starting position is incorrect, and a notification of re-determining the passage starting position can be issued. The notification can be displayed by the autonomous working machine through its screen, speaker, etc. It can also be sent to the user terminal and displayed on the mapping interface of the user terminal. After the user obtains the notification, the passage starting position is determined again in step 1301. Alternatively, if the confidence of the captured passage images is very low, less than a preset confidence threshold, a notification of re-determining the passage starting position can also be issued.

[0166] Similarly, after the plurality of pass images are captured at the end position of the pass, the position of the autonomous working machine can be determined using the pass images and the first image, i.e. the end position of the pass is re-localized according to the plurality of pass images. If it is determined that the position of the autonomous working machine is not in the second working area (here, the second working area includes the second map), it is considered that the start position of the pass is incorrect, and a notification of re-determining the start position of the pass or re-establishing the pass map can be sent. Alternatively, if the confidence of the captured pass images is low, less than a pre-set confidence threshold, a notification of re-determining the start position of the pass or re-establishing the pass map can also be sent. The way of establishing the second map of the second working area will be described in subsequent embodiments.

[0167] Further, after the end position of the pass is determined, the pass map has been obtained. The map establishment of the second working area can be continued. The autonomous working machine can be controlled to move to the start position of the second working area; the autonomous working machine can be controlled to move along the boundary of the second working area from the boundary start position to the boundary end position, and capture a second image at each second target position during the movement, the distance between the boundary start position and the boundary end position is less than or equal to a third threshold. It can be regarded as moving along the boundary of the second working area for one round (i.e. the boundary start position and the boundary end position are very close), and capturing the second image during the movement, to establish the second map of the second working area. The process of establishing the second map is similar to the process of establishing the first map, and the difference is that when determining the pose of the autonomous working machine in the first coordinate system, the feature points of the second image can be matched with the historical feature points, including: the image features of the first image, and / or the image features of the second image.

[0168] In another embodiment, after the end position of the pass is determined, the autonomous working machine can be controlled to move in the second working area from the end position of the pass; the autonomous working machine can be controlled to capture a second image during the movement to generate a second map of the second working area. Wherein, moving in the second working area includes: controlling the autonomous working machine to automatically recognize and move along the boundary, or remotely controlling the autonomous working machine by the user to move in the second working area.

[0169] Further, after moving along the second working area for one round, the map of the second working area can be corrected using the second image collected at the boundary end position of the second working area. This process is known as loop closure detection in the industry, that is, identifying whether the autonomous working machine returns to a previously visited location, and using this information to correct the accumulated error. In theory, the boundary end position and the boundary start position should detect the loop closure, and this principle is used to correct the error of the map of the second working area. Since the establishment of the passage map and the establishment of the second working area map are actually a continuous process, the error of the passage map can also be corrected after correcting the error of the second working area map. That is, the second image collected at the boundary start position and the boundary end position is used to correct the passage map. The specific correction process is known in the art and will not be described in detail here.

[0170] As mentioned above, in another embodiment, the first map of the first working area and the second map of the second working area can also be obtained before the passage map is established. This can ensure the continuity of the maps, and can timely detect whether the passage start position is located in the first map, and can timely detect whether the passage end position is located in the second map.

[0171] In this case, the image features of the passage image can be matched with the image features from the first image and the second image at the same time to determine the pose of the autonomous working machine in the first coordinate system; and the image features of the passage image and the corresponding pose are stored to obtain the passage map.

[0172] At this point, the construction of the passage map between the first working area and the second working area can be completed. However, as another more preferred way, the already established passage map can be re-learned to update the passage map, so as to improve the accuracy of the passage map and facilitate the subsequent autonomous working machine to improve the positioning accuracy during the working process.

[0173] The process of re-learning can include: controlling the autonomous working machine to move according to the passage map, and performing feature supplementing during the movement, the feature supplementing including supplementing the passage image at each third target position, and using the supplemented passage image to perfect the passage map. Wherein, the moving direction of the autonomous working machine during the re-learning process can be the same as the moving direction during the establishment of the passage map, or the autonomous working machine can move in the opposite direction to the establishment of the passage map under the remote control of the user and perform feature supplementing.

[0174] The above-mentioned third target position can be the same as the first target position, or can be different. The third target position can include part of the first target position, or can include part of the first target position.

[0175] The way of collecting the passageway image in the feature supplement process is similar to the way of collecting the passageway image in the passageway map establishing process, which can be referred to the description in the previous embodiments and will not be repeated here.

[0176] It should be further noted that the secondary learning process of the passageway map is performed at least after the passageway map is established, which can be performed before the map of the second working area is established or after the map of the second working area is established.

[0177] It can be seen that the user only needs to determine the passageway starting position and control the autonomous working machine to move from the first working area to the second working area in the whole process, and the rest is automatically completed by the autonomous working machine. The establishment of the passageway map is realized on the basis of the least possible intervention of the user, and the ease of use is improved.

[0178] After the above mapping process is completed, the first map of the first working area, the map of the second working area and the passageway map are combined to obtain a final map, and the machine can automatically move and work in different working areas according to the map. Specifically, if the autonomous working machine is in the working process in the working mode, first, after working in the first working area is completed, the autonomous working machine can be controlled to move to the passageway starting position, and the autonomous working machine collects the fourth image in the moving process. After moving to the passageway starting position, the autonomous working machine is repositioned based on the correspondence between the passageway image and the pose included in the passageway map, and moves from the passageway starting position to the passageway ending position according to the passageway map, so as to reach the second working area. When the autonomous working machine completes the work in the first working area, it can automatically move to the passageway starting position in any direction and move according to the passageway map, and then enter the second working area to continue working, which is more intelligent.

[0179] Taking the automatic mower as an example, after the automatic mower completes the mowing in the first working area by using a method such as “bow cutting” or random cutting, the autonomous working machine is controlled to move to the passageway starting position. The fourth image is collected in the moving process, and the autonomous working machine is repositioned. After reaching the passageway starting position, the autonomous working machine is controlled to move from the passageway starting position to the passageway ending position along the passageway according to the correct position and orientation and the correspondence between the passageway image and the pose stored in the passageway map, so as to reach the second working area.

[0180] The autonomous working machine can strictly move according to the passageway map in the working map during the transition (i.e., moving from the first working area to the second working area); or a self-planning bias of the autonomous working machine can be increased, that is, the autonomous working machine can move freely to a certain extent.

[0181] On one hand, increasing the self-planning bias can make the path of each movement not exactly the same, avoiding long-term movement along a fixed path to damage the passage area. On the other hand, the autonomous working machine can use the images collected by the camera to identify the type of area and obstacle avoidance processing when self-planning. For example, the automatic mower can use the images collected by the camera to identify the lawn and obstacles during the passage movement, so as to avoid damaging the lawn or collision.

[0182] Further, after the construction of the passage map is completed, the constructed passage map can be configured with attributes, and the autonomous working machine obtains attribute configuration information of the passage, and updates the passage map using the attribute configuration information, wherein the configured attributes can include but are not limited to at least one of type, name, number, and width.

[0183] Among them, the way to configure the attributes of the constructed passage can include but is not limited to the following two:

[0184] The first way: the autonomous working machine configures the attributes of the passage according to the preset configuration rule. For example, configure the number of the passage according to the order of automatic numbering, automatically configure the type of the passage as a passage, and default the width of the passage as 1 meter, etc.

[0185] In this way, in addition to updating the attribute information in the passage map, the automatically configured attribute information can also be sent to the user terminal via the server end, and updated and displayed in the mapping interface of the user terminal. In the mapping interface, the attribute information of the passage can be displayed in different forms such as color, text, and icon.

[0186] The second way: the user configures the attributes of the passage on the user terminal. For example, the attributes of the passage can be configured through the attribute configuration elements provided in the mapping interface, or the attributes of the passage can be configured in the passage configuration interface entered through the mapping interface. Then the user terminal sends the attribute configuration information to the autonomous working machine via the server end, and the autonomous working machine updates the attribute configuration information in the passage map.

[0187] The above two ways can be used alternatively, or can be used together. For example, after the autonomous working machine configures the attributes of the passage according to the preset configuration rule, the user can modify the attribute configuration information of the passage through the user terminal, and send the modified attribute configuration information to the autonomous working machine and update it in the working map. The mechanism of configuring and modifying the attributes of the passage improves the flexibility and ease of use of the map construction.

[0188] It should be noted that the above embodiment is applicable to the scenario that the first map is established, the passage map is established, and then the second working area is mapped. In addition, it is also applicable to the scenario that the first working area and the second working area are mapped, and then the passage map between the first working area and the second working area is constructed.

[0189] The passage between the first working area and the second working area can be one passage or multiple passages. In addition, there can be passages between one working area and multiple working areas, which means that there can be multiple passage start positions or passage end positions on the boundary of one working area.

[0190] FIG. 7 is a flowchart of a control method executed by a user terminal according to an embodiment of the present application. As shown in FIG. 7, the method can include the following steps:

[0191] Step 701: display a mapping interface, the mapping interface including at least a first working area.

[0192] In the process of constructing the map, the user terminal displays the mapping interface, and the mapping interface can display the moving track and position information of the autonomous working machine in real time. After the mapping of the first working area is completed, the first map of the first working area is displayed in the mapping interface. The first map can include boundary information, semantic information, etc. As shown in FIG. 8, the first map includes the boundary of the first working area, and can also include the semantics (for example, the type is lawn) of the first working area, and can also include the current position information of the autonomous mower, etc.

[0193] Step 703: in response to the passage start position setting element on the mapping interface being triggered, send a passage start position setting instruction to the autonomous working machine, the passage start position setting instruction being used to determine the passage start position.

[0194] The passage start position setting element can be a "start building passage" component as shown in FIG. 8. When the autonomous working machine moves in the first working area, the user can trigger the passage start position setting element by clicking, double-clicking, long-pressing, etc. After the event that the passage start position setting element is triggered is acquired by the user terminal, the passage start position setting instruction is sent to the server side and then sent to the autonomous working machine via the server side. The passage start position setting instruction can also be directly sent to the autonomous working machine by the user terminal.

[0195] As one of the implementable manners, the user can send a control instruction to the autonomous working machine through a remote controller, a button or a controller on the autonomous working machine, a user terminal bound with the autonomous working machine, etc. In response to the control instruction, the autonomous working machine is controlled to move to a specified position in the first working area. As another implementable manner, the user can specify a channel starting position on a mapping interface displayed on the user terminal. The information of the position is sent by the user terminal to the autonomous working machine via a server. In response to receiving the position information of the channel starting position, the autonomous working machine is controlled to move to the corresponding position.

[0196] As a more preferred embodiment, in response to a remote control element on the mapping interface being triggered, a remote control mode is started. In the remote control mode, the user can send a control instruction to the autonomous working machine through the user terminal. For example, a remote control component can be provided on the mapping interface. The user terminal sends a remote control instruction to the autonomous working machine by triggering the remote control component to control the autonomous working machine to first move into the first working area (including inside the first working area and the boundary of the first working area), and then control the autonomous working machine to move in the first working area to find a suitable position as the channel starting position. The position confidence of the image collected by the autonomous working machine at the suitable position is greater than or equal to a confidence threshold. When the position is identified, the user can send a channel starting position setting instruction to the autonomous working machine through the user terminal to determine the channel starting position.

[0197] Correspondingly, in response to the user's control instruction, the autonomous working machine is controlled to first move into the first working area, and then control the autonomous working machine to move in the first working area to find a suitable position as the channel starting position, and collect images during the movement for repositioning.

[0198] When the suitable position is identified, the signal indicator light on the autonomous working machine displays red. When the suitable position is not identified, the signal indicator light on the autonomous working machine displays yellow.

[0199] As one of the implementable manners, in response to the position of the autonomous working machine satisfying the channel starting position setting condition, the following at least one of the page information is displayed on the mapping interface: a channel starting position setting element that is prohibited to be triggered, information of failure of starting position setting, and information prompting the user to re-perform the channel starting position setting. In response to the position of the autonomous working machine not satisfying the channel starting position setting condition, the channel starting position setting element is in a triggerable state. The user can trigger the channel starting position setting element to send a channel starting position setting instruction to the autonomous working machine to trigger determination of the channel starting position to start channel mapping.

[0200] As one of the implementable manners, the passage start position setting condition can include, but is not limited to, at least one of the following: the autonomous working machine is in the first working area when the passage start position setting instruction is received; or the position confidence of the autonomous working machine is greater than or equal to a confidence threshold when the passage start position setting instruction is received.

[0201] In addition to the manner of triggering the passage start position setting instruction through the passage start position setting element on the mapping interface, the sending of the passage start position setting instruction can also be triggered on the mapping interface through inputting a voice, a text instruction, or other manners. Here, enumeration is not performed.

[0202] It should be noted that the above embodiments are applicable to the scenario of establishing a first map, then establishing a passage map, and then establishing a map for a second working area. In addition, the embodiments are also applicable to the scenario of establishing a passage map between a first working area and a second working area after the maps of the first working area and the second working area are constructed.

[0203] It should be noted that the mapping interface involved in the embodiments of the present disclosure can be a Web Page based on HyperText Markup Language (HTML), that is, an HTML page, or can also be a Web Page based on HTML and Java language, that is, a Java Server Page (JSP), or can also be a Web Page written in other languages, and the embodiments do not particularly limit this.

[0204] The element involved in the embodiments of the present disclosure can be in various forms such as a hyperlink, a control (also referred to as a component), an icon, and the like on a page. The user can trigger the element through various manners such as a touch gesture, voice control, mouse control, and a shortcut key. The touch gesture can be, for example, a manner such as clicking, double-clicking, and long-pressing of a user's finger on a screen. In addition, the position of the element involved in the embodiments of the present disclosure is only schematic in the drawings, and a suitable position can be set according to an actual page layout condition.

[0205] Step 705: In the remote control mode, a movement instruction is sent to the autonomous working machine, and the movement instruction is used to control the autonomous working machine to move from the passage start position to the passage end position.

[0206] After the passage start position is determined, the remote control mode can be entered. As one of the implementable manners, a remote control component can be provided on the mapping interface, and the movement instruction is sent to the autonomous working machine by triggering the remote control component, so as to control the autonomous working machine to move from the passage start position to the second working area. The remote control component can include components for controlling the moving direction and the moving speed of the autonomous working machine.

[0207] In addition to triggering the sending of the control instruction through the remote control component on the mapping interface, the sending of the control instruction can also be triggered through inputting voice, text instructions and other ways on the mapping interface. Here, it is not enumerated one by one.

[0208] In addition, in the case that the remote autonomous working machine moves to the second working area, if it is determined that the autonomous working machine moves to the second working area, the end-of-tunnel position setting element on the mapping interface can be triggered through clicking, double-clicking, long pressing and the like. For example, after the "start building a tunnel" component (corresponding to the start-of-tunnel position setting element) in FIG. 8 is triggered, as shown in FIG. 9, the "end building a tunnel" component (corresponding to the end-of-tunnel position setting element) can be displayed on the mapping interface. If the autonomous working machine moves to the second working area, the user triggers the "end building a tunnel" component, and the user terminal sends the end-of-tunnel position setting instruction to the autonomous working machine. After the autonomous working machine receives the end-of-tunnel position setting instruction, the end-of-tunnel position is determined. For example, the autonomous working machine determines the position where the end-of-tunnel position setting instruction is received as the end-of-tunnel position.

[0209] Among them, when it is determined that the autonomous working machine moves to the second working area, it can be manually determined by the user or recognized by the autonomous working machine. For example, the autonomous working machine recognizes that it enters the second working area through the collected tunnel image, and then sends a prompt information for confirming the end-of-tunnel position. The prompt information can take various forms, and can be sent by the autonomous working machine through its own screen, loudspeaker and the like, or can be sent to the user terminal and displayed by the user terminal. As one of the implementable ways, before the user terminal receives the prompt information, the above-mentioned end-of-tunnel position setting element is in a non-triggerable state, and in response to receiving the prompt information, the end-of-tunnel position setting element is set to a triggerable state, and the user can trigger the end-of-tunnel position setting element to send the end-of-tunnel position setting instruction to the autonomous working machine to determine the end-of-tunnel position.

[0210] After the tunnel map is established, the autonomous working machine can synchronize the tunnel map to the user terminal for display on the mapping interface. Further, an attribute configuration element can be provided on the mapping interface, and the user can configure the tunnel attribute in the tunnel map through the attribute configuration element. In response to the attribute configuration element being triggered, the attribute configuration information of the tunnel can be sent to the autonomous working machine. The attribute configuration information can include, but is not limited to, at least one of the type, name, number and width of the tunnel.

[0211] The above describes particular embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in an order other than as described in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some embodiments, multitasking and parallel processing can be advantageous or possible.

[0212] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0213] Based on the same inventive concept as the foregoing embodiments, the embodiments of the present disclosure provide an autonomous working machine, the width of the autonomous working machine is greater than or equal to 60 centimeters, as shown in FIG. 10, the autonomous working machine comprises: a processor 1210 and a memory 1211 in which a computer program is stored; wherein the processor 1210 shown in FIG. 10 is not used to indicate that the number of the processor 1210 is one, but is only used to indicate the positional relationship of the processor 1210 relative to other devices, and in actual application, the number of the processor 1210 can be one or more; similarly, the memory 1211 shown in FIG. 10 also has the same meaning, that is, it is only used to indicate the positional relationship of the memory 1211 relative to other devices, and in actual application, the number of the memory 1211 can be one or more. When the processor 1210 runs the computer program, the control method of the above embodiments is realized.

[0214] The autonomous working machine can further comprise: at least one network interface 1212. Various components in the autonomous working machine are coupled together through a bus system 1213. It can be understood that the bus system 1213 is used to realize the connection communication between the components. The bus system 1213 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, various buses are marked as the bus system 1213 in FIG. 10.

[0215] The memory 1211 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories.

[0216] The memory 1211 in the embodiments of the present disclosure is configured to store various types of data to support the operation of the autonomous working machine. Examples of the data include: any computer programs for operating on the autonomous working machine, such as operating systems and application programs; messages; pictures; videos; and the like. Among them, the operating system contains various system programs, for example, a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program can contain various application programs, for example, a media player (Media Player), a browser (Browser), and the like, for implementing various application services. Here, the program for implementing the method of the embodiments of the present disclosure can be contained in the application program.

[0217] Based on the same inventive concept as the foregoing embodiments, the embodiments of the present disclosure also provide a computer storage medium, which stores a computer program, and the computer storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), and the like. The computer storage medium can also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, and the like. The computer program stored in the computer storage medium is run by a processor to implement the control method applied to the autonomous working machine. For specific step processes implemented by the computer program when executed by the processor, please refer to the description of the method embodiments, which will not be described here.

[0218] The embodiments of the present disclosure also provide a computer program product, which includes a computer program that implements the steps of the method of any one of the foregoing method embodiments when executed by a processor.

[0219] Any technical features in the above embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations are described, but it is understood that the scope of the present disclosure encompasses all possible combinations.

[0220] In this document, the terms "comprise", "contain", or any other variant thereof are intended to cover a non-exclusive inclusion, such that the process, method, article, or apparatus that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0221] The above description is merely illustrative of the application and not restrictive thereof; the scope of the application should be determined solely by the appended claims.

Claims

1. A method for controlling an autonomous working machine, the method comprising: acquiring a first map of a first working area, the first map being generated from first images of the first working area, wherein the first map is based on a first coordinate system; building a corridor map, wherein the corridor map corresponds to a corridor area connected to the first working area, comprising: determining a corridor start position of the corridor area based on the first map in response to a corridor start position setting instruction; controlling the autonomous working machine to move from the corridor start position to a corridor end position in response to a movement instruction; controlling the autonomous working machine to capture corridor images at first target positions during the movement; matching image features of the corridor images with image features of the first images to determine a pose of the autonomous working machine in the first coordinate system; storing the image features of the corridor images and the corresponding pose to obtain the corridor map. 2.The method of claim 1, wherein the corridor map and the first map belong to a same map. 3.The method of any preceding claim, wherein before building the corridor map, the method further comprises: evaluating a quality indicator of the first map; inhibiting execution of a corridor mapping operation when the quality indicator does not satisfy a corridor mapping requirement. 4.The method of any preceding claim, wherein before building the corridor map, the method further comprises: acquiring a second map of a second working area, comprising: controlling the autonomous working machine to capture second images in the second working area to generate the second map; correspondingly, matching image features of the corridor images with image features of the first images to determine a pose of the autonomous working machine in the first coordinate system, comprising: matching image features of the corridor images with image features derived from the first images and the second images to determine a pose of the autonomous working machine in the first coordinate system. 5.The method of any preceding claim, wherein the method further comprises: controlling the autonomous working machine to move in the second working area starting from the corridor end position; controlling the autonomous working machine to capture second images during the movement to generate a second map of the second working area. 6.The method of any preceding claim, wherein the method further comprises: determining that the corridor start position is within the first map; and determining that the corridor end position is within the second map. 7.The method of any preceding claim, wherein determining a corridor start position of the corridor area based on the first map in response to a corridor start position setting instruction, comprises: controlling the autonomous working machine to capture a corridor image at a current position in response to a corridor start position setting instruction; matching image features of the corridor image with image features of the first image to determine a position confidence of the autonomous working machine; determining the current position as the corridor start position in response to the position confidence being greater than or equal to a confidence threshold. ​ 8. The control method of any preceding claim, further comprising: in response to the position confidence being less than the confidence threshold, controlling the autonomous working machine to send information of a failure of setting a channel start position to a user terminal, to prompt the user to re-set the channel start position.

9. The control method of any preceding claim, further comprising, after determining the channel start position of the channel region: controlling the autonomous working machine to capture channel images in different directions at the channel start position, and extract image features; adding the image features to the channel map to perfect the map information of the channel start position.

10. The control method of any preceding claim, wherein controlling the autonomous working machine to capture channel images in different directions at the channel start position comprises: controlling the autonomous working machine to rotate at the channel start position, and capture channel images by a camera of the autonomous working machine during the rotation, the channel images comprising a plurality of images in different directions; or controlling the camera to rotate relative to a body of the autonomous working machine, and capture channel images by the camera during the camera rotation, the channel images comprising a plurality of images in different directions; or controlling a panoramic camera of the autonomous working machine to capture a panoramic image, the channel images being the panoramic image.

11. The control method of any preceding claim, wherein controlling the autonomous working machine to rotate at the channel start position, and capture channel images by a camera of the autonomous working machine during the rotation comprises: controlling the autonomous working machine to rotate at the channel start position, and capture channel images by the camera of the autonomous working machine at preset angular intervals during the rotation, the preset angular intervals being greater than or equal to 30 degrees and less than or equal to 60 degrees.

12. The control method of any preceding claim, wherein controlling the autonomous working machine to capture channel images at each first target position during the movement comprises: controlling the autonomous working machine to capture the channel images in different directions at the channel end position, and extract image features; adding the image features to the channel map to perfect the map information of the channel end position.

13. A control method of an autonomous working machine, applied to a user terminal to establish a channel map between a first working region and a second working region, the control method comprising: displaying a mapping interface, the mapping interface comprising at least the first working region; in response to a channel start position setting element on the mapping interface being triggered, sending a channel start position setting instruction to an autonomous working machine, the channel start position setting instruction being used to determine a channel start position; in a remote control mode, sending a movement instruction to the autonomous working machine, the movement instruction being used to control the autonomous working machine to move from the channel start position to a channel end position.

14. The control method of claim 13, further comprising: ​ In the remote control mode, in response to a position of the autonomous working machine satisfying a passage start position setting condition, a triggerable passage start position setting element is displayed on the mapping interface.

15. The control method of any preceding claim, in response to a position of the autonomous working machine not satisfying a passage start position setting condition, displaying on the mapping interface at least one of the following: a page information of the passage start position setting element which is prohibited to be triggered, an information of start position setting failure, an information prompting a user to re-perform the passage start position setting.

16. The control method of any preceding claim, the passage start position setting condition comprises at least one of the following: the autonomous working machine is within the first working area when the passage start position setting instruction is received; or, a position confidence of the autonomous working machine is greater than or equal to a confidence threshold when the passage start position setting instruction is received.

17. An autonomous working machine comprising: a processor; a memory for storing instructions executable by the processor; the processor configured to perform the method of any one of claims 1 to 12.

Images