Operation map construction method, device, lawn mowing robot and storage medium

Through laser point cloud data and feature image recognition technology, the working areas of the grass mowing robot are automatically divided, solving the problem of low efficiency in building work maps in the new environment, and achieving efficient obstacle recognition and area division.

CN115235485BActive Publication Date: 2024-11-22SHENZHEN MAMMOTION INNOVATION CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202210806394.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-08
Publication Date
2024-11-22
Estimated Expiration
2042-07-08

AI Technical Summary

Technical Problem

When existing mowing robots mow grass in a new environment, they need to manually detect the site and build the work map, resulting in inefficient construction of the work map and prone to missed division or incorrect division.

Method used

By collecting laser point cloud data from the target map, candidate obstacles are determined, and target obstacles are identified based on feature images, working areas and non-working areas are divided, and areas are automatically divided in combination with image vision technology and laser point cloud technology.

Benefits of technology

It improves the efficiency of building the work map, avoids manual errors, and can determine the location of all obstacles at one time, avoiding the problems of slow lamination speed and low mapping efficiency caused by large amounts of obstacles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115235485B_ABST
    Figure CN115235485B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose a method, device, lawn mowing robot and storage medium for constructing an operation map, including: collecting laser point cloud data in an environment corresponding to a target map; determining candidate obstacles in the target map according to the laser point cloud data; acquiring feature images of the candidate obstacles, and determining target obstacles among the candidate obstacles based on the feature images; and dividing the target map into operation areas and non-operation areas according to the target obstacles. This scheme can improve the efficiency of constructing an operation map.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for constructing an operation map, a lawn mowing robot, and a storage medium. Background Art

[0002] Mowing robots are widely used in maintaining lawns in home gardens and mowing large lawns. Mowing robots integrate technologies such as motion control, multi-sensor fusion, and path planning. In order to control the mowing robot to achieve mowing operations, it is necessary to plan the mowing path of the mowing robot so that it can completely cover all operating areas.

[0003] When a lawn mowing robot is mowing in a new environment, staff are required to conduct real-time detection of the site and transmit the data to the lawn mowing robot to create an electronic map for use by the lawn mowing robot. Different lawns need to be measured and input again, that is, the current work map construction efficiency is low. Summary of the invention

[0004] The embodiments of the present application provide a method and device for constructing an operation map, a lawn mowing robot, and a storage medium, which can improve the efficiency of constructing an operation map.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a work map, comprising:

[0006] Get the preset mowing area;

[0007] Collect laser point cloud data in the environment corresponding to the target map;

[0008] Determining candidate obstacles in the target map according to the laser point cloud data;

[0009] Acquire a feature image of the candidate obstacle, and determine a target obstacle among the candidate obstacles based on the feature image;

[0010] According to the target obstacle, an operation area and a non-operation area are divided in the target map.

[0011] Optionally, in some embodiments, determining candidate obstacles in the target map according to the laser point cloud data includes:

[0012] Acquire the map coordinate system of the target map;

[0013] Extracting the reflection value and the three-dimensional coordinates corresponding to each three-dimensional laser point from the laser point cloud data;

[0014] Based on the map coordinates, the reflection value corresponding to the three-dimensional laser point and the three-dimensional coordinates, candidate obstacles are determined in the target map.

[0015] Optionally, in some embodiments, determining a candidate obstacle in the target map based on the map coordinates, the reflection value corresponding to the three-dimensional laser point, and the three-dimensional coordinates includes:

[0016] Determine a point cloud coordinate system corresponding to the laser point cloud data;

[0017] Based on the three-dimensional coordinates corresponding to the three-dimensional laser point and the conversion relationship between the map coordinate system and the point cloud coordinate system, rendering the reflection value corresponding to the three-dimensional laser point into the target map;

[0018] According to the pixel values ​​in the rendered target map, candidate obstacles are determined in the target map.

[0019] Optionally, in some embodiments, rendering the reflection value corresponding to the three-dimensional laser point into the target map based on the three-dimensional coordinates corresponding to the three-dimensional laser point and the conversion relationship between the map coordinate system and the point cloud coordinate system includes:

[0020] Based on the conversion relationship between the map coordinate system and the point cloud coordinate system, the three-dimensional coordinates corresponding to the three-dimensional laser point are converted to obtain the map coordinates of the three-dimensional laser point in the target map;

[0021] According to the map coordinates of the three-dimensional laser point in the target map, the reflection value corresponding to the three-dimensional laser point is rendered into the target map.

[0022] Optionally, in some embodiments, determining a target obstacle from among the candidate obstacles based on the feature image comprises:

[0023] Inputting the feature image into a preset image classification network to obtain a classification label of the candidate obstacle;

[0024] The candidate obstacle whose classification label is the target label is determined as the target obstacle.

[0025] Optionally, in some embodiments, dividing the operating area and the non-operating area in the target map according to the target obstacle includes:

[0026] At least obtaining contour information of the target obstacle below a preset height;

[0027] Outputting an isolation curve surrounding the target obstacle according to the contour information and the position of the target obstacle in the target map;

[0028] The area within the isolation curve is determined as a non-operating area, and the area other than the non-operating area is determined as an operating area.

[0029] Optionally, in some embodiments, after dividing the operating area and the non-operating area in the target map according to the target obstacle, the following further includes:

[0030] highlighting the working area using a first color, and;

[0031] The non-operating area is highlighted in a second color.

[0032] In a second aspect, an embodiment of the present application provides a work map construction device, including:

[0033] The acquisition module is used to collect laser point cloud data of the target map;

[0034] A first determination module, configured to determine a candidate obstacle in the target map according to the laser point cloud data;

[0035] An acquisition module, used to acquire a feature image of the candidate obstacle;

[0036] A second determination module, configured to determine a target obstacle among the candidate obstacles based on the feature image;

[0037] A division module is used to divide the target map into an operation area and a non-operation area according to the target obstacle.

[0038] After collecting laser point cloud data in the environment corresponding to the target map, the embodiment of the present application determines candidate obstacles in the target map according to the laser point cloud data, then obtains a feature image of the candidate obstacle, and determines the target obstacle in the candidate obstacle based on the feature image. Finally, according to the target obstacle, the target map is divided into a working area and a non-working area. In the mowing solution provided by the present application, the laser point cloud data is used to determine the candidate obstacle in the target map, and the target obstacle is determined according to the feature image of the candidate obstacle. Finally, according to the target obstacle, the target map is divided into a working area and a non-working area. That is, through the combination of image vision technology and laser point cloud technology, the problem of missed division or wrong division caused by manual division of working areas and non-working areas can be avoided, thereby improving the efficiency of building working maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1a It is a scene schematic diagram of the operation map construction method provided in the embodiment of the present application;

[0041] Figure 1b It is a flowchart of a method for constructing a work map provided in an embodiment of the present application;

[0042] Figure 2a is a structural schematic diagram of a work map construction device provided in an embodiment of the present application;

[0043] Figure 2b This is another structural diagram of the operation map construction device provided in the embodiment of the present application.

[0044] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0046] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for both fixing and circuit connection.

[0047] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0048] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0049] Embodiments of the present application provide a method and device for constructing an operation map, a lawn mowing robot, and a storage medium.

[0050] Among them, the operation map construction device can be specifically integrated in the microcontroller unit (MCU) of the lawn mower robot, and can also be integrated in the intelligent terminal or server. The MCU is also called a single chip microcomputer (Single Chip Microcomputer) or a single chip microcomputer. It is a chip-level computer that appropriately reduces the frequency and specifications of the central processing unit (CPU), and forms memory, counter (Timer), USB, analog-to-digital conversion / digital-to-analog conversion, UART, PLC, DMA and other peripheral interfaces to make different combination controls for different application occasions. The lawn mower robot can walk automatically, prevent collisions, automatically return to charge within the range, has safety detection and battery power detection, and has a certain climbing ability. It is especially suitable for lawn mowing and maintenance in places such as home courtyards and public green spaces. Its characteristics are: automatic mowing, cleaning grass clippings, automatic rain shelter, automatic charging, automatic obstacle avoidance, small size, electronic virtual fence, network control, etc.

[0051] The terminal may be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication. The server may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. This application does not limit this.

[0052] For example, see Figure 1aThe present application provides a mowing system, including a mowing robot 10, a server 20 and a user device 30 that are connected to each other by communication. The mowing robot 10 is equipped with a laser radar, which can collect laser point cloud data in the environment corresponding to the target map through the laser radar. Then, the mowing robot 10 can determine the candidate obstacles in the target map according to the laser point cloud data, wherein the target map can be an environment map or an electronic map. Subsequently, the camera in the mowing robot 10 obtains the characteristic image of the candidate obstacle, and determines the target obstacle among the candidate obstacles based on the characteristic image. Finally, the mowing robot 10 divides the target map into an operating area and a non-operating area according to the target obstacle. After dividing the operating area and the non-operating area, the mowing robot 10 can synchronize the data of the operating area and the data of the non-operating area to the server 20 and the user device 30, so as to facilitate the subsequent monitoring of the mowing operation of the mowing robot 10. The mowing solution provided by this application uses laser point cloud data to determine candidate obstacles in the target map, and determines the target obstacle based on the characteristic image of the candidate obstacle. Finally, the target map is divided into the working area and the non-working area based on the target obstacle. That is, through the combination of image vision technology and laser point cloud technology, the problem of missing or wrong division caused by manual division of the working area and the non-working area can be avoided; in addition, since the laser radar uniformly scans the working environment, the positions of all obstacles can be determined at one time, without the need to circle the obstacles one by one, avoiding the problem of slow overall obstacle circle speed and low map construction efficiency due to the large size of the obstacles. Therefore, this solution improves the efficiency of working map construction.

[0053] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0054] A method for constructing an operation map comprises: collecting laser point cloud data in an environment corresponding to a target map, determining candidate obstacles in the target map according to the laser point cloud data, acquiring feature images of the candidate obstacles, and determining target obstacles among the candidate obstacles based on the feature images, and dividing an operation area and a non-operation area in the target map according to the target obstacles.

[0055] See also Figure 1b , Figure 1b This is a flow chart of a method for constructing a work map provided in an embodiment of the present application. The specific flow of the method for constructing a work map may be as follows:

[0056] 101. Collect laser point cloud data in the environment corresponding to the target map.

[0057] The target map is a mowing map corresponding to the mowing robot. The mowing robot can perform mowing operations in the area corresponding to the target map. The operating area and non-operating area can be divided in the target map later, and the target map does not include buildings such as houses where operations cannot be performed.

[0058] For example, a three-dimensional laser radar can be installed on the body of a lawn mowing robot. The three-dimensional laser radar is a measuring instrument that instantaneously measures the three-dimensional coordinate values ​​of a space through the principle of laser ranging (including pulse laser and phase laser). The spatial point cloud data obtained by three-dimensional laser scanning technology can be used to quickly establish a three-dimensional visualization model of a scene with complex and irregular structures.

[0059] In practical applications, 3D laser radar obtains the position information and 3D point cloud corresponding to each acquisition point based on the simultaneous localization and mapping (SLAM) method. 3D laser radar can be portable, backpack or vehicle-mounted, thus realizing mobile scanning.

[0060] For example, the acquisition point initially acquired by the three-dimensional laser radar is used as the coordinate origin, and a point cloud coordinate system is constructed. The initial acquisition here refers to the first frame of three-dimensional point cloud corresponding to the three-dimensional point cloud map acquired by the three-dimensional laser radar. The acquisition point can be the location of the center of gravity of the three-dimensional acquisition device or a fixed reference point on the device, as long as it meets the requirements of establishing a coordinate system and defining the coordinate origin. In one example, in the point cloud coordinate system, the Z axis is located in the vertical scanning plane, with upward being positive, and the X and Y axes are both located in the horizontal scanning plane and the three are perpendicular to each other to form a left-handed coordinate system. During the mobile scanning process, the real-time position and posture of the three-dimensional laser radar and the three-dimensional point cloud at that moment can be obtained in real time based on the SLAM method.

[0061] 102. Determine candidate obstacles in the target map based on the laser point cloud data.

[0062] Among them, the reflection intensity is an important characteristic of the laser sensor, which can reflect the material characteristics in the environment. Therefore, the reflection intensity can be used to determine the candidate obstacles in the target map. That is, optionally, in some embodiments, the step of "determining the candidate obstacles in the target map according to the laser point cloud data" can specifically include:

[0063] (11) Obtaining the map coordinate system of the target map;

[0064] (12) extracting the reflection value and three-dimensional coordinates corresponding to each three-dimensional laser point from the laser point cloud data;

[0065] (13) Based on the map coordinates, the reflection value corresponding to the three-dimensional laser point and the three-dimensional coordinates, the candidate obstacles are determined in the target map.

[0066] For example, the corresponding pixel value can be rendered in the target map according to the reflection value corresponding to each three-dimensional laser point. For example, the corresponding pixel value of a three-dimensional laser point with a reflection value of a is 10, and the corresponding pixel value of a three-dimensional laser point with a reflection value of b is 45. The specific settings can be made according to the actual situation and will not be repeated here.

[0067] The map coordinate system of the target map is a two-dimensional coordinate system, and the coordinates of the three-dimensional laser point also include height information. It is also necessary to obtain the point cloud coordinate system corresponding to the laser point cloud data so as to subsequently determine the candidate obstacles in the target map. That is, optionally, in some embodiments, the step of "determining the candidate obstacles in the target map based on the map coordinates, the reflection values ​​corresponding to the three-dimensional laser points, and the three-dimensional coordinates" may specifically include:

[0068] (21) Determine the point cloud coordinate system corresponding to the laser point cloud data;

[0069] (22) Based on the three-dimensional coordinates corresponding to the three-dimensional laser point and the conversion relationship between the map coordinate system and the point cloud coordinate system, the reflection value corresponding to the three-dimensional laser point is rendered into the target map;

[0070] (23) Determine candidate obstacles in the target map based on the pixel values ​​in the rendered target map.

[0071] For example, specifically, the three-dimensional coordinates corresponding to the three-dimensional laser point can be converted into map coordinates in the target map based on the conversion relationship between the map coordinate system and the point cloud coordinate system. Subsequently, based on the converted map coordinates, the reflection value corresponding to the three-dimensional laser point is rendered into the target map. Finally, based on the pixel value in the rendered target map, the candidate obstacle is determined in the target map.

[0072] Optionally, in some embodiments, the step of “rendering the reflection value corresponding to the three-dimensional laser point into the target map based on the three-dimensional coordinates corresponding to the three-dimensional laser point and the conversion relationship between the map coordinate system and the point cloud coordinate system” may specifically include:

[0073] (41) based on the conversion relationship between the map coordinate system and the point cloud coordinate system, converting the three-dimensional coordinates corresponding to the three-dimensional laser point to obtain the map coordinates of the three-dimensional laser point in the target map;

[0074] (42) Rendering the reflection value corresponding to the three-dimensional laser point into the target map according to the map coordinates of the three-dimensional laser point in the target map.

[0075] Among them, the three-dimensional coordinates corresponding to the three-dimensional laser point can be converted by a preset formula. The preset formula represents the conversion relationship between the three-dimensional coordinate system and the two-dimensional coordinate system, that is, the conversion relationship between the point cloud coordinate system and the map coordinate system. It should be noted that in the field of image processing, the coordinate system conversion is to convert the three-dimensional world coordinate system of the space into the two-dimensional pixel coordinate system of image processing. Commonly used coordinate systems include the world coordinate system, the camera coordinate system and the image coordinate system. The world coordinate system (world coordinate) (xw, yw, zw), also known as the measurement coordinate system, is a three-dimensional rectangular coordinate system, which can be used as a reference to describe the spatial position of the camera and the object to be measured; the camera coordinate system (camera coordinate) (xc, yc, zc) is also a three-dimensional rectangular coordinate system, the origin is located at the optical center of the lens, the xc and yc axes are parallel to the two sides of the image plane, and the zc axis is the optical axis of the lens, which is perpendicular to the image plane; the image coordinate system (image coordinate) (x, y) is a two-dimensional rectangular coordinate system on the image plane. The origin of the image coordinate system is the intersection of the lens optical axis and the image plane (also called the principal point). Its x-axis is parallel to the xc-axis of the camera coordinate system, and its y-axis is parallel to the yc-axis of the camera coordinate system.

[0076] Specifically, the coordinate transformation relationship can be determined based on the external parameters between the laser radar device and the image acquisition device (i.e., the acquisition device of the target map), as well as the internal parameters of the image acquisition device. External parameters refer to the parameters of the image acquisition device in the world coordinate system, such as the position and rotation direction of the image acquisition device; internal parameters refer to parameters related to the characteristics of the image acquisition device itself, such as the focal length and pixel size of the image acquisition device.

[0077] For example, to convert the point cloud 3D coordinates pij to the map 2D coordinates p'i'j', the conversion can be performed as follows:

[0078] x'=(x / z)*fx+cx

[0079] y'=(y / z)*fy+cy

[0080] Among them, fx, fy are the focal lengths of the image acquisition device, cx, cy are the principal points of the image acquisition device, and the corresponding two-dimensional coordinates of the map are p'i'j'(x', y') in the point cloud three-dimensional coordinates pij (x, y, z).

[0081] 103. Acquire a feature image of the candidate obstacles, and determine a target obstacle among the candidate obstacles based on the feature image.

[0082] It should be noted that since the candidate obstacle is determined by laser point cloud data, that is, the obstacle is determined based on the reflection of the lidar signal, however, lawn mowing robots are generally used for outdoor operations. If there are other partially light-transmitting materials between the scanner and the object to be measured, such as rain, snow, dust, etc. that are common in outdoor environments, part of the laser energy will be reflected back in advance. As long as the trigger threshold is reached, it will be considered as the object to be measured, resulting in measurement errors. Therefore, the candidate obstacle determined by the laser point cloud data may not be a real obstacle. Therefore, in this application, visual technology and point cloud technology are combined to locate and identify obstacles.

[0083] For example, specifically, an image of the candidate obstacle may be collected, and features may be extracted from the collected image to obtain a feature image of the candidate obstacle. Specifically, a convolutional neural network (CNN) may be used to extract features from the collected image. Further, the convolutional neural network may be used to determine the target obstacle from the candidate obstacles. That is, optionally, in some embodiments, the step of “determining the target obstacle from the candidate obstacles based on the feature image” may specifically include:

[0084] (51) Inputting the feature image into a preset image classification network to obtain the classification label of the candidate obstacle;

[0085] (52) The candidate obstacle whose classification label is the target label is determined as the target obstacle.

[0086] The image classification network may be pre-trained, and the image classification network may specifically include:

[0087] Convolution layer: mainly used for feature extraction of input images (such as training samples or images to be recognized), wherein the size of the convolution kernel and the number of convolution kernels can be determined according to the actual application. For example, the sizes of the convolution kernels from the first convolution layer to the fourth convolution layer can be (7, 7), (5, 5), (3, 3), (3, 3) respectively; optionally, in order to reduce the complexity of calculation and improve the calculation efficiency, in this embodiment, the convolution kernel sizes of the four convolution layers can be set to (3, 3), the activation function all adopts "relu (rectified linear unit)", and the padding (padding refers to the space between the element border and the element content defined by the attribute) mode is set to "same". The "same" filling mode can be simply understood as filling the edge with 0, and the number of 0s on the left (top) is the same as or one less than the number of 0s on the right (bottom). Optionally, the convolutional layers can be directly connected to each other to speed up the network convergence. In order to further reduce the amount of calculation, downsampling (pooling) operations can be performed in all layers from the second to the fourth convolutional layers or in any 1 to 2 layers. The downsampling operation is basically the same as the convolution operation, except that the downsampled convolution kernel only takes the maximum value (max pooling) or the average value (average pooling) of the corresponding position. For the convenience of description, in the embodiments of the present invention, downsampling operations are performed in the second convolutional layer and the third convolutional layer, and the downsampling operation is specifically maxpooling.

[0088] It should be noted that, for the convenience of description, in the embodiment of the present application, the layer where the activation function is located and the downsampling layer (also called the pooling layer) are both included in the convolution layer. It should be understood that the structure can also be considered to include the convolution layer, the layer where the activation function is located, the downsampling layer (i.e., the pooling layer) and the fully connected layer. Of course, it can also include an input layer for inputting data and an output layer for outputting data, which will not be repeated here.

[0089] Fully connected layer: It can map the learned features to the sample label space. It mainly plays the role of "classifier" in the entire convolutional neural network. Each node of the fully connected layer is connected to all nodes output by the previous layer (such as the downsampling layer in the convolutional layer). Among them, a node in the fully connected layer is called a neuron in the fully connected layer. The number of neurons in the fully connected layer can be determined according to the needs of the actual application. For example, in this text detection model, the number of neurons in the fully connected layer can be set to 512, or 128, etc. Similar to the convolutional layer, in the fully connected layer, nonlinear factors can be added by adding activation functions. For example, the activation function sigmoid (S-type function) can be added.

[0090] Specifically, the image classification network can be used to identify the feature image, obtain the probability that the candidate obstacle belongs to each type of obstacle, and output the corresponding classification label based on the probability. Finally, the candidate obstacle whose classification label is the target label is determined as the target obstacle. For example, the candidate obstacle whose classification label is the flower bed is determined as the target obstacle.

[0091] 104. Based on the target obstacles, the target map is divided into operation area and non-operation area.

[0092] Specifically, the contour information of the target obstacle may be obtained, and the operating area and the non-operating area may be divided in the target map. For example, a curve may be output to surround the target obstacle. After the non-operating area is determined, the operating area may be divided in the target map based on the preset operating boundary and the non-operating area. That is, optionally, in some embodiments, the step of "dividing the operating area and the non-operating area in the target map according to the target obstacle" may specifically include:

[0093] (61) obtaining at least the contour information of the target obstacle below a preset height;

[0094] (62) outputting an isolation curve surrounding the target obstacle according to the contour information and the position of the target obstacle in the target map;

[0095] (63) The area within the isolation curve is determined as a non-operating area, and the area outside the non-operating area is determined as an operating area.

[0096] Optionally, the preset height can be set to be slightly higher than the height of the mowing robot. For example, if the height of the mowing robot is 30 cm, the preset height can be set to 35 cm to ensure that the mowing robot will not be blocked by obstacles and interrupt the mowing operation when performing the mowing operation.

[0097] It should be noted that when there are multiple target obstacles in the target map, the distance between adjacent target obstacles can be calculated, and the target obstacles with a distance less than a threshold can be divided into the same non-operating area. The threshold can be set according to the size of the mowing robot, which can avoid the problem that the designated operating area is too small and the mowing robot cannot operate, thereby avoiding this type of operating area affecting the process of subsequent mowing operations, thereby improving the subsequent mowing efficiency.

[0098] Furthermore, in some embodiments, the operation area and the non-operation area may be distinguished by different colors, that is, optionally, after the step of "dividing the operation area and the non-operation area in the target map according to the target obstacle", the following may be specifically included:

[0099] The working area is highlighted with a first color, and the non-working area is highlighted with a second color.

[0100] For example, specifically, the first color may be yellow, and the second color may be red. The specific selection may be made based on actual conditions, and details will not be repeated here.

[0101] After collecting the laser point cloud data in the environment corresponding to the target map, the embodiment of the present application determines the candidate obstacles in the target map according to the laser point cloud data, then obtains the characteristic image of the candidate obstacles, and determines the target obstacle in the candidate obstacles based on the characteristic image, and finally, divides the working area and the non-working area in the target map according to the target obstacle. In the mowing scheme provided by the present application, the candidate obstacles are determined in the target map using the laser point cloud data, and the target obstacles are determined according to the characteristic image of the candidate obstacles, and finally, the working area and the non-working area are divided in the target map according to the target obstacle, that is, through the combination of image vision technology and laser point cloud technology, the problem of missing or wrong division caused by manual division of the working area and the non-working area can be avoided; in addition, since the laser radar uniformly scans the working environment, the positions of all obstacles can be determined at one time, without the need to circle the obstacles one by one, avoiding the problem of slow overall obstacle circle speed and low map construction efficiency due to the large volume of the obstacles. Therefore, the present scheme improves the efficiency of working map construction.

[0102] See also Figure 2a , Figure 2a The schematic diagram of the structure of the operation map construction device provided in the embodiment of the present application, wherein the operation map construction device may include a collection module 201, a first determination module 202, an acquisition module 203, a second determination module 204 and a division module 205, which may be specifically as follows:

[0103] The acquisition module 201 is used to acquire laser point cloud data in the environment corresponding to the target map.

[0104] For example, the acquisition module 201 can use the spatial point cloud data acquired by the three-dimensional laser scanning technology to quickly establish a three-dimensional visualization model of a scene with a complex and irregular structure.

[0105] The first determination module 202 is used to determine candidate obstacles in the target map according to the laser point cloud data.

[0106] Optionally, in some embodiments, the first determining module 202 may specifically include:

[0107] An acquisition unit, used for acquiring a map coordinate system of a target map;

[0108] An extraction unit, used for extracting the reflection value and three-dimensional coordinates corresponding to each three-dimensional laser point from the laser point cloud data;

[0109] The determination unit is used to determine the candidate obstacle in the target map based on the map coordinates, the reflection value corresponding to the three-dimensional laser point and the three-dimensional coordinates.

[0110] Optionally, in some embodiments, the determining unit may specifically include:

[0111] A first determining subunit is used to determine a point cloud coordinate system corresponding to the laser point cloud data;

[0112] A rendering subunit, for rendering the reflection value corresponding to the 3D laser point into the target map based on the 3D coordinates corresponding to the 3D laser point and the conversion relationship between the map coordinate system and the point cloud coordinate system;

[0113] The second determination subunit is used to determine candidate obstacles in the target map according to pixel values ​​in the rendered target map.

[0114] Optionally, in some embodiments, the rendering subunit can be specifically used to:) based on the conversion relationship between the map coordinate system and the point cloud coordinate system, convert the three-dimensional coordinates corresponding to the three-dimensional laser point to obtain the map coordinates of the three-dimensional laser point in the target map; according to the map coordinates of the three-dimensional laser point in the target map, render the reflection value corresponding to the three-dimensional laser point into the target map.

[0115] The acquisition module 203 is used to acquire the characteristic image of the candidate obstacle.

[0116] The second determination module 204 is configured to determine a target obstacle from among the candidate obstacles based on the feature image.

[0117] The second determination module 204 may specifically use a convolutional neural network to determine a target obstacle from candidate obstacles. Optionally, in some embodiments, the second determination module 204 may specifically be used to: input the feature image into a preset image classification network to obtain a classification label of the candidate obstacle; and set the classification label as the candidate obstacle with the target label.

[0118] The division module 205 is used to divide the target map into an operation area and a non-operation area according to the target obstacles.

[0119] Specifically, the division module 205 can obtain the contour information of the target obstacle and divide the operating area and the non-operating area in the target map. That is, optionally, in some embodiments, the division module 205 can be specifically used to: at least obtain the contour information of the target obstacle in the target map; output an isolation curve surrounding the target obstacle based on the contour information and the position of the target obstacle in the target map; determine the area within the isolation curve as the non-operating area, and determine the area outside the non-operating area as the operating area.

[0120] Optionally, in some embodiments, see Figure 2b The operation map construction device of the present application may further include a display module 206, and the display module 206 may be used to: highlight the operation area with a first color, and highlight the non-operation area with a second color.

[0121] After the acquisition module 201 of the embodiment of the present application acquires the laser point cloud data in the environment corresponding to the target map, the first determination module 202 determines the candidate obstacle in the target map according to the laser point cloud data, then the acquisition module 203 acquires the characteristic image of the candidate obstacle, the second determination module 204 determines the target obstacle in the candidate obstacle based on the characteristic image, and finally, the division module 205 divides the working area and the non-working area in the target map according to the target obstacle. In the mowing scheme provided by the present application, the candidate obstacle is determined in the target map using the laser point cloud data, and the target obstacle is determined according to the characteristic image of the candidate obstacle. Finally, the working area and the non-working area are divided in the target map according to the target obstacle, that is, through the combination of image vision technology and laser point cloud technology, the problem of missing or wrong division caused by manual division of the working area and the non-working area can be avoided, thereby improving the efficiency of building the working map.

[0122] In addition, the present application also provides a lawn mowing robot. Figure 3 As shown, it shows a schematic diagram of the structure of the lawn mowing robot involved in the embodiment of the present application, specifically:

[0123] The lawn mowing robot may include components such as a control module 301, a travel mechanism 302, a cutting module 303, and a power supply 304. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0124] The control module 301 is the control center of the lawn mower robot. The control module 301 may specifically include components such as a central processing unit (CPU), a memory, an input / output port, a system bus, a timer / counter, a digital-to-analog converter, and an analog-to-digital converter. The CPU executes various functions of the lawn mower robot and processes data by running or executing software programs and / or modules stored in the memory, and calling data stored in the memory; preferably, the CPU may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the CPU.

[0125] The memory can be used to store software programs and modules. The CPU executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the CPU with access to the memory.

[0126] The traveling mechanism 302 is electrically connected to the control module 301 and is used to respond to the control signal transmitted by the control module 301 to adjust the traveling speed and traveling direction of the lawn mower robot, thereby realizing the self-moving function of the lawn mower robot.

[0127] The cutting module 303 is electrically connected to the control module 301 and is used to respond to the control signal transmitted by the control module to adjust the height and rotation speed of the cutting disc to achieve mowing operation.

[0128] The power supply 304 can be logically connected to the control module 301 through a power management system, so as to realize functions such as managing charging, discharging, and power consumption management through the power management system. The power supply 304 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0129] Although not shown, the lawn mowing robot may also include a communication module, a sensor module, a prompt module, etc., which will not be described in detail here.

[0130] The communication module is used to receive and send signals during the process of sending and receiving information. By establishing a communication connection with the user equipment, base station or server, the signal sending and receiving between the user equipment, base station or server is realized.

[0131] The sensor module is used to collect internal environment information or external environment information, and feed the collected environment data back to the control module for decision-making, so as to realize the precise positioning and intelligent obstacle avoidance function of the lawn mowing robot. Optionally, the sensor may include: ultrasonic sensor, infrared sensor, collision sensor, rain sensor, lidar sensor, inertial measurement unit, wheel speed meter, image sensor, position sensor and other sensors, which are not limited to this.

[0132] The prompt module is used to prompt the user of the current working status of the lawn mower robot. In this solution, the prompt module includes but is not limited to indicator lights, buzzers, etc. For example, the lawn mower robot can prompt the user of the current power status, motor working status, sensor working status, etc. through indicator lights. For another example, when the lawn mower robot is detected to be faulty or stolen, an alarm prompt can be implemented through a buzzer.

[0133] Specifically in this embodiment, the processor in the control module 301 will load the executable files corresponding to the processes of one or more application programs into the memory according to the following instructions, and the processor will run the application programs stored in the memory to implement various functions, as follows:

[0134] Collect laser point cloud data in the environment corresponding to the target map, determine candidate obstacles in the target map based on the laser point cloud data, obtain feature images of the candidate obstacles, and determine the target obstacle among the candidate obstacles based on the feature images, and divide the operating area and non-operating area in the target map according to the target obstacle.

[0135] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0136] After collecting laser point cloud data in the environment corresponding to the target map, the embodiment of the present application determines the candidate obstacle in the target map according to the laser point cloud data, then obtains the characteristic image of the candidate obstacle, and determines the target obstacle in the candidate obstacle based on the characteristic image, and finally, divides the working area and the non-working area in the target map according to the target obstacle. In the mowing scheme provided by the present application, the candidate obstacle is determined in the target map using the laser point cloud data, and the target obstacle is determined according to the characteristic image of the candidate obstacle, and finally, the working area and the non-working area are divided in the target map according to the target obstacle, that is, through the combination of image vision technology and laser point cloud technology, the problem of missing or wrong division caused by manual division of the working area and the non-working area can be avoided, thereby improving the efficiency of the construction of the working map; in addition, since the laser radar uniformly scans the working environment, the positions of all obstacles can be determined at one time, without the need to circle the obstacles one by one, avoiding the problem of slow overall circle speed of obstacles and low map construction efficiency due to the large volume of obstacles.

[0137] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0138] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the operation map construction methods provided in the embodiment of the present application. For example, the instructions can execute the following steps:

[0139] Collect laser point cloud data in the environment corresponding to the target map, determine candidate obstacles in the target map based on the laser point cloud data, obtain feature images of the candidate obstacles, and determine the target obstacle among the candidate obstacles based on the feature images, and divide the operating area and non-operating area in the target map according to the target obstacle.

[0140] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0141] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0142] Since the instructions stored in the storage medium can execute the steps in any one of the work map construction methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the work map construction methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0143] The above is a detailed introduction to a work map construction method, device, lawn mowing robot and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for constructing a work map, characterized in that: include: Collect laser point cloud data in the environment corresponding to the target map; Acquire a two-dimensional map coordinate system of the target map, extract a reflection value and a three-dimensional coordinate corresponding to each three-dimensional laser point from the laser point cloud data, determine a three-dimensional point cloud coordinate system corresponding to the laser point cloud data, determine a conversion relationship between the map coordinate system and the point cloud coordinate system according to a focal length and a principal point of an image acquisition device, render the reflection value corresponding to the three-dimensional laser point into the target map based on the three-dimensional coordinate corresponding to the three-dimensional laser point and the conversion relationship, and determine a candidate obstacle in the target map according to a pixel value in the rendered target map; Acquire a feature image of the candidate obstacle, and determine a target obstacle among the candidate obstacles based on the feature image; According to the target obstacle, an operation area and a non-operation area are divided in the target map.

2. The method according to claim 1, characterized in that The rendering of the reflection value corresponding to the three-dimensional laser point into the target map based on the three-dimensional coordinates corresponding to the three-dimensional laser point and the conversion relationship includes: Based on the conversion relationship between the map coordinate system and the point cloud coordinate system, the three-dimensional coordinates corresponding to the three-dimensional laser point are converted to obtain the map coordinates of the three-dimensional laser point in the target map; According to the map coordinates of the three-dimensional laser point in the target map, the reflection value corresponding to the three-dimensional laser point is rendered into the target map.

3. The method according to claim 1 or 2, characterized in that: The determining a target obstacle among the candidate obstacles based on the feature image comprises: Inputting the feature image into a preset image classification network to obtain a classification label of the candidate obstacle; The candidate obstacle whose classification label is the target label is determined as the target obstacle.

4. The method according to claim 1 or 2, characterized in that: The dividing of the operating area and the non-operating area in the target map according to the target obstacle comprises: At least obtaining contour information of the target obstacle below a preset height; Outputting an isolation curve surrounding the target obstacle according to the contour information and the position of the target obstacle in the target map; The area within the isolation curve is determined as a non-operating area, and the area other than the non-operating area is determined as an operating area.

5. The method according to claim 1, characterized in that After dividing the operating area and the non-operating area in the target map according to the target obstacle, the method further includes: highlighting the working area using a first color, and; The non-operating area is highlighted in a second color.

6. A work map construction device, characterized in that: include: The acquisition module is used to collect laser point cloud data in the environment corresponding to the target map; A first determination module is used to obtain a two-dimensional map coordinate system of the target map, extract a reflection value and a three-dimensional coordinate corresponding to each three-dimensional laser point from the laser point cloud data, determine a three-dimensional point cloud coordinate system corresponding to the laser point cloud data, determine a conversion relationship between the map coordinate system and the point cloud coordinate system according to a focal length and a principal point of an image acquisition device, render the reflection value corresponding to the three-dimensional laser point into the target map based on the three-dimensional coordinate corresponding to the three-dimensional laser point and the conversion relationship, and determine a candidate obstacle in the target map according to a pixel value in the rendered target map; An acquisition module, used to acquire a feature image of the candidate obstacle; A second determination module, configured to determine a target obstacle among the candidate obstacles based on the feature image; A division module is used to divide the target map into an operation area and a non-operation area according to the target obstacle.

7. A lawn mowing robot, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the work map construction method according to any one of claims 1 to 5 are implemented.

8. A storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the steps of the work map construction method according to any one of claims 1 to 5 are implemented.

Citation Information

Cited By

  • Operation map construction method and apparatus, mowing robot, and storage medium

    EP4535114A1

  • Operation map construction method and apparatus, mowing robot, and storage medium

    WO2024008016A1