Edge path planning method and device, computer readable medium and electronic device

Abstract

The application discloses a kind of edge path planning method, device, computer readable medium and electronic equipment, the method includes: obtaining boundary image in the direction of advance of mobile device, and the boundary pixel point of boundary image;Obtain point cloud image, wherein the point cloud image contains point cloud data;Point cloud data includes the position information of each data point and reflection intensity;Boundary image and point cloud image are transformed into matrix, to determine the candidate boundary point corresponding to boundary pixel point from point cloud data;According to the reflection intensity of candidate boundary point, determine the target boundary point corresponding to boundary pixel point from candidate boundary point;According to the position information of target boundary point, determine the edge path of mobile device.This application first locates boundary by candidate boundary point, and then further filters candidate boundary point according to reflection intensity, to determine the real target boundary point of working area, improve the positioning accuracy of the boundary of mobile device working area.

Description

Technical Field

[0001] The present application belongs to the field of image processing and artificial intelligence technology, and specifically relates to a method, device, computer-readable medium and electronic device for edge path planning. Background Art

[0002] At present, self-moving equipment has provided great convenience for people's daily work and life. For example, self-moving equipment is used for lawn maintenance, environmental cleaning, cargo handling, etc. An important part of the work of self-moving equipment is to move along the edge of the work area, which requires the self-moving equipment to accurately know the current path along the edge. The traditional method of moving along the edge is to preset a working map for the self-moving equipment, and then use RTK (Real Time Kinematic) positioning to control the self-moving equipment along the edge. However, in some cases when RTK positioning jumps, it will cause inaccurate positioning of the self-moving equipment, causing the self-moving equipment to deviate from the actual path along the edge, and the accuracy of the path along the edge planning is poor.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, computer-readable medium and electronic device for edge path planning to solve the problem of poor accuracy of edge path planning in related technologies.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a method for planning an edge path is provided, including:

[0007] Acquire a boundary image in a moving direction of the mobile device and boundary pixels of the boundary image;

[0008] Acquire a point cloud image, wherein the point cloud image includes point cloud data; the point cloud data includes position information and reflection intensity of each data point;

[0009] Performing a matrix transformation on the boundary image and the point cloud image to determine candidate boundary points corresponding to the boundary pixel points from the point cloud data;

[0010] Determining a target boundary point corresponding to the boundary pixel point from the candidate boundary points according to the reflection intensity of the candidate boundary points;

[0011] The path along the edge of the mobile device is determined according to the location information of the target boundary point.

[0012] According to one aspect of an embodiment of the present application, a device for planning an edge path is provided, comprising:

[0013] A boundary image acquisition module, configured to acquire a boundary image in the direction in which the mobile device moves, and boundary pixels of the boundary image;

[0014] A point cloud image acquisition module, configured to acquire a point cloud image, wherein the point cloud image includes point cloud data; the point cloud data includes position information and reflection intensity of each data point;

[0015] an image transformation module, configured to perform matrix transformation on the boundary image and the point cloud image, so as to determine candidate boundary points corresponding to the boundary pixels from the point cloud data;

[0016] a target boundary point determination module, configured to determine a target boundary point corresponding to the boundary pixel point from the candidate boundary points according to the reflection intensity of the candidate boundary points;

[0017] The edge path determination module is used to determine the edge path of the mobile device according to the position information of the target boundary point.

[0018] In one embodiment of the present application, the image transformation module is specifically used to:

[0019] Performing a matrix transformation on the boundary image and the point cloud image to determine a reference point corresponding to the boundary pixel point from the point cloud data;

[0020] The point cloud data within a preset neighborhood range corresponding to the reference point is determined as a candidate boundary point corresponding to the boundary pixel point.

[0021] In one embodiment of the present application, the target boundary point determination module is specifically configured to:

[0022] Calculating the difference in reflection intensity between each two candidate boundary points;

[0023] The target boundary point is determined from the candidate boundary points corresponding to the maximum difference.

[0024] In one embodiment of the present application, the target boundary point determination module is further configured to:

[0025] From the two candidate boundary points corresponding to the maximum difference value, the candidate boundary point closest to the mobile device is extracted as the target boundary point.

[0026] In one embodiment of the present application, the edge path determination module is specifically configured to:

[0027] Obtaining an extrinsic parameter matrix for converting between the coordinate system where the target boundary point is located and the global coordinate system;

[0028] According to the extrinsic parameter matrix and the position information of the target boundary point, the target boundary point is converted to the global coordinate system to obtain the global position information of the target boundary point;

[0029] An edge path is constructed according to the global position information.

[0030] In one embodiment of the present application, the boundary image acquisition module is specifically configured to:

[0031] Acquire an original image in the forward direction of the mobile device, wherein the original image includes a working area and a non-working area;

[0032] A boundary between the working area and the non-working area in the original image is extracted to obtain a boundary image.

[0033] In one embodiment of the present application, the apparatus further comprises:

[0034] The boundary point sorting module is used to calculate the distance between each target boundary point and the self-moving device; and sort each target boundary point according to the distance between each target boundary point and the self-moving device.

[0035] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the edge path planning method in the above technical solution is implemented.

[0036] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions so that the electronic device executes the edge path planning method as in the above technical solution.

[0037] According to one aspect of an embodiment of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the edge path planning method described in the above technical solution.

[0038] In the technical solution provided in the embodiment of the present application, a boundary image in the forward direction of the self-mobile device and a point cloud image obtained by the self-mobile device are obtained, the point cloud image including the position information and reflection intensity of each data point, and then the boundary image and the point cloud image are matrix-transformed to determine the candidate boundary points corresponding to the boundary pixel points from the point cloud data, and based on the reflection intensity of the candidate boundary points, the target boundary points corresponding to the boundary pixel points are determined from the candidate boundary points, and then the position information of the target boundary points is used to determine the path along the edge of the self-mobile device. The method of the present application is equivalent to first preliminarily locating the boundary of the working area of ​​the self-mobile device through the boundary image, determining the candidate boundary points, and then further screening the candidate boundary points based on the reflection intensity to determine the true target boundary points of the working area, thereby improving the positioning accuracy of the boundary of the working area of ​​the self-mobile device.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0041] Figure 1 The schematic diagram shows the structure of a self-moving device provided by the implementation of this application.

[0042] Figure 2 The flowchart of the edge path planning method provided by one embodiment of the present application is schematically shown.

[0043] Figure 3A A schematic diagram of an original image provided by an embodiment of the present application is schematically shown.

[0044] Figure 3B The figure schematically shows a schematic diagram of image segmentation provided by one embodiment of the present application.

[0045] Figure 3C The diagram schematically shows a boundary image provided by an embodiment of the present application.

[0046] Figure 4 The structural block diagram of the edge path planning device provided in an embodiment of the present application is schematically shown.

[0047] Figure 5 The following schematically shows a block diagram of a computer system structure of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0048] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0049] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0052] In one embodiment of the present application, the technical solution of the present application is implemented by a terminal device. The terminal device obtains a boundary image in the direction of advance of the self-mobile device, and the boundary pixel points of the boundary image; at the same time, the terminal device obtains a point cloud image, wherein the point cloud image contains point cloud data; the point cloud data includes the position information and reflection intensity of each data point. The terminal device then performs a matrix transformation on the boundary image and the point cloud image to determine the candidate boundary points corresponding to the boundary pixel points from the point cloud data. Next, the terminal device determines the target boundary points corresponding to the boundary pixel points from the candidate boundary points based on the reflection intensity of the candidate boundary points. Finally, the terminal device determines the edge path of the self-mobile device based on the position information of the target boundary point.

[0053] For example, the terminal device is a self-moving device. Figure 1The structural diagram of a self-moving device provided by the implementation of the present application is schematically shown. The self-moving device 100 may be a device including a self-moving auxiliary function, and the self-moving auxiliary function may be implemented by a vehicle-mounted terminal. The corresponding self-moving device may be a vehicle with the vehicle-mounted terminal. The self-moving device may also be a semi-autonomous device or a fully autonomous mobile device. For example, various types of mobile robots, such as sweeping robots, mopping robots, food delivery robots, transport robots, lawn mowing robots, mobile robots with navigation functions, etc., the embodiment of the present application does not limit the specific type and function of the self-moving device 100. It can be understood that the self-moving device 100 in this embodiment may also include other devices with self-moving functions.

[0054] In an embodiment of the present application, a camera device 110 is provided on the self-mobile device 100, and the camera device 110 can obtain a boundary image and a point cloud image in the forward direction of the self-mobile device, wherein the boundary image includes a plurality of boundary pixel points; the point cloud image includes point cloud data, and the point cloud data includes the position information of each data point and the reflection intensity. The camera device 110 can be fixed or non-fixed and rotatable, which is not limited in the embodiment of the present application. The environmental image captured by the camera device 110 can be a color image, a black and white image, an infrared image, etc., which is not limited in the embodiment of the present application. Exemplarily, the camera device 110 can include an RGB camera and a lidar sensor, and the RGB camera captures the environment in the forward direction of the self-mobile device to obtain a boundary image, and at the same time, the lidar sensor collects a point cloud image with reflection intensity.

[0055] The camera device 110 is connected to a control module 120 within the mobile device 100. The control module 120 is used to receive control instructions from the mobile device 100 or generate various control instructions for the mobile device 100. The control module 120 is also connected to the driving components of the mobile device 100, such as the steering shaft, steering wheels, and motors of the mobile device 100, to control the movement and steering of the mobile device 100.

[0056] In an embodiment of the present application, after the camera device 110 captures the obtained boundary image and point cloud image, the boundary image and point cloud image are transmitted to the control module 120. The control module 120 performs a matrix transformation on the boundary image and point cloud image, and finds candidate boundary points corresponding to the boundary pixel points in the boundary image in the point cloud data. The control module 120 then determines the target boundary point corresponding to the boundary pixel point from the candidate boundary points based on the reflection intensity of the candidate boundary point, and determines the edge path of the self-moving device based on the position information of the target boundary point. After determining the edge path, the control module 120 generates a control instruction based on the edge path to control the self-moving device to move along the edge according to the edge path.

[0057] The following describes in detail the edge path planning method provided by this application in conjunction with specific implementation methods.

[0058] Figure 2 The flowchart of the edge path planning method provided by one embodiment of the present application is schematically shown. Figure 2 As shown, the method includes steps 210 to 250, which are specifically as follows:

[0059] Step 210: Acquire a boundary image in the direction in which the mobile device moves, and boundary pixels of the boundary image.

[0060] Specifically, when a mobile device moves within the working area, the area outside the working area is called the non-working area. The boundary between the working area and the non-working area is the boundary of the mobile device during movement, and also the boundary of the working area. The boundary image refers to an image that includes the boundary of the mobile device's working area. The pixels in the boundary image that represent the boundary are called boundary pixels.

[0061] In one embodiment of the present application, the specific process of obtaining a boundary image may include: obtaining an original image in the forward direction of the mobile device, the original image including a working area and a non-working area; extracting the boundary between the working area and the non-working area in the original image to obtain a boundary image.

[0062] Specifically, the physical environment in the direction of travel of the mobile device is photographed, resulting in an original image. The original image includes an active area and a non-active area. The boundary between the active area and the non-active area is then extracted from the original image to form a boundary image. Specifically, the original image can be segmented to separate the active area from the non-active area. For example, pixels representing the active area in the original image are marked as 1, and pixels representing the non-active area are marked as 0. Multiple boundary pixels representing the boundary between the active area and the non-active area are then extracted from the segmented image to form a boundary image. For example, the pixels in the segmented image are traversed to identify whether each pixel's neighborhood includes two different types of pixels, where the type of pixel refers to whether the pixel is in the active area or the non-active area. If a pixel's neighborhood includes two different types of pixels, the pixel is considered a boundary pixel. For example, if a pixel's neighborhood includes both pixels marked as 0 and pixels marked as 1, the pixel is considered a boundary pixel.

[0063] For example, Figure 3A The following schematically shows a schematic diagram of an original image provided by an embodiment of the present application. Figure 3AAs shown, the original image includes a working area 310 and a non-working area 320. The original image is segmented, and the pixel values ​​of the pixels representing the working area 310 in the original image are set to 255, showing white; the pixel values ​​of the pixels representing the non-working area 320 in the original image are set to 0, showing black. The segmented image is shown in FIG. Figure 3B Then traverse Figure 3B In the pixel points of the segmented image shown in the figure, when there are both pixel points with pixel values ​​of 0 and pixel values ​​of 255 in the neighborhood of a pixel point, then the pixel point is a boundary pixel point, and the following is obtained: Figure 3C Boundary image shown.

[0064] In one embodiment of the present application, the image after image segmentation processing can be directly obtained as the original image, and then boundary pixels are extracted therefrom to form a boundary image.

[0065] Step 220: Acquire a point cloud image, wherein the point cloud image contains point cloud data; the point cloud data includes position information and reflection intensity of each data point.

[0066] Specifically, a point cloud image can be obtained by photographing the physical environment in the direction of travel of the self-mobile device through a laser radar sensor. The information contained in the point cloud image can be called point cloud data, and the point cloud data includes the position information and reflection intensity of each data point. The position information of a data point refers to the position coordinate information of the physical point corresponding to the data point in the physical environment in the laser radar coordinate system. The reflection intensity of a data point reflects the light reflection ability of the physical point corresponding to the data point, and can also be called reflectivity. Among them, the physical points described in the embodiments of the present application can be points that constitute dynamic objects or static objects in the physical environment. For example, the physical points can be points of trees, stones, grass, etc., which are not limited here.

[0067] Step 230: Perform matrix transformation on the boundary image and the point cloud image to determine candidate boundary points corresponding to boundary pixels in the point cloud data.

[0068] Specifically, performing a matrix transformation on the boundary image and the point cloud image refers to processing the points in the image (for example, one or more of the processes such as rotation, translation, and correction), projecting the boundary image to the point cloud image, or projecting the point cloud image to the boundary image. The matrix transformation can make the boundary pixel points in the boundary image correspond to the data points in the point cloud image, that is, find the regional point cloud data corresponding to the boundary pixel points, and the regional point cloud data is recorded as the candidate boundary point. It should be noted that the regional point cloud data in this embodiment refers to the point cloud data composed of multiple data points, which means that one boundary pixel point can correspond to multiple data points in the point cloud image, and the multiple data points are all recorded as candidate boundary points corresponding to the boundary pixel point.

[0069] Finding candidate boundary points corresponding to boundary pixels through matrix transformation is equivalent to preliminarily locating the true boundary of the working area of ​​the self-mobile device, that is, the preliminarily located true boundary is composed of candidate boundary points.

[0070] In one embodiment of the present application, the process of determining candidate boundary pixel points includes: performing matrix transformation on the boundary image and the point cloud image to determine the reference point corresponding to the boundary pixel point from the point cloud data; and determining the point cloud data within a preset neighborhood range corresponding to the reference point as the candidate boundary point corresponding to the boundary pixel point.

[0071] Specifically, taking the projection of the boundary image to the point cloud image as an example, the matrix transformation can be performed according to the following formula:

[0072] P lidar =K*(R*P camera +T)

[0073] Among them, P camera Represents the pixel coordinates of the boundary pixel point, P lidar represents the point cloud data coordinates corresponding to the boundary pixel points after matrix transformation. K is the intrinsic parameter matrix of the camera device, R is the correction matrix of the camera device, and T is the rotation and translation matrix that transforms the pixel coordinate system to the lidar coordinate system, also known as the extrinsic parameter matrix. The rotation and translation matrix T can be calculated in advance using multiple sample data. It is understood that after obtaining the rotation and translation matrix T, it can be used inversely to project the point cloud image onto the boundary image.

[0074] P obtained by the above matrix transformation lidar The value is the reference point corresponding to the boundary pixel point in the point cloud data. It can be seen that the reference point is the data point closest to the boundary pixel point in the point cloud data, that is, the data point that can best be represented as the boundary pixel point in the point cloud data. It should be noted that P lidar The value is the abbreviation of the point cloud data coordinates of a point obtained by transformation, which does not mean that the point cloud data coordinates have only one coordinate value. The point cloud data coordinates are usually the position coordinates in the three-dimensional coordinate system, so P lidar The value is a short form of a 3D position coordinate.

[0075] In one embodiment of the present application, P obtained by the above matrix transformation is lidar value, it may not be possible to find a data point that is completely consistent with it in the point cloud image. In this case, the data point in the point cloud image that is consistent with the P lidar The data point with the closest value is used as the reference point for the boundary pixel.

[0076] After determining the reference point of the boundary pixel, the point cloud data within a preset neighborhood corresponding to the reference point is determined as a candidate boundary point corresponding to the boundary pixel. For example, the point cloud data within a circular or spherical area formed with a preset radius and centered at the reference point is used as the candidate boundary point corresponding to the boundary pixel.

[0077] Step 240: Determine a target boundary point corresponding to the boundary pixel point from the candidate boundary points based on the reflection intensity of the candidate boundary points.

[0078] Specifically, target boundary points are extracted from candidate boundary points based on reflection intensity. These target boundary points are points on the actual boundary of the mobile device's operating area. Target boundary points are located at the junction of the operating and non-operating areas. Points at these junctions typically have a strong ability to reflect light. Therefore, based on the variation in reflection intensity at the candidate boundary points, candidate boundary points with significant variations in reflection intensity are selected as target boundary points.

[0079] In one embodiment of the present application, the process of determining the target boundary point includes: calculating the difference in reflection intensity between every two candidate boundary points; and determining the target boundary point from the candidate boundary points corresponding to the maximum difference.

[0080] Specifically, the difference in reflection intensity between candidate boundary points is calculated. The two candidate boundary points corresponding to the largest difference are the ones with the greatest change in reflection intensity. These two candidate boundary points are then used as the target boundary points corresponding to the boundary pixels. By screening candidate boundary points based on reflection intensity to obtain the target boundary points, the boundary between the active and inactive areas can be accurately identified, thereby improving the accuracy of the path along the edge.

[0081] Generally speaking, there should be a one-to-one correspondence between boundary pixels and target boundary points. Therefore, after obtaining the two candidate boundary points with the largest change in reflection intensity, the candidate boundary point closest to the mobile device can be selected as the target boundary point. For a mobile device, its along-edge path should be within the working area. Therefore, the candidate boundary point closest to the mobile device is the boundary point closest to the working area and is therefore a more suitable target boundary point. This further improves the accuracy of the along-edge path.

[0082] Step 250: Determine the path along the edge of the mobile device based on the location information of the target boundary point.

[0083] Specifically, the position information of the target boundary point is the position coordinate of the target boundary point in the point cloud camera coordinate system. By converting the position coordinate into the global coordinate system, the edge path of the mobile device can be generated.

[0084] In the technical solution provided in the embodiment of the present application, a boundary image and a point cloud image in the forward direction of the self-mobile device are obtained, and the point cloud image includes the position information and reflection intensity of each data point. Then, the boundary image and the point cloud image are matrix transformed to determine the candidate boundary points corresponding to the boundary pixel points from the point cloud data, and according to the reflection intensity of the candidate boundary points, the target boundary points corresponding to the boundary pixel points are determined from the candidate boundary points, and then the position information of the target boundary points is used to determine the edge path of the self-mobile device; the candidate boundary points in the present application are equivalent to first preliminarily locating the boundary of the working area of ​​the self-mobile device through the boundary image, and then further screening the boundary points according to the reflection intensity to determine the real target boundary points of the working area, thereby improving the positioning accuracy of the boundary of the working area of ​​the self-mobile device.

[0085] In one embodiment of the present application, the specific process of determining the edge path of a mobile device based on the position information of the target boundary point includes: obtaining an extrinsic parameter matrix for converting between the coordinate system where the target boundary point is located and the global coordinate system; based on the extrinsic parameter matrix and the position information of the target boundary point, converting the target boundary point to the global coordinate system to obtain the global position information of the target boundary point; and constructing the edge path based on the global position information.

[0086] Specifically, the extrinsic parameter matrix used to transform between the coordinate system of the target boundary point and the global coordinate system refers to a rotation matrix and a translation matrix used to transform between the coordinate system of the target boundary point and the global coordinate system. Based on the extrinsic parameter matrix, at least one of a rotation transformation and a translation transformation is performed on the position information of the target boundary point to obtain the global position information of the target boundary point, and then an edge path is generated based on the global position information of the target boundary point.

[0087] In one embodiment of the present application, the target boundary point can be converted to the global coordinate system according to the following formula:

[0088] Y=(M*X+N)+X0

[0089] Where X represents the position information of the target boundary point; Y represents the global position information of the target boundary point after transformation; M represents the rotation transformation matrix, N represents the translation transformation matrix, and M and N are collectively referred to as the extrinsic parameter matrix; X0 represents the current position information of the mobile device in the global coordinate system.

[0090] In one embodiment of the present application, when the point cloud camera and the self-moving device are rigid bodies, the relative position of the point cloud camera and the self-moving device will not change. At this time, M=1, N=0, and Y=X+X0.

[0091] According to the coordinate conversion process, this application converts the position information of the boundary pixel points based on the current position information of the self-mobile device in the global coordinate system. The current position information of the self-mobile device in the global coordinate system can be obtained through RTK positioning. When the RTK positioning jumps, the position information of the boundary pixel points at the two moments before and after can be compared to obtain the amplitude of the RTK positioning jump, so that the RTK positioning can be corrected.

[0092] In one embodiment of the present application, after obtaining the global position information of each target boundary point, it also includes: calculating the distance between each target boundary point and the self-mobile device; and sorting each target boundary point according to the distance between each target boundary point and the self-mobile device.

[0093] Specifically, the distance from each target boundary point to the mobile device is calculated, and the target boundary points are sorted from closest to farthest to generate an edge path for the mobile device. This allows the mobile device to move along the shortest edge path, improving its edge efficiency. Furthermore, the edge path formed by multiple target boundary points can predict the mobile device's movement direction, allowing the mobile device to prepare for movement in advance, further improving edge efficiency.

[0094] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0095] The following introduces an embodiment of the device of the present application, which can be used to execute the edge path planning method in the above embodiment of the present application. Figure 4 The following schematically shows a block diagram of the structure of the edge path planning device provided in an embodiment of the present application. Figure 4 As shown, the edge path planning device provided in the embodiment of the present application includes:

[0096] The boundary image acquisition module 410 is used to acquire a boundary image in the direction in which the mobile device moves, and boundary pixels of the boundary image;

[0097] The point cloud image acquisition module 420 is used to acquire a point cloud image, wherein the point cloud image includes point cloud data; the point cloud data includes the position information and reflection intensity of each data point;

[0098] An image transformation module 430 is configured to perform a matrix transformation on the boundary image and the point cloud image to determine candidate boundary points corresponding to the boundary pixels from the point cloud data;

[0099] a target boundary point determination module 440 for determining a target boundary point corresponding to the boundary pixel point from the candidate boundary points according to the reflection intensity of the candidate boundary points;

[0100] The edge path determination module 450 is configured to determine an edge path of the mobile device according to the location information of the target boundary point.

[0101] In one embodiment of the present application, the image transformation module 430 is specifically configured to:

[0102] Performing a matrix transformation on the boundary image and the point cloud image to determine a reference point corresponding to the boundary pixel point from the point cloud data;

[0103] The point cloud data within a preset neighborhood range corresponding to the reference point is determined as a candidate boundary point corresponding to the boundary pixel point.

[0104] In one embodiment of the present application, the target boundary point determination module 440 is specifically configured to:

[0105] Calculating the difference in reflection intensity between each two candidate boundary points;

[0106] The target boundary point is determined from the candidate boundary points corresponding to the maximum difference.

[0107] In one embodiment of the present application, the target boundary point determination module 440 is further configured to:

[0108] From the two candidate boundary points corresponding to the maximum difference value, the candidate boundary point closest to the mobile device is extracted as the target boundary point.

[0109] In one embodiment of the present application, the edge path determination module 450 is specifically configured to:

[0110] Obtaining an extrinsic parameter matrix for converting between the coordinate system where the target boundary point is located and the global coordinate system;

[0111] According to the extrinsic parameter matrix and the position information of the target boundary point, the target boundary point is converted to the global coordinate system to obtain the global position information of the target boundary point;

[0112] An edge path is constructed according to the global position information.

[0113] In one embodiment of the present application, the boundary image acquisition module 410 is specifically configured to:

[0114] Acquire an original image in the forward direction of the mobile device, wherein the original image includes a working area and a non-working area;

[0115] A boundary between the working area and the non-working area in the original image is extracted to obtain a boundary image.

[0116] In one embodiment of the present application, the apparatus further comprises:

[0117] The boundary point sorting module is used to calculate the distance between each target boundary point and the self-moving device; and sort each target boundary point according to the distance between each target boundary point and the self-moving device.

[0118] The specific details of the edge path planning device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0119] Figure 5 The block diagram schematically shows a computer system structure of an electronic device used to implement an embodiment of the present application.

[0120] It should be noted that Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the random access memory 503. The CPU 501, the read-only memory 502, and the random access memory 503 are connected to each other via a bus 504. An input / output interface 505 (i.e., an I / O interface) is also connected to the bus 504.

[0122] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a local area network card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that a computer program read therefrom can be installed into the storage section 508 as needed.

[0123] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 509 and / or installed from a removable medium 511. When the computer program is executed by the central processing unit 501, the various functions defined in the system of the present application are performed.

[0124] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0127] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0128] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0129] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for planning a path along an edge, characterized in that: include: Acquire a boundary image in the direction of travel of the mobile device and boundary pixels of the boundary image; wherein, when the neighborhood pixels of a pixel point include pixels of the working area and pixels of the non-working area, the pixel point is considered to be a boundary pixel point; Acquire a point cloud image, wherein the point cloud image includes point cloud data; the point cloud data includes position information and reflection intensity of each data point; Performing a matrix transformation on the boundary image and the point cloud image to determine candidate boundary points corresponding to the boundary pixel points from the point cloud data; Calculating the difference in reflection intensity between each two candidate boundary points; Determine the target boundary point from the candidate boundary points corresponding to the maximum difference; The edge path of the self-moving device is determined according to the position information of the target boundary point; the edge path is the movement path of the self-moving device along the edge of the working area.

2. The edge path planning method according to claim 1, characterized in that: The performing matrix transformation on the boundary image and the point cloud image to determine candidate boundary points corresponding to the boundary pixel points from the point cloud data includes: Performing a matrix transformation on the boundary image and the point cloud image to determine a reference point corresponding to the boundary pixel point from the point cloud data; The point cloud data within a preset neighborhood range corresponding to the reference point is determined as a candidate boundary point corresponding to the boundary pixel point.

3. The edge path planning method according to claim 2, characterized in that: The determining of the target boundary point from the candidate boundary points corresponding to the maximum difference includes: From the two candidate boundary points corresponding to the maximum difference value, the candidate boundary point closest to the mobile device is extracted as the target boundary point.

4. The edge path planning method according to claim 1, characterized in that: The determining of the edge path of the mobile device according to the location information of the target boundary point includes: Obtaining an extrinsic parameter matrix for converting between the coordinate system where the target boundary point is located and the global coordinate system; According to the extrinsic parameter matrix and the position information of the target boundary point, the target boundary point is converted to the global coordinate system to obtain the global position information of the target boundary point; An edge path is constructed according to the global position information.

5. The edge path planning method according to claim 1, characterized in that: The boundary image obtained from the moving direction of the mobile device includes: Acquire an original image in the forward direction of the mobile device, wherein the original image includes a working area and a non-working area; A boundary between the working area and the non-working area in the original image is extracted to obtain a boundary image.

6. The method for planning a path along an edge according to any one of claims 1 to 5, wherein: Before determining the edge path of the mobile device according to the location information of the target boundary point, the method further includes: Calculating the distance from each target boundary point to the mobile device; Each of the target boundary points is sorted according to the distance from each of the target boundary points to the self-moving device.

7. A device for planning a path along an edge, characterized in that: include: A boundary image acquisition module is configured to acquire a boundary image in the direction in which the mobile device is moving, and boundary pixels of the boundary image; wherein a pixel is considered a boundary pixel if its neighborhood includes pixels in the working area and pixels in the non-working area; A point cloud image acquisition module, configured to acquire a point cloud image, wherein the point cloud image includes point cloud data; the point cloud data includes position information and reflection intensity of each data point; an image transformation module, configured to perform matrix transformation on the boundary image and the point cloud image, so as to determine candidate boundary points corresponding to the boundary pixels from the point cloud data; a target boundary point determination module, configured to calculate the difference in reflection intensity between each two candidate boundary points; and determine the target boundary point from the candidate boundary points corresponding to the maximum difference; The edge path determination module is used to determine the edge path of the self-moving device according to the position information of the target boundary point; the edge path is the movement path of the self-moving device along the edge of the working area.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for planning an edge path according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor executes the executable instructions so that the electronic device executes the edge path planning method according to any one of claims 1 to 6.

Images