Obstacle avoidance method and device, self-moving equipment and computer program product

Abstract

The invention is suitable for the technical field of autonomous movement, and provides an obstacle avoidance method and device, self-moving equipment and a computer program product. The obstacle avoidance method comprises the following steps: acquiring an outer edge of a drivable area in real time based on a road surface image; an obstacle area is updated in real time, a self-moving device is controlled to execute obstacle avoidance operation on the obstacle area, the obstacle area comprises an area defined by the outer edge of the drivable area and a view field range corresponding to a preset view field angle, and the preset view field angle is smaller than a camera collection view angle of the road surface image. According to the embodiment of the invention, obstacle avoidance can be realized for obstacles which are not in the predefined obstacle category, and the safety of the autonomous movement process is improved.

Description

Technical Field

[0001] The present application belongs to the field of autonomous mobility technology, and in particular relates to an obstacle avoidance method, apparatus, autonomous mobility device, and computer program product. Background Art

[0002] With the advancement of artificial intelligence, autonomous mobility for devices like robots and cars, without human intervention, has become a reality. Technologies for autonomous mobility often rely on object detection algorithms. These algorithms detect obstacles in the vehicle's surroundings and enable accurate obstacle avoidance. However, in real-world applications, object detection can fail. Specifically, unclassified obstacles may appear on the road, preventing the autonomous device from identifying and accurately avoiding them. Summary of the Invention

[0003] The embodiments of the present application provide an obstacle avoidance method, apparatus, autonomous mobile device, and computer program product, which can avoid obstacles that are not in predefined obstacle categories and improve the safety of the autonomous movement process.

[0004] A first aspect of an embodiment of the present application provides an obstacle avoidance method, comprising: obtaining the outer edge of a drivable area in real time based on a road surface image; updating an obstacle area in real time, and controlling a mobile device to perform obstacle avoidance operations on the obstacle area, wherein the obstacle area includes an area enclosed by the outer edge of the drivable area and a field of view corresponding to a preset field of view angle, wherein the preset field of view angle is smaller than the camera acquisition angle of the road surface image.

[0005] In some embodiments of the first aspect, the real-time acquisition of the outer edge of the drivable area based on the road surface image includes: acquiring the drivable area based on the road surface image; performing edge detection on the drivable area to obtain the edge of the drivable area; and classifying the edge of the drivable area to obtain the outer edge.

[0006] In some embodiments of the first aspect, classifying the edges of the drivable area to obtain the outer edge includes: obtaining the minimum circumscribed rectangle corresponding to each edge of the drivable area; in response to the area of the minimum circumscribed rectangle being greater than a first area threshold, using the corresponding edge as the outer edge.

[0007] In some embodiments of the first aspect, the obstacle area also includes an area enclosed by the edges of obstacles within the drivable area; after obtaining the minimum circumscribed rectangle corresponding to each edge of the drivable area, it also includes: in response to the area of the minimum circumscribed rectangle being greater than a second area threshold, and the number of non-drivable points within the minimum circumscribed rectangle being greater than the number of drivable points, the corresponding edge is used as the edge of the obstacle within the drivable area, and the second area threshold is less than the first area threshold.

[0008] In some implementations of the first aspect, the obstacle area also includes an area where classified obstacles are located; and the obstacle avoidance method further includes: performing obstacle recognition based on the road surface image to obtain the classified obstacles.

[0009] In some embodiments of the first aspect, controlling the self-mobile device to perform an obstacle avoidance operation on the obstacle area includes: obtaining image coordinates of edge points of the obstacle area; converting the image coordinates of the edge points into the depth of the edge points based on a mapping relationship between image coordinates and depth; obtaining radar coordinates of the edge points based on the image coordinates of the edge points, the depth of the edge points, the intrinsic parameters of the camera, and the extrinsic parameters between the camera and the radar; and controlling the self-mobile device to perform the obstacle avoidance operation based on the radar coordinates of the edge points.

[0010] In some embodiments of the first aspect, before obtaining the radar coordinates of the edge point based on the image coordinates of the edge point, the depth of the edge point, the intrinsic parameters of the camera, and the extrinsic parameters between the camera and the radar, the method further includes: obtaining a ground point cloud collected by the radar; projecting the ground point cloud onto the road surface image; and in response to the ground point cloud being located in the drivable area, obtaining the mapping relationship based on the image coordinates and depth of the ground point cloud.

[0011] An obstacle avoidance device provided in a second aspect of an embodiment of the present application includes: an edge acquisition unit for acquiring the outer edge of a drivable area in real time based on a road surface image; an obstacle avoidance unit for updating an obstacle area in real time and controlling a mobile device to perform obstacle avoidance operations on the obstacle area, wherein the obstacle area includes an area enclosed by the outer edge of the drivable area and a field of view corresponding to a preset field of view angle, wherein the preset field of view angle is smaller than the camera acquisition angle of the road surface image.

[0012] A third aspect of an embodiment of the present application provides a self-mobile device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned obstacle avoidance method when executing the computer program.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned obstacle avoidance method are implemented.

[0014] A fifth aspect of the embodiments of the present application provides a computer program product, which, when the computer program is run, enables the above-mentioned obstacle avoidance method to be executed.

[0015] In an embodiment of the present application, based on the road image, the outer edge of the drivable area is obtained in real time, the obstacle area is updated in real time, and the self-mobile device is controlled to perform obstacle avoidance operations on the obstacle area. Since the obstacle area includes the area formed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle, and the preset field of view angle is smaller than the camera acquisition angle of the road image, for larger and irregular obstacles, even if the full image cannot be obtained and classified through the road image, the obstacle area is formed based on the existing outer edge and the field of view corresponding to the preset field of view angle. The part of the obstacle closest to the self-mobile device can be fitted, and the obstacle avoidance operation can be performed safely. In the process of obstacle avoidance, the full image of the obstacle is gradually seen and the obstacle area is updated, and the cycle is repeated until the obstacle is bypassed. Therefore, obstacles that are not in the pre-defined obstacle category can be bypassed, thereby improving the safety of the autonomous movement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a schematic diagram of an implementation flow of an obstacle avoidance method provided in an embodiment of the present application;

[0018] Figure 2 Schematic diagram of the area enclosed by the outer edge and the field of view corresponding to the preset field of view angle provided in an embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of a specific implementation process for obtaining an outer edge provided in an embodiment of the present application;

[0020] Figure 4 This is a first schematic diagram of a segmentation mask provided in an embodiment of the present application;

[0021] Figure 5 is a second schematic diagram of a segmentation mask provided in an embodiment of the present application;

[0022] Figure 6This embodiment of the present application provides Figure 5 Corresponding visual image diagram;

[0023] Figure 7 This is a schematic diagram of a specific implementation flow of controlling the obstacle avoidance operation of a mobile device provided in an embodiment of the present application;

[0024] Figure 8 This embodiment of the present application provides Figure 4 Corresponding visual image diagram;

[0025] Figure 9 This embodiment of the present application provides Figure 4 Schematic diagram of the corresponding point cloud image;

[0026] Figure 10 This is a schematic structural diagram of an obstacle avoidance device provided in an embodiment of the present application;

[0027] Figure 11 It is a structural diagram of a self-moving device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without making any creative work are protected by this application.

[0029] With the advancement of artificial intelligence, autonomous mobility for devices like robots and cars, without human intervention, has become a reality. Technologies for autonomous mobility often rely on object detection algorithms. These algorithms detect obstacles in the vehicle's surroundings and enable accurate obstacle avoidance. However, in real-world applications, object detection can fail. Specifically, unclassified obstacles may appear on the road, preventing the autonomous device from identifying and accurately avoiding them.

[0030] In view of this, the present application proposes an obstacle avoidance method that can fit the part of the obstacle closest to the mobile device, safely perform obstacle avoidance operations, and gradually see the full picture of the obstacle and update the obstacle area during the obstacle avoidance process, and loop until the obstacle is bypassed. Therefore, obstacles that are not in the pre-defined obstacle category can be avoided, thereby improving the safety of the autonomous movement process.

[0031] In order to illustrate the technical solution of the present application, specific embodiments are provided below.

[0032] Figure 1The following is a schematic diagram illustrating an implementation flow of an obstacle avoidance method provided in an embodiment of the present application, which can be applied to a self-moving device. The self-moving device refers to a device with autonomous mobility capabilities, which can be a robot, a smart car, a drone, etc., and this application does not limit this.

[0033] Specifically, the above obstacle avoidance method may include the following steps S101 to S102.

[0034] Step S101 : acquiring the outer edge of the drivable area in real time based on the road surface image.

[0035] In the embodiments of the present application, the road surface image is an image captured by a camera on a mobile device and is used to identify obstacles on the road surface on which the mobile device is located. The drivable area is the area within which the mobile device can safely travel during autonomous movement. The outer edge refers to the outermost edge of the drivable area. Based on the road surface image, the outer edge of the drivable area can be obtained from the image in real time.

[0036] Step S102: updating the obstacle area in real time and controlling the mobile device to perform obstacle avoidance operations on the obstacle area.

[0037] The obstacle area may include the area enclosed by the outer edge of the drivable area and the field of view corresponding to a preset field of view (FOV). The preset field of view is smaller than the camera angle used to capture the road image. The camera angle used to capture the road image is the field of view of the mobile device's camera.

[0038] Since obstacle avoidance usually requires a bounding box based on the obstacle, for large, irregular obstacles, if the obstacle is at the edge of the camera's field of view, it will be difficult for the self-mobile device to obtain the complete outline of the obstacle, form a bounding box based on the complete outline, and identify the category of the obstacle. Figure 2 , after identifying the outer edge of the drivable area, the application can Figure 2 The field of view corresponding to a smaller preset field of view angle ( Figure 2 The area enclosed by the dotted line ( Figure 2 The obstacle area is represented by the hatched area (shown in the middle). On the one hand, in the absence of a complete outline, a bounding box can be generated based on the obstacle area. On the other hand, since the outer edge of the drivable area corresponds to the part of the obstacle closest to the ego-moving device, although the obstacle size is not completely accurate, the distance to the nearest obstacle is known, thus avoiding the risk of collision with the obstacle.

[0039] During the obstacle avoidance process, the self-mobile device moves. As the self-mobile device moves, the camera captures new images of obstacles, updating the road image. In step S101, the outer edge of the drivable area, acquired in real time, is updated as the road image is updated. At this time, the area enclosed by the outer edge and the field of view corresponding to the preset field of view angle is also updated. This allows the obstacle area to be updated in real time, gradually capturing a full view of the obstacle as the device circumvents it. The self-mobile device is then controlled to perform obstacle avoidance operations within the obstacle area, ultimately safely circumventing the obstacle.

[0040] In an embodiment of the present application, based on the road image, the outer edge of the drivable area is obtained in real time, the obstacle area is updated in real time, and the self-mobile device is controlled to perform obstacle avoidance operations on the obstacle area. Since the obstacle area includes the area formed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle, and the preset field of view angle is smaller than the camera acquisition angle of the road image, for larger and irregular obstacles, even if the full image cannot be obtained and classified through the road image, the obstacle area is formed based on the existing outer edge and the field of view corresponding to the preset field of view angle. The part of the obstacle closest to the self-mobile device can be fitted, and the obstacle avoidance operation can be performed safely. In the process of obstacle avoidance, the full image of the obstacle is gradually seen and the obstacle area is updated, and the cycle is repeated until the obstacle is bypassed. Therefore, obstacles that are not in the pre-defined obstacle category can be bypassed, thereby improving the safety of the autonomous movement process.

[0041] In some embodiments of the present application, the obstacle area may include one or more of the following:

[0042] 1. The area enclosed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle;

[0043] 2. The area enclosed by the edges of obstacles within the drivable area;

[0044] 3. The area where the classified obstacles are located.

[0045] By identifying different types of obstacle areas, the comprehensiveness of obstacles can be improved, which helps to improve the autonomous movement safety of self-moving equipment.

[0046] Each type of obstacle area is described below.

[0047] For an area enclosed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle, it is first necessary to identify the outer edge of the drivable area.

[0048] Specifically, obtaining the outer edge of the drivable area in real time based on the road surface image may include: steps S301 to S303.

[0049] Step S301: Acquire a drivable area based on a road surface image.

[0050] Specifically, the road image can be input into the deep learning model to obtain the segmentation mask of the drivable area in the road image. Figure 4 As shown in , the segmentation mask image represents the drivable area and the non-drivable area through different pixel values, for example Figure 4 In the image, the points with a pixel value of 0 belong to the non-drivable area, and the points with a pixel value greater than 0 belong to the drivable area. In other implementations, the segmentation mask of the drivable area can also be obtained from the road surface image by using an image segmentation algorithm.

[0051] Step S302: performing edge detection on the drivable area to obtain the edge of the drivable area.

[0052] The edge of the drivable area can be extracted by using an edge detection algorithm. This application does not limit the edge detection algorithm.

[0053] Step S303: classify the edges of the drivable area to obtain outer edges.

[0054] Ideally, the edge of the drivable area includes only the outer edge, meaning there are no obstacles within the drivable area. In practice, the edge of the drivable area often also includes the inner edge, which is the outer edge of any obstacles within the drivable area. In other words, when edge detection is performed on the drivable area, two types of edges can be detected: the outer edge of the drivable area and the edges of any obstacles within the drivable area. By classifying the edges of the drivable area, these two types of edges can be separated.

[0055] Specifically, classifying the edges of the drivable area to obtain the outer edge may include: obtaining the minimum circumscribed rectangle corresponding to each edge of the drivable area; and in response to the area of the minimum circumscribed rectangle being greater than a first area threshold, treating the corresponding edge as the outer edge.

[0056] The first area threshold can be determined based on the size of the road image or the segmentation mask. For example, if the road image size is 1280x720, the first area threshold can be 20,000. That is, if the area of the minimum bounding rectangle corresponding to an edge line is greater than 20,000, the edge line is considered to be the outer edge of the drivable area.

[0057] At this time, the area enclosed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle can be used as the first obstacle area.

[0058] For the area enclosed by the edges of obstacles inside the drivable area, in some embodiments of the present application, after obtaining the minimum enclosing rectangle corresponding to each edge of the drivable area, it may also include: in response to the area of the minimum enclosing rectangle being greater than the second area threshold, and the number of non-drivable points within the minimum enclosing rectangle being greater than the number of drivable points, taking the corresponding edge as the edge of the obstacle inside the drivable area.

[0059] The second area threshold is smaller than the first area threshold. The specific value can also be determined based on the size of the road image or segmentation mask. For example, if the original image size is 1280x720, the first area threshold can be 900. That is, if the number of non-drivable points within the minimum bounding rectangle of a particular edge line exceeds the number of drivable points, and the area of the minimum bounding rectangle is greater than 900, the edge line is considered to be the edge of an obstacle within the drivable area. In this case, the area enclosed by the edge of the obstacle within the drivable area can be considered the second type of obstacle area.

[0060] Please refer to Figure 5 and Figure 6 , Figure 5 The segmentation mask of the drivable area and the Figure 5 The corresponding visualization diagram can be used to detect the edges of the drivable area and classify them. Figure 5 The area within the drivable area where low obstacles are located is considered an obstacle area for obstacle avoidance. This allows for effective obstacle avoidance even if the low obstacle does not fall into the pre-defined obstacle category. It is also understood that combining the area and the type of internal points in the judgment can reduce the impact of mis-segmentation of the drivable area.

[0061] For the area where the classified obstacles are located, in some embodiments of the present application, the obstacle avoidance method may further include: performing obstacle recognition based on the road surface image to obtain the classified obstacles.

[0062] Obstacle identification can be achieved using an object detection model or other existing algorithms. Classified obstacles here refer to obstacles with predefined classifications. Because the object detection model is trained based on a dataset whose data labels are predefined categories, this approach is suitable for identifying obstacles with predefined classifications.

[0063] Specifically, the road surface image can be input into the target detection model, or the road surface image and the point cloud can be input into the target detection model together. The target detection model includes a branch for predicting the drivable area, and the branch for predicting the drivable area includes one or more convolution layers, which outputs the drivable area. The target detection model also includes a target detection branch for outputting classified obstacles. In this way, the identification of classified obstacles and the prediction of drivable areas can be achieved simultaneously through a target detection model. The classified obstacles can be used to determine the area where the classified obstacles are located. The drivable area can be used to determine the area enclosed by the field of view corresponding to the preset field of view angle.

[0064] In some embodiments of the present application, Figure 7 As shown, controlling the mobile device to perform an obstacle avoidance operation on the obstacle area may include: steps S701 to S704.

[0065] Step S701: Obtain image coordinates of edge points of the obstacle area.

[0066] Among them, edge points refer to points on the edge line of the obstacle area. For each point, its image coordinates on the road surface image can be obtained.

[0067] Step S702 : based on the mapping relationship between image coordinates and depth, convert the image coordinates of the edge point into the depth of the edge point.

[0068] Specifically, before converting the image coordinates into the depth of the edge point based on the mapping relationship between the image coordinates and the depth, the method may also include: obtaining a ground point cloud collected by the radar; projecting the ground point cloud onto the road surface image; and in response to the ground point cloud being within the drivable area, obtaining a mapping relationship based on the image coordinates and depth of the ground point cloud.

[0069] Among them, the ground point cloud representation is based on the point cloud on the ground collected by the radar, and can be obtained by classifying the point cloud collected by the radar. Figure 8 and Figure 9 Shown respectively Figure 3 Visualization and point cloud of the drivable area shown. Figure 9 The orange portion represents the ground point cloud within the drivable area. Projecting the ground point cloud onto the road image reveals that if the projected ground point cloud falls within the drivable area—that is, if the ground point cloud is within the drivable area as shown in the segmentation mask—then the image coordinates and corresponding depth of the ground point cloud are recorded.

[0070] Fit the plane equation of the recorded image coordinates (x, y) and depth depth. The plane equation can be expressed as:

[0071] depth = a*x+b / (y*y*y)+c;

[0072] Among them, a, b, and c are the coefficients of the equation respectively.

[0073] By using the least squares method to solve the coefficients of the plane equation, we can obtain the mapping relationship between image coordinates and depth. In this way, we can convert image coordinates into the depth of edge points based on the mapping relationship between image coordinates and depth.

[0074] Step S703 : Acquire radar coordinates of the edge point based on the image coordinates of the edge point, the depth of the edge point, the intrinsic parameters of the camera, and the extrinsic parameters between the camera and the radar.

[0075] Based on the image coordinates of the edge point, its depth, and the camera's intrinsic parameters, the image coordinates of the edge point can be converted to the camera coordinate system. Based on the extrinsic parameters between the camera and radar, the camera coordinate system can be converted to the radar coordinate system to obtain the radar coordinates of the edge point.

[0076] Step S704: Based on the radar coordinates of the edge point, control the mobile device to perform an obstacle avoidance operation.

[0077] Based on the radar coordinates of the edge points, a bounding box can be obtained that covers the entire obstacle area. In the radar coordinate system, a bounding box is typically defined by the following parameters: center point coordinates (x, y, z), dimensions (l, w, h), and a rotation angle θ. Here, l, w, and h represent the length (along the radar coordinate system's X-axis), width (along the radar coordinate system's Y-axis), and height (along the radar coordinate system's Z-axis), respectively. θ is the yaw angle of the bounding box around the radar coordinate system's Z-axis, indicating the direction of the obstacle.

[0078] As can be seen from the preceding description, the bounding boxes here can include those for classified obstacles, obstacles within the drivable area, and obstacles at the edge of the camera's field of view. Based on the bounding boxes, the autonomous vehicle can be controlled to perform obstacle avoidance operations, allowing it to avoid obstacles within the bounding boxes. This allows the autonomous vehicle to effectively avoid both classified and unclassified obstacles, improving the safety of autonomous movement.

[0079] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders.

[0080] like Figure 101 is a schematic structural diagram of an obstacle avoidance device 1000 provided in an embodiment of the present application, wherein the obstacle avoidance device 1000 is configured on a self-moving device.

[0081] Specifically, the obstacle avoidance device 1000 may include:

[0082] The edge acquisition unit 1001 is used to acquire the outer edge of the drivable area in real time based on the road surface image;

[0083] The obstacle avoidance unit 1002 is used to update the obstacle area in real time and control the mobile device to perform obstacle avoidance operations on the obstacle area. The obstacle area includes the area enclosed by the outer edge of the drivable area and the field of view corresponding to the preset field of view angle, and the preset field of view angle is smaller than the camera acquisition angle of the road surface image.

[0084] In some embodiments of the present application, the edge acquisition unit 1001 can be used to: acquire the drivable area based on the road surface image; perform edge detection on the drivable area to obtain the edge of the drivable area; and classify the edge of the drivable area to obtain the outer edge.

[0085] In some embodiments of the present application, the edge acquisition unit 1001 can be used to: obtain the minimum circumscribed rectangle corresponding to each edge of the drivable area; in response to the area of the minimum circumscribed rectangle being greater than a first area threshold, use the corresponding edge as the outer edge.

[0086] In some embodiments of the present application, the obstacle area also includes an area enclosed by the edges of obstacles within the drivable area; the edge acquisition unit 1001 can be used to: in response to the area of the minimum circumscribed rectangle being greater than a second area threshold, and the number of non-drivable points within the minimum circumscribed rectangle being greater than the number of drivable points, use the corresponding edge as the edge of the obstacle within the drivable area, and the second area threshold is less than the first area threshold.

[0087] In some embodiments of the present application, the obstacle area also includes an area where classified obstacles are located; the obstacle avoidance device 1000 may further include a target detection unit, which is used to: identify obstacles based on the road surface image to obtain the classified obstacles.

[0088] In some embodiments of the present application, the obstacle avoidance unit 1002 can be used to: obtain the image coordinates of the edge point of the obstacle area; convert the image coordinates of the edge point into the depth of the edge point based on the mapping relationship between the image coordinates and the depth; obtain the radar coordinates of the edge point based on the image coordinates of the edge point, the depth of the edge point, the intrinsic parameters of the camera, and the extrinsic parameters between the camera and the radar; and control the self-moving device to perform an obstacle avoidance operation based on the radar coordinates of the edge point.

[0089] In some embodiments of the present application, the obstacle avoidance unit 1002 can be used to: obtain the ground point cloud collected by the radar; project the ground point cloud onto the road surface image; in response to the ground point cloud being within the drivable area, obtain the mapping relationship based on the image coordinates and depth of the ground point cloud.

[0090] It should be noted that for the convenience and simplicity of description, the specific working process of the above obstacle avoidance device 1000 can be referred to Figures 1 to 9 The corresponding process of the method will not be described in detail here.

[0091] like Figure 11 , which is a schematic diagram of a self-moving device provided in an embodiment of the present application. Specifically, the self-moving device 11 may include: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110, such as an obstacle avoidance program. When the processor 110 executes the computer program 112, the steps in the above-mentioned obstacle avoidance method embodiments are implemented, such as Figure 1 Alternatively, when the processor 110 executes the computer program 112, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 10 The functions of the edge acquisition unit 1001 and the obstacle avoidance unit 1002 are shown.

[0092] The computer program may be divided into one or more modules / units, which are stored in the memory 111 and executed by the processor 110 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the mobile device.

[0093] For example, the computer program can be divided into an edge acquisition unit and an obstacle avoidance unit. The specific functions of each unit are as follows: the edge acquisition unit is used to acquire the outer edge of the drivable area in real time based on the road image; the obstacle avoidance unit is used to update the obstacle area in real time and control the mobile device to perform obstacle avoidance operations on the obstacle area. The obstacle area includes the area enclosed by the outer edge of the drivable area and the field of view corresponding to a preset field of view angle, where the preset field of view angle is smaller than the camera's angle of view for capturing the road image.

[0094] The self-mobile device may include, but is not limited to, a processor 110 and a memory 111. It will be understood by those skilled in the art that Figure 11 The self-mobile device is merely an example and does not constitute a limitation of the self-mobile device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the self-mobile device may also include input and output devices, network access devices, buses, etc.

[0095] The processor 110 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0096] The memory 111 may be an internal storage unit of the self-mobile device, such as a hard disk or memory of the self-mobile device. The memory 111 may also be an external storage device of the self-mobile device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the self-mobile device. Furthermore, the memory 111 may also include both an internal storage unit of the self-mobile device and an external storage device. The memory 111 is used to store the computer program and other programs and data required by the self-mobile device. The memory 111 may also be used to temporarily store data that has been output or is to be output.

[0097] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned self-moving device can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0099] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0100] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices / self-moving devices and methods can be implemented in other ways. For example, the device / self-moving device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0104] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0105] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An obstacle avoidance method, characterized in that: include: Based on the road surface image, the outer edge of the drivable area is obtained in real time; The obstacle area is updated in real time, and the mobile device is controlled to perform obstacle avoidance operations on the obstacle area. The obstacle area includes an area enclosed by the outer edge of the drivable area and a field of view corresponding to a preset field of view angle, and the preset field of view angle is smaller than the camera acquisition angle of the road surface image.

2. The obstacle avoidance method according to claim 1, wherein: The method of obtaining the outer edge of the drivable area in real time based on the road surface image includes: acquiring the drivable area based on the road surface image; Performing edge detection on the drivable area to obtain an edge of the drivable area; The edge of the drivable area is classified to obtain the outer edge.

3. The obstacle avoidance method according to claim 2, wherein: The classifying the edge of the drivable area to obtain the outer edge includes: Obtaining the minimum circumscribed rectangle corresponding to each edge of the drivable area; In response to the area of the minimum circumscribed rectangle being greater than a first area threshold, the corresponding edge is used as the outer edge.

4. The obstacle avoidance method according to claim 3, wherein: The obstacle area also includes an area enclosed by the edges of obstacles within the drivable area; After obtaining the minimum circumscribed rectangle corresponding to each edge of the drivable area, the method further includes: In response to the area of the minimum circumscribed rectangle being greater than a second area threshold and the number of non-drivable points within the minimum circumscribed rectangle being greater than the number of drivable points, the corresponding edge is used as the edge of the obstacle inside the drivable area, and the second area threshold is less than the first area threshold.

5. The obstacle avoidance method according to claim 1, wherein: The obstacle area also includes the area where the classified obstacles are located; The obstacle avoidance method further includes: Obstacle recognition is performed based on the road surface image to obtain the classified obstacles.

6. The obstacle avoidance method according to any one of claims 1 to 5, characterized in that: The controlling the mobile device to perform an obstacle avoidance operation on the obstacle area includes: Obtaining image coordinates of edge points of the obstacle area; Based on a mapping relationship between image coordinates and depth, converting the image coordinates of the edge point into the depth of the edge point; Acquire radar coordinates of the edge point based on the image coordinates of the edge point, the depth of the edge point, an intrinsic parameter of the camera, and an extrinsic parameter between the camera and the radar; Based on the radar coordinates of the edge point, the self-moving device is controlled to perform an obstacle avoidance operation.

7. The obstacle avoidance method according to claim 6, wherein: Before acquiring the radar coordinates of the edge point based on the image coordinates of the edge point, the depth of the edge point, the intrinsic parameters of the camera, and the extrinsic parameters between the camera and the radar, the method further includes: Obtaining a ground point cloud collected by the radar; Projecting the ground point cloud onto the road surface image; In response to the ground point cloud being located within the drivable area, the mapping relationship is acquired based on the image coordinates and depth of the ground point cloud.

8. An obstacle avoidance device, characterized in that: include: An edge acquisition unit, used to acquire the outer edge of the drivable area in real time based on the road surface image; An obstacle avoidance unit is configured to update an obstacle area in real time and control the mobile device to perform obstacle avoidance operations on the obstacle area. The obstacle area includes an area enclosed by the outer edge of the drivable area and a field of view corresponding to a preset field of view angle, where the preset field of view angle is smaller than the camera acquisition angle of the road surface image.

9. A self-propelled device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the obstacle avoidance method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The present invention comprises a computer program, which, when executed, enables the obstacle avoidance method according to any one of claims 1 to 7 to be executed.

Images