Obstacle position estimation method, apparatus and device, and storage medium

Abstract

The present application relates to the field of visual navigation, and in particular to an obstacle position estimation method, apparatus and device, and a storage medium. The obstacle is a base obstacle. The method comprises: acquiring an image comprising a base obstacle; determining a recognition frame of the base obstacle in the image; starting from a midpoint of a first edge of the recognition frame, searching for an estimated center point of the base obstacle in a first direction which is perpendicular to the first edge and points to the recognition frame; determining a first radius on the basis of the estimated center point and the midpoint, and determining a second radius on the basis of the estimated center point in a direction parallel to the first edge; and determining the position of a target center point on the basis of a radius bias between the first radius and the second radius. The target center point is found in the first direction determined on the basis of the midpoint, and verification is performed on the basis of a radius bias, so that the accuracy of the position of the target center point can be effectively improved, thereby facilitating implementation of more accurate path planning.

Description

Obstacle position estimation method, device, equipment and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202411284677.2, filed on September 12, 2024 in the China Patent Office, and entitled "Obstacle position estimation method, device, equipment and storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of visual navigation, in particular to an obstacle position estimation method, device, equipment and storage medium. BACKGROUND

[0003] In the process of performing a task, a sweeping robot or other robot can identify an obstacle in a scene, and perform path planning according to the identified obstacle in the scene, so as to reduce contact and collision with the obstacle.

[0004] When a robot such as a sweeping robot uses a monocular camera to collect images, the monocular camera collects a planar image, which is not conducive to obtaining the thickness of the obstacle. Therefore, an estimation method is usually used to determine the position of the obstacle. For example, for a base type obstacle such as a fan base or a bar base, the position of the object is determined by image blur. However, the accuracy of the estimated position is not high, which is not conducive to accurate path planning according to the obstacle. TECHNICAL PROBLEM

[0005] Therefore, the embodiments of the present application provide an obstacle position estimation method, device, equipment and storage medium to solve the problem that the accuracy of the estimated obstacle position is not high, which is not conducive to accurate path planning according to the obstacle. TECHNICAL SOLUTION

[0006] The first aspect of the embodiments of the present application provides an obstacle position estimation method, wherein the obstacle is a base type obstacle, and the method comprises:

[0007] obtaining an image including the base type obstacle;

[0008] determining a bounding box of the base type obstacle in the image;

[0009] finding an estimated center point of the base type obstacle from a midpoint of a first edge of the bounding box in a first direction perpendicular to the first edge and towards the bounding box;

[0010] determining a first radius according to the estimated center point and the midpoint, and determining a second radius according to the estimated center point in a direction parallel to the first edge;

[0011] The position of the target center point is determined according to the radius deviation of the first radius and the second radius.

[0012] With reference to the first aspect, in a first possible implementation manner of the first aspect, the position of the target center point is determined according to the radius deviation of the first radius and the second radius, and the method comprises the following steps.

[0013] The position of the estimated center point is updated according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point;

[0014] The position of the estimated center point is updated according to the updated position of the center point, if the radius deviation is greater than the deviation threshold, the position of the estimated center point is re-updated until the radius deviation is less than or equal to the deviation threshold, and the estimated center point for which the radius deviation is finally determined is taken as the target center point.

[0015] With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the position of the estimated center point is updated according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point, and the method comprises the following steps.

[0016] When the radius deviation belongs to a first deviation range, the estimated center point is determined to move a first distance according to the correspondence;

[0017] When the radius deviation belongs to a second deviation range, the estimated center point is determined to move a second distance according to the correspondence;

[0018] The radius deviation in the first deviation range is less than the radius deviation in the second deviation range, and the first distance is less than the second distance.

[0019] With reference to the first possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, before the estimated center point for which the radius deviation is finally determined is taken as the target center point, the method further comprises the following steps.

[0020] When the radius deviation is less than or equal to the deviation threshold, the first radius is compared with a predetermined effective radius range;

[0021] When the first radius does not belong to the predetermined effective radius range, the estimated center point is determined to be an invalid center point.

[0022] With reference to the first aspect, in a fourth possible implementation manner of the first aspect, before the estimated center point of the base type obstacle is searched for from the midpoint of the first edge of the identification frame to a first direction of the identification frame perpendicular to the first edge, the method further comprises the following steps.

[0023] acquire a third distance from the midpoint to the camera;

[0024] determine an offset in the first direction from the midpoint according to a preset correspondence between distance and offset;

[0025] determine a search starting point of the estimated center point according to the offset in the first direction from the midpoint.

[0026] In a fifth possible implementation manner of the first aspect, before the estimated center point of the base-type obstacle is found from the midpoint of the first edge of the identified frame in a direction perpendicular to the first edge and toward the first direction of the identified frame, the method further includes:

[0027] determine a maximum search range of the estimated center point according to a predetermined ratio of the width of the first edge of the identified frame.

[0028] In a sixth possible implementation manner of the first aspect, before the estimated center point of the base-type obstacle is found from the midpoint of the first edge of the identified frame in a direction perpendicular to the first edge and toward the first direction of the identified frame, the method further includes:

[0029] determine a first pose of the robot when the image is acquired, and a second pose of the robot in a stationary state;

[0030] correct a position corresponding to a pixel in the image according to the first pose and the second pose.

[0031] A second aspect of the embodiments of the present application provides a position estimation device of an obstacle, the obstacle being a base-type obstacle, and the device including:

[0032] an image acquisition unit configured to acquire an image including the base-type obstacle;

[0033] an identified frame determination unit configured to determine an identified frame of the base-type obstacle in the image;

[0034] an estimated center point finding unit configured to find an estimated center point of the base-type obstacle from a midpoint of a first edge of the identified frame in a direction perpendicular to the first edge and toward a first direction of the identified frame;

[0035] a radius determination unit configured to determine a first radius according to the estimated center point and the midpoint, and determine a second radius according to the estimated center point in a direction parallel to the first edge;

[0036] The target center point determination unit is configured to determine a position of a target center point according to a radius deviation between the first radius and the second radius.

[0037] With reference to the second aspect, in a first possible implementation manner of the second aspect, the target center point determination unit comprises:

[0038] The position updating sub-unit is configured to update the position of the estimated center point according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point;

[0039] The target center point determination sub-unit is configured to update the radius deviation according to the updated position of the center point, and if the radius deviation is greater than the deviation threshold, re-update the position of the estimated center point until the radius deviation is less than or equal to the deviation threshold, and determine the estimated center point for which the radius deviation is last determined as the target center point.

[0040] With reference to the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the position updating sub-unit comprises:

[0041] The first distance determination module is configured to determine that the estimated center point moves a first distance according to the correspondence when the radius deviation belongs to a first deviation range.

[0042] The second distance determination module is configured to determine that the estimated center point moves a second distance according to the correspondence when the radius deviation belongs to a second deviation range.

[0043] The first deviation range is smaller than the second deviation range, and the first distance is smaller than the second distance.

[0044] With reference to the first possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the device further comprises:

[0045] The range comparison unit is configured to compare the first radius with a predetermined effective radius range when the radius deviation is less than or equal to the deviation threshold.

[0046] The invalid center point determination unit is configured to determine the estimated center point as an invalid center point when the first radius does not belong to the predetermined effective radius range.

[0047] With reference to the second aspect, in a fourth possible implementation manner of the second aspect, the device further comprises:

[0048] The third distance acquisition unit is configured to acquire a third distance between the midpoint and the camera.

[0049] The offset amount determination unit is configured to determine an offset amount in the first direction from the midpoint according to a preset correspondence between distances and offset amounts.

[0050] The search starting point determination unit is configured to determine a search starting point of the estimated center point according to the offset amount in the first direction from the midpoint.

[0051] With reference to the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the second aspect, the apparatus further includes:

[0052] The maximum search range determination unit is configured to determine a maximum search range of the estimated center point according to a predetermined ratio of the width of the first edge of the recognition box.

[0053] With reference to any one of the second aspect to the fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, the apparatus further includes:

[0054] The pose determination unit is configured to determine a first pose of the robot when the image is acquired, and a second pose of the robot in a stationary state.

[0055] The position correction unit is configured to correct a position corresponding to a pixel in the image according to the first pose and the second pose.

[0056] The third aspect of the embodiments of the present application provides an obstacle position estimation device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of the first aspect when executing the computer program.

[0057] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method according to any one of the first aspect. Advantages

[0058] Compared with the prior art, the embodiments of the present application have the following advantages: the embodiments of the present application determine the recognition box of the base type obstacle in the image, and find the estimated center point of the base type obstacle by taking the midpoint of the first edge of the recognition box as the starting point and moving in the first direction perpendicular to the first edge. After determining the estimated center point, the first radius is determined according to the estimated center point and the midpoint, the second radius is determined according to the estimated center point in the direction parallel to the first edge, and the position of the target center point is accurately determined according to the radius deviation of the first radius and the second radius. Since the first direction is determined by the midpoint of the target center point, and the radius deviation is checked, the accuracy of the position of the target center point can be effectively improved, which is beneficial to more accurate path planning. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0060] FIG. 1 is an implementation flow diagram of a method for estimating the position of an obstacle according to an embodiment of the present application;

[0061] FIG. 2 is a schematic diagram of an obstacle position estimation scenario according to an embodiment of the present application;

[0062] FIG. 3 is an implementation flow diagram of a method for correcting physical coordinates according to an embodiment of the present application;

[0063] FIG. 4 is an implementation flow diagram of a method for determining a search starting point according to an embodiment of the present application;

[0064] FIG. 5 is an implementation flow diagram of a method for determining the position of a target center point according to an embodiment of the present application;

[0065] FIG. 6 is a schematic diagram of an obstacle position estimation device according to an embodiment of the present application;

[0066] FIG. 7 is a schematic diagram of an obstacle position estimation device according to an embodiment of the present application. Embodiments of the present application

[0067] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known structures, devices, circuits and methods have not been described in detail in order to avoid obscuring the present application.

[0068] In order to illustrate the technical solutions described in the present application, the following will be described through specific embodiments.

[0069] Sweeping robots and other types of service robots need to move safely and effectively in complex environments when performing cleaning, navigation or other tasks. To achieve this goal, robots are usually equipped with sensors for detecting obstacles, such as monocular cameras on sweeping robots, which are used to perceive the surrounding environment and identify potential obstacles. These obstacles can include furniture, appliances or other household items that the robot must be able to identify and bypass to avoid collisions and damage.

[0070] Monocular camera is a commonly used visual sensor, which provides visual information of the environment around the robot by capturing two-dimensional images. However, one of the main limitations of monocular camera is that it can only provide a planar image, lacking depth information, which makes it difficult to directly measure the size of the obstacle, especially the thickness. Therefore, the robot usually needs to rely on algorithm estimation to infer the three-dimensional position and shape of the obstacle.

[0071] In practical applications, for base type obstacles such as fan base or bar stool base, the robot may use a fuzzy estimation method to determine, due to the inaccuracy of the estimation result, the robot cannot accurately avoid obstacles when planning a path.

[0072] To solve the above problems, the embodiment of the application provides a position estimation method of an obstacle. The execution subject of the method can be a robot. The robot can include a sweeping robot or other robots performing tasks. As shown in FIG. 1, the method includes:

[0073] In S101, an image including a base type obstacle is obtained.

[0074] The base type obstacle in the embodiment of the application is an obstacle including a base. The base type obstacle can include obstacles of circular objects such as fan base and bar stool base.

[0075] The image in the embodiment of the application can be an image collected by a monocular camera during the execution of the task by the robot.

[0076] The base type obstacle in the image in the embodiment of the application can be identified by an image recognition method, including whether the object included in the image is a base type obstacle by a deep learning method, a random forest method, etc.

[0077] In a possible implementation, whether the image includes a base type obstacle can also be identified by pre-set features of the base type obstacle. For example, the features of the base type obstacle can be pre-set, including shape features and size features of circular objects such as fan base, bar stool base, etc. at different angles. After the image is collected, shape matching is performed according to the pre-set features of the base type obstacle, including shape features of circular objects such as fan base, bar stool base, etc. to determine whether the image includes a base type obstacle.

[0078] When it is identified in the embodiment of the application that the image includes a part of the image of the base type obstacle, such as the left side, the right side, the upper side or the lower side of the base type obstacle located at the edge of the image, since the image of the base type obstacle is incomplete, the image can be continuously collected until the image includes a complete base type obstacle.

[0079] In S102, a bounding box of the base type obstacle in the image is determined.

[0080] The bounding box of each object can be determined according to an object recognition method. Alternatively, a rectangular box including the base type obstacle can also be determined as the bounding box of the base type obstacle according to the shape information of the base type obstacle.

[0081] The bounding box of the base type obstacle includes the entire area of the base type obstacle. For example, in the scene image shown in FIG. 2, the base and the support column on the base are included. The bounding box is determined according to the leftmost point, the rightmost point, the frontmost point and the rearmost point of the base image, and the bounding box is schematically shown as a dashed line shape. The edge of the bounding box is parallel to the edge of the image, and the bounding box does not include the area of the support column on the base.

[0082] In S103, an estimated center point of the base type obstacle is searched for in a first direction of the bounding box and perpendicular to a first edge of the bounding box, with a midpoint of the first edge as a search center.

[0083] As shown in FIG. 2, after the bounding box of the base type obstacle is determined, the size information of the bounding box can be determined, including the width w and the height h of the bounding box. The width w is the length of the edge of the bounding box parallel to the upper and lower edges of the image. The height is the length of the edge of the bounding box parallel to the left and right edges of the image. When the coordinate system of the image is constructed, the coordinates of the lower left corner vertex Pl and the lower right corner vertex Pr of the bounding box can be determined. For example, after the coordinate system is constructed with the upper left corner vertex of the image as the origin, the horizontal direction to the right as the x-axis direction and the vertical direction as the y-axis direction, the coordinates of the lower left corner vertex Pl and the lower right corner vertex Pr of the bounding box are determined according to the constructed coordinate system. According to the determined coordinates of the lower left corner vertex Pl and the lower right corner vertex Pr of the bounding box, the coordinates of the midpoint Pc of the first edge of the bounding box where Pl and Pr are located can be obtained.

[0084] The first edge can be the lower edge of the bounding box in the image, but is not limited to this, and can also be the upper edge of the bounding box in the image.

[0085] In order to further improve the position estimation accuracy of the obstacle, the pixel points in the image can also be subjected to distortion correction before the position of the base type obstacle is determined, including methods such as perspective distortion correction for distortion correction of the coordinates of the bounding box. Based on the position of the pixel points after the distortion correction, in combination with the distance determined by the monocular camera and the position of the monocular camera in the world coordinate system, the physical coordinates of each pixel point on the bounding box are determined. For example, the physical coordinates of the lower left corner vertex Pl, the lower right corner vertex Pr and the midpoint Pc of the lower edge of the bounding box can be determined as Pl o, Pr o and Pc o respectively.

[0086] In a possible implementation, before finding the target center point, the physical coordinates of the pixels in the image can also be corrected according to the pose information of the robot, as shown in FIG. 3, the correction process can include:

[0087] In S301, a first pose of the robot when the image is acquired and a second pose of the robot when the robot is in a stationary state are determined.

[0088] The first pose of the robot when the image is acquired and the second pose of the robot when the robot is in a stationary state can be determined by a pose sensor, for example, the first pose and the second pose can be acquired by an IMU (Inertial Measurement Unit).

[0089] The height of the monocular camera can be determined by external parameter calibration. The first pitch angle pitch1 and the first roll angle roll1 corresponding to the first pose can be determined by the pose sensor, and the second pitch angle pitch2 and the second roll angle roll2 corresponding to the second pose can be determined.

[0090] Suppose that the x-axis direction of the physical coordinate system is the direction of the monocular camera, the y-axis direction is the direction that is the same as the x-axis direction and is perpendicular to the x-axis, and the z-axis direction is the vertical upward direction, then the angle between the projection line of the connection line between any point Pt in the scene and the optical center of the monocular camera in the xz plane and the gravity direction is anglePtPitch=atan2(pt.x,H), and the angle between the projection line of the connection line between the point Pt and the optical center of the monocular camera in the yz plane and the gravity direction is anglePtRoll=atan2(pt.y,H). Wherein, atan2(x,y) represents arctan(y / x).

[0091] In S302, the positions corresponding to the pixels in the image are corrected according to the first pose and the second pose.

[0092] According to the first pitch angle pitch1 and the first roll angle roll1 corresponding to the first pose, the second pitch angle pitch2 and the second roll angle roll2 corresponding to the second pose, and the determined angle between the connection line between any point in the scene and the optical center of the monocular camera and the gravity direction, the physical coordinates (coordinates in the world coordinate system where the robot is located) Pt’(Pt’.x,Pt’.y) after the pose correction of the physical coordinates can be determined, which can be represented as:

[0093] Pt’.x=H*tan(anglePtPitch-pitch1+pitch2)

[0094] pt'.y = H * tan(anglePtRoll - roll1 - roll2)

[0095] The position of the determined target center point can be converted into a real position in the world coordinate system according to the corrected physical coordinates.

[0096] In the determination of the position of the target center point, the center of the recognition box can be searched according to the determined midpoint. In order to improve the search efficiency, the search starting point can be determined according to the distance between the midpoint and the monocular camera. The process can be as shown in FIG. 4, including:

[0097] In S401, the third distance between the midpoint and the camera is obtained.

[0098] As shown in FIG. 2, when the monocular camera of the robot is installed at a predetermined height, the robot can be calibrated in advance for external and internal parameters, including calibrating the height of the monocular camera. In combination with the size of the base type obstacle, the distance between the pixel point in the image and the monocular camera can be identified, such as obtaining the third distance between the midpoint and the camera.

[0099] In S402, the offset amount in the first direction from the midpoint is determined according to the preset correspondence between the distance and the offset amount.

[0100] That is, the first direction is the direction of the midpoint pointing to the center of the recognition box, that is, the direction perpendicular to the first edge (the lower edge of the recognition box) and pointing to the side of the recognition box from the midpoint.

[0101] The closer the distance between the midpoint and the monocular camera, the larger the image of the base type obstacle in the obtained image, and therefore, the larger the distance between the coordinates of the target center point and the midpoint. The search for the estimated center point can be performed after a large distance is offset in the first direction. The farther the distance between the midpoint and the monocular camera, the smaller the image of the base type obstacle in the obtained image, and therefore, the smaller the distance between the coordinates of the target center point and the midpoint. The search for the estimated center point can be performed after a small distance is offset in the first direction.

[0102] For example, a plurality of distance ranges can be preset, and the greater the distance range to which the third distance belongs, the greater the offset amount for the search starting point.

[0103] For example, the distance threshold can be set to include 0.2 meters, 0.25 meters, 0.35 meters, the third distance is d, and the coordinates of the midpoint are (Pc.x, Pc.y). If d is less than 0.2 meters, the search starting point Pc' of the initial search is pixel (Pc.x, Pc.y-36), that is, offset by 36 pixels in the first direction as the search starting point; if d is greater than 0.2 meters and less than 0.25 meters, the search starting point Pc' of the initial search is pixel (Pc.x, Pc.y-30), that is, offset by 30 pixels in the first direction as the search starting point; if d is greater than 0.25 meters and less than 0.35 meters, the search starting point Pc' of the initial search is pixel (Pc.x, Pc.y-12), that is, offset by 12 pixels in the first direction as the search starting point; if d is greater than 0.35 meters, the search starting point Pc' of the initial search is pixel (Pc.x, Pc.y-6), that is, offset by 6 pixels in the first direction as the search starting point. If the coordinates of the search starting point, Pc'.y is less than 0, Pc'.y can be set to 0.

[0104] In S403, the search starting point of the estimated center point is determined according to the offset amount in the first direction from the midpoint.

[0105] After determining the offset amount according to the distance between the midpoint and the monocular camera, the search starting point of the estimated center point can be determined in the first direction according to the offset amount, so that the calculation of the estimated center point can be effectively reduced, and the calculation efficiency of the target center point can be improved.

[0106] In a possible implementation, the embodiments of the present application can also determine the maximum search range of the estimated center point according to a predetermined proportion of the width of the first edge. As shown in FIG. 2, the first edge is the lower edge of the recognition frame, and the width of the lower edge is w. The predetermined proportion of the width of the lower edge can be w / 3 or the like. That is, when the estimated center point starts searching from the search starting point, if the effective target center point is still not found when searching to the boundary of the maximum search range, the identification of the position of the base type obstacle in the image can be paused. Alternatively, the predetermined proportion of the width of the lower edge can also be compared with the length of the left edge or the right edge of the recognition frame. If the predetermined proportion of the width of the lower edge is greater than the length of the left edge or the right edge, the maximum search range can be determined according to the length of the left edge or the right edge.

[0107] When searching according to the search starting point, the estimated center point is determined in the first direction from the search starting point of the red frame, until the target center point is searched or the maximum search range is searched.

[0108] In S104, the first radius is determined according to the estimated center point and the midpoint, and the second radius is determined according to the estimated center point in the direction parallel to the first edge.

[0109] In the process of determining the first radius according to the estimated center point and the midpoint, and determining the second radius according to the estimated center point along the direction parallel to the first edge, the relevant pixel points can be corrected for distortion, and then the physical coordinates of the relevant pixel points can be calculated. After the physical coordinates of the relevant pixel points are corrected by the pose sensor, the first radius determined according to the estimated center point and the midpoint, and the second radius determined according to the estimated center point along the direction parallel to the first edge can be calculated.

[0110] The relevant pixel points include the estimated center point Pc', the midpoint Pc, and the intersection points Pl' and Pr' of the parallel line determined according to the estimated center point and parallel to the first edge and the recognition frame. After the distortion correction and the pose correction, the physical coordinates corresponding to the estimated center point Pc' are Pc'_o, the physical coordinates corresponding to the midpoint Pc are Pc'_o, the physical coordinates corresponding to the intersection point Pl' are Pl'_o, and the physical coordinates corresponding to the intersection point Pr' are Pr'_o. Therefore, the first radius radiu1 is the distance from Pc_o to Pc'_o, and the second radius radiu2 is half of the distance from Pl'_o to Pr'_o.

[0111] In S105, the position of the target center point is determined according to the radius deviation of the first radius and the second radius.

[0112] The radius deviation radiubias of the first radius radiu1 and the second radius radiu2 is the absolute value of the difference between the first radius radiu1 and the second radius radiu2.

[0113] In the process of determining the position of the target center point according to the radius deviation, as shown in FIG. 5, the process can include:

[0114] In S501, the position of the estimated center point is updated according to the corresponding relationship between the predetermined radius deviation and the moving distance of the estimated center point.

[0115] When the radius deviation is greater than the predetermined deviation threshold, the position of the estimated center point can be updated according to the corresponding relationship between the radius deviation and the moving distance of the estimated center point, that is, other pixels are selected as the estimated center point for calculation in the first direction.

[0116] In order to improve the search efficiency, the number of pixels that need to be moved when the next estimated center point is determined can be determined according to the size of the radius deviation.

[0117] For example, a plurality of deviation ranges can be set, which can include a first deviation range and a second deviation range. If the radius deviation belongs to the first deviation range, the estimated center point is determined to move a first distance according to the corresponding relationship, and if the radius deviation belongs to the second deviation range, the estimated center point is determined to move a second distance according to the corresponding relationship. Wherein, any one radius deviation in the first deviation range is smaller than any one radius deviation in the second deviation range, and the first distance is smaller than the second distance.

[0118] Specifically, if the radius deviation is less than 0.02 meters, it can be considered that the current estimated center point is the target center point, and the result can be output and the search calculation is exited. The Pc’_o physical coordinates can be taken as the center of the object, and the second radius radiu2 is determined as the radius of the base type obstacle. If the radius deviation is greater than 0.02 meters and less than 0.05 meters, the position of Pc’ can be moved up by 2 pixels as the updated estimated center point, and the effectiveness of the updated estimated center point is continued to be calculated. If the radius deviation is greater than 0.05 meters, the position of the updated estimated center point Pc’ can be moved up by 4 pixels and the effectiveness of the updated estimated center point is calculated.

[0119] In S502, the radius deviation is updated according to the position of the updated center point, and if the radius deviation is greater than the deviation threshold, the position of the estimated center point is re-updated until the radius deviation is less than or equal to the deviation threshold, and the estimated center point at which the radius deviation is finally determined is taken as the target center point.

[0120] For the updated estimated center point, the first radius and the second radius corresponding to the updated estimated center point are calculated, and whether the radius deviation of the first radius and the second radius is less than or equal to the deviation threshold is judged. If the requirement is not met, the estimated center point is further updated according to the size of the radius deviation until the updated radius deviation is less than or equal to the deviation threshold, at which time the current estimated center point can be determined as the target center point, and the current first radius or the second radius is the radius of the base type obstacle.

[0121] In a possible implementation, when the radius deviation is less than or equal to the deviation threshold, the first radius can be further compared with a predetermined effective radius range, and if it does not belong to the predetermined effective radius range, the current estimated center point can be determined as an invalid center point. Wherein, the effective radius range can be a range greater than 0.5 meters or less than 0.1 meters. Since there can be other similar obstacles to the base type obstacle in the scene, the effective radius range can reduce the influence of other similar obstacles on the position estimation accuracy.

[0122] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0123] Fig. 6 is a schematic diagram of an obstacle position estimation device provided by an embodiment of the present application, the obstacle being a base type obstacle, the device comprising:

[0124] An image acquisition unit 601 is configured to acquire an image including the base type obstacle.

[0125] An identification frame determination unit 602 is configured to determine an identification frame of the base type obstacle in the image.

[0126] An estimated center point finding unit 603 is configured to find an estimated center point of the base type obstacle in a first direction of the identification frame perpendicular to a first edge of the identification frame and towards the first edge with a midpoint of the first edge as a starting point.

[0127] A radius determination unit 604 is configured to determine a first radius according to the estimated center point and the midpoint, and determine a second radius according to the estimated center point in a direction parallel to the first edge.

[0128] A target center point determination unit 605 is configured to determine a position of a target center point according to a radius deviation between the first radius and the second radius.

[0129] In combination with the second aspect, in a first possible implementation manner of the second aspect, the target center point determination unit comprises:

[0130] A position updating subunit is configured to update the position of the estimated center point according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point.

[0131] A target center point determination subunit is configured to update the radius deviation according to the updated position of the center point, and if the radius deviation is greater than a deviation threshold, re-update the position of the estimated center point until the radius deviation is less than or equal to the deviation threshold, and take the estimated center point with the last determined radius deviation as the target center point.

[0132] In combination with the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the position updating subunit comprises:

[0133] A first distance determination module is configured to determine that the estimated center point moves a first distance according to the correspondence when the radius deviation belongs to a first deviation range.

[0134] A second distance determination module is configured to determine that the estimated center point moves a second distance according to the correspondence when the radius deviation belongs to a second deviation range.

[0135] The radius deviation in the first deviation range is smaller than the radius deviation in the second deviation range, and the first distance is smaller than the second distance.

[0136] With reference to the first possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the device further includes:

[0137] The range comparison unit is configured to compare the first radius with a predetermined valid radius range when the radius deviation is smaller than or equal to a deviation threshold.

[0138] The invalid center point determination unit is configured to determine the estimated center point as an invalid center point when the first radius does not belong to the predetermined valid radius range.

[0139] With reference to the second aspect, in a fourth possible implementation manner of the second aspect, the device further includes:

[0140] The third distance acquisition unit is configured to acquire a third distance between the midpoint and the camera.

[0141] The offset amount determination unit is configured to determine an offset amount in the first direction from the midpoint according to a preset correspondence between distances and offset amounts.

[0142] The search starting point determination unit is configured to determine a search starting point of the estimated center point according to the offset amount in the first direction from the midpoint.

[0143] With reference to the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the second aspect, the device further includes:

[0144] The maximum search range determination unit is configured to determine a maximum search range of the estimated center point according to a predetermined ratio of the width of the first edge of the bounding box.

[0145] With reference to any one of the second aspect to the fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, the device further includes:

[0146] The pose determination unit is configured to determine a first pose of the robot when the image is acquired, and a second pose of the robot in a stationary state.

[0147] The position correction unit is configured to correct positions of pixels in the image according to the first pose and the second pose.

[0148] The device for estimating the position of the obstacle shown in FIG. 6 corresponds to the method for estimating the position of the obstacle shown in FIG. 1.

[0149] FIG. 7 is a schematic diagram of an obstacle position estimation device according to an embodiment of the present application. As shown in FIG. 7, the obstacle position estimation device 7 according to the embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and capable of running on the processor 70, e.g., an obstacle position estimation program. The processor 70 implements the steps of the obstacle position estimation method embodiments described above when executing the computer program 72. Alternatively, the processor 70 implements the functions of the modules / units in the apparatus embodiments described above when executing the computer program 72.

[0150] For example, the computer program 72 can be divided into one or more modules / units stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 72 in the obstacle position estimation device 7.

[0151] The obstacle position estimation device 7 can be a desktop computer, a notebook, a palm computer, a cloud server, or the like. The obstacle position estimation device can include, but is not limited to, the processor 70 and the memory 71. Those skilled in the art can understand that FIG. 7 is only an example of the obstacle position estimation device 7, and does not limit the obstacle position estimation device 7, which can include more or fewer components than those shown, or combine certain components, or different components, e.g., the obstacle position estimation device can also include an input / output device, a network access device, a bus, etc.

[0152] The processor 70 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor.

[0153] The storage 71 can be an internal storage unit of the obstacle position estimation device 7, such as a hard disk or a memory of the obstacle position estimation device 7. The storage 71 can also be an external storage device of the obstacle position estimation device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, or the like, equipped on the obstacle position estimation device 7. Further, the storage 71 can include both an internal storage unit and an external storage device of the obstacle position estimation device 7. The storage 71 is used to store the computer program and other programs and data required by the obstacle position estimation device. The storage 71 can also be used to temporarily store data that has been output or will be output.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0155] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can refer to the relevant description of other embodiments.

[0156] Those of ordinary skill in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0157] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other manners. For example, the described apparatus / terminal device embodiments are merely schematic. For example, the division of the modules or units is merely logical function division. There can be another division manner for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units.

[0158] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0159] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0160] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by computer program instruction related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier signal and telecommunication signal.

[0161] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of estimating a position of an obstacle, characterized by, The obstacle is a base type obstacle, and the method comprises: acquiring an image including the base type obstacle; determining a bounding box of the base type obstacle in the image; finding an estimated center point of the base type obstacle from a midpoint of a first edge of the bounding box to a first direction of the bounding box perpendicular to the first edge; determining a first radius according to the estimated center point and the midpoint, and determining a second radius according to the estimated center point along a direction parallel to the first edge; determining a position of a target center point according to a radius deviation of the first radius and the second radius.

2. The method of claim 1, wherein, determining a position of a target center point according to a radius deviation of the first radius and the second radius, comprising: updating the position of the estimated center point according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point; updating the position of the estimated center point according to the updated position of the center point, and if the radius deviation is greater than a preset deviation threshold, re-updating the position of the estimated center point until the radius deviation is less than or equal to the deviation threshold, and taking the estimated center point for which the radius deviation is finally determined as the target center point.

3. The method of claim 2, wherein, updating the position of the estimated center point according to a predetermined correspondence between the radius deviation and a moving distance of the estimated center point, comprising: when the radius deviation belongs to a first deviation range, determining that the estimated center point moves a first distance according to the correspondence; when the radius deviation belongs to a second deviation range, determining that the estimated center point moves a second distance according to the correspondence; wherein the radius deviation in the first deviation range is less than the radius deviation in the second deviation range, and the first distance is less than the second distance.

4. The method of claim 2, wherein, Before taking the estimated center point for which the radius deviation is finally determined as the target center point, the method further comprises: when the radius deviation is less than or equal to the deviation threshold, comparing the first radius with a predetermined effective radius range; when the first radius does not belong to the predetermined effective radius range, determining that the estimated center point is an invalid center point.

5. The method of claim 1, wherein, Before finding the estimated center point of the base type obstacle from the midpoint of the first edge of the bounding box to the first direction of the bounding box perpendicular to the first edge, the method further comprises: acquiring a third distance from the midpoint to a camera; determining an offset amount along the first direction from the midpoint according to a predetermined correspondence between a distance and an offset amount; determining a search starting point of the estimated center point according to the offset amount along the first direction from the midpoint.

6. The method of claim 5, wherein, Before finding the estimated center point of the base type obstacle from the midpoint of the first edge of the bounding box to the first direction of the bounding box perpendicular to the first edge, the method further comprises: determining a maximum search range of the estimated center point according to a predetermined ratio of the width of the first edge of the bounding box.

7. The method according to any one of claims 1 to 6, characterized in that, Before finding the estimated center point of the base type obstacle from the midpoint of the first edge of the bounding box to the first direction of the bounding box perpendicular to the first edge, the method further comprises: determine a first pose of the robot when the image is acquired, and a second pose of the robot when the robot is in a stationary state; correct a position corresponding to a pixel in the image according to the first pose and the second pose.

8. An obstacle position estimation device characterized by comprising: The obstacle is a base type obstacle, and the device comprises: an image acquisition unit configured to acquire an image comprising the base type obstacle; an identification frame determination unit configured to determine an identification frame of the base type obstacle in the image; an estimated center point finding unit configured to find an estimated center point of the base type obstacle in a first direction of the identification frame and perpendicular to a first edge of the identification frame with a midpoint of the first edge of the identification frame as a starting point; a radius determination unit configured to determine a first radius according to the estimated center point and the midpoint, and determine a second radius according to the estimated center point and a direction parallel to the first edge; a target center point determination unit configured to determine a position of a target center point according to a radius deviation between the first radius and the second radius.

9. An obstacle position estimation device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

Images