Method and apparatus for determining the orientation of an object

Abstract

The application provides a coordinate direction determination method and device of an object, and belongs to the technical field of robot vision. The coordinate direction determination method of the object comprises the following steps: obtaining target environment data, wherein the target environment data comprises a wall position and an object point cloud of a target object; determining a target bounding rectangle of the object point cloud, wherein the target bounding rectangle comprises intersecting first and second edges; determining a first distance value between the object point cloud and the target wall according to the wall position; and determining the coordinate direction of the target object according to the ratio of the length of the first edge to the length of the second edge and the first distance value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot vision, in particular to a method and a device for determining the coordinate direction of an object. BACKGROUND

[0002] In the related art, in application scenarios such as robot semantic navigation, it is usually necessary to know the orientation of an object in an environment.

[0003] The orientation of an object is the posture of the object in the real world. If the orientation of an object is to be determined, a method based on template matching is needed in the related art, and therefore an engineering drawing model of the object needs to be known, and the actual orientation of the object is determined according to the orientation of the object marked on the engineering drawing model.

[0004] However, for many scenarios, it is difficult to obtain the engineering drawing model of an object, and therefore it is difficult to determine the orientation of the object. SUMMARY

[0005] The present application aims to at least solve one of the problems in the prior art or the related art.

[0006] To this end, a first aspect of the present application provides a method for determining the coordinate direction of an object.

[0007] A second aspect of the present application provides a device for determining the coordinate direction of an object.

[0008] A third aspect of the present application provides a readable storage medium.

[0009] A fourth aspect of the present application provides a computer program product.

[0010] A fifth aspect of the present application provides an electronic device.

[0011] A sixth aspect of the present application provides a robot.

[0012] Therefore, the first aspect of the present application provides a method for determining the coordinate direction of an object, comprising: obtaining target environment data, the target environment data comprising a wall position and an object point cloud of a target object; determining a target bounding rectangle of the object point cloud, the target bounding rectangle comprising a first side and a second side intersecting with each other; determining a first distance value between the object point cloud and the target wall according to the wall position; and determining the coordinate direction of the target object according to a ratio of the length of the first side to the length of the second side and the first distance value.

[0013] A second aspect of the present invention provides an apparatus for determining the coordinate orientation of an object, comprising: an acquisition module for acquiring a wall position and an object point cloud of a target object; a determination module for determining a target bounding rectangle of the object point cloud, the target bounding rectangle including an intersecting first side and a second side; and determining a first distance value between the object point cloud and the target wall based on the wall position; and determining the coordinate orientation of the target object based on the ratio of the length of the first side to the length of the second side and the first distance value.

[0014] A third aspect of this application provides an apparatus for determining the coordinate orientation of an object, comprising: a memory for storing programs or instructions; and a processor for executing programs or instructions to implement the steps of determining the coordinate orientation of an object as provided in any of the above technical solutions. Therefore, this apparatus for determining the coordinate orientation of an object also includes all the beneficial effects of the method for determining the coordinate orientation of an object as provided in any of the above technical solutions, and will not be repeated here to avoid repetition.

[0015] The fourth aspect of this application provides a readable storage medium having a program or instructions stored thereon. When the program or instructions are executed by a processor, they implement the steps of the method for determining the coordinate orientation of an object as provided in any of the above technical solutions. Therefore, the readable storage medium also includes all the beneficial effects of the method for determining the coordinate orientation of an object as provided in any of the above technical solutions. To avoid repetition, these effects will not be repeated here.

[0016] The fifth aspect of this application provides a computer program product stored in a storage medium. When executed by at least one processor, the computer program product implements the steps of the method for determining the coordinate orientation of an object as provided in any of the above technical solutions. Therefore, the computer program product also includes all the beneficial effects of the method for determining the coordinate orientation of an object as provided in any of the above technical solutions. To avoid repetition, these effects will not be repeated here.

[0017] The sixth aspect of this application provides an electronic device, including a means for determining the coordinate direction of an object as provided in any of the foregoing technical solutions; and / or a readable storage medium as provided in any of the foregoing technical solutions; and / or a computer program product as provided in any of the foregoing technical solutions, thus also including the means for determining the coordinate direction of an object as provided in any of the foregoing technical solutions; and / or a readable storage medium as provided in any of the foregoing technical solutions; and / or all the beneficial effects of the computer program product as provided in any of the foregoing technical solutions will not be repeated here to avoid repetition.

[0018] The seventh aspect of this application provides a robot, including a device for determining the coordinate direction of an object as provided in any of the above technical solutions; and / or a readable storage medium as provided in any of the above technical solutions; and / or a computer program product as provided in any of the above technical solutions, thus also including the device for determining the coordinate direction of an object as provided in any of the above technical solutions; and / or a readable storage medium as provided in any of the above technical solutions; and / or all the beneficial effects of the computer program product as provided in any of the above technical solutions are not described in detail here to avoid repetition.

[0019] The embodiments of this application can determine the orientation of the target object, that is, the coordinate direction of the object in the robot's visual coordinate system, based on the object point cloud and the wall position. This process does not rely on the engineering drawing model of the object and can effectively identify the orientation of various objects, improving the convenience of obtaining the object orientation and thus improving the robot's pathfinding performance. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0021] Figure 1 A flowchart illustrating a method for determining the coordinate orientation of an object according to an embodiment of this application is shown;

[0022] Figure 2 A schematic diagram of a target enclosing rectangle according to an embodiment of this application is shown;

[0023] Figure 3 A schematic diagram of a sub-rectangle according to an embodiment of this application is shown;

[0024] Figure 4 A structural block diagram of an object coordinate orientation determination device according to an embodiment of this application is shown;

[0025] Figure 5 A structural block diagram of an object coordinate orientation determination device according to an embodiment of this application is shown. Detailed Implementation

[0026] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0028] The following reference Figures 1 to 5 This application describes a method and apparatus for determining the coordinate orientation of an object according to some embodiments.

[0029] In some embodiments of this application, a method for determining the coordinate orientation of an object is provided. Figure 1 A flowchart illustrating a method for determining the coordinate orientation of an object according to an embodiment of this application is shown, such as... Figure 1 As shown, the method includes:

[0030] Step 102: Obtain target environment data, which includes the location of the walls and the point cloud of the target objects;

[0031] Step 104: Determine the target bounding rectangle of the object point cloud. The target bounding rectangle includes the intersecting first and second sides.

[0032] Step 106: Determine the first distance value between the object point cloud and the target wall based on the wall's location;

[0033] Step 108: Determine the coordinate direction of the target object based on the ratio of the length of the first side to the length of the second side and the first distance value.

[0034] In the embodiments of this application, the robot includes a home robot, such as a robot vacuum cleaner or a service robot. The home robot can automatically perform certain tasks according to user instructions, such as delivering a TV remote control to a family member watching TV on the living room sofa. At this time, the user can place the TV remote control on the storage compartment of the home robot and give the instruction "deliver the TV remote control to the living room sofa". The robot will then automatically navigate to the "sofa" in the "living room" according to the robot's vision system.

[0035] During robot movement, the robot relies on its vision system to identify routes and target objects. The robot uses image sensors and microwave radar sensors to build indoor point cloud data, and through the recognition and processing of point cloud data, it plans its walking route and finds its destination.

[0036] Since furniture such as sofas and chairs have orientations, taking the scenario of the robot delivering a remote control as an example, in order to accurately deliver the remote control to the person sitting on the sofa, the robot needs to walk to the front of the sofa, rather than the side or back of the sofa. Therefore, the robot needs to identify the orientation of the sofa when delivering the remote control to the front of the sofa.

[0037] Specifically, in this embodiment of the application, robot vision sensors, such as image sensors and microwave radar sensors, are used to acquire environmental data of the robot's environment. The environmental data includes the location of walls and target objects, such as the point cloud of a sofa.

[0038] After identifying the object point cloud of the sofa, the robot further determines the target bounding rectangle of the object point cloud. In some embodiments, the target bounding rectangle includes the minimum bounding rectangle of the object point cloud, which includes a first side and a second side intersecting the first side. If the lengths of the first side and the second side are not the same, then the first side and the second side correspond to the long side and the wide side of the minimum bounding rectangle, respectively.

[0039] After obtaining the target bounding rectangle of the target object, determine the ratio between the length of the first side of the target bounding rectangle and the length of the second side of the target bounding rectangle. This ratio is also the aspect ratio of the target bounding rectangle, and therefore can also express the aspect ratio of the target object.

[0040] According to prior knowledge, for some objects with significant characteristics in terms of length-to-width ratio, after defining the direction from the head to the tail of the object as the object's orientation, the object's orientation is often directly related to its length-to-width ratio. For example, the head and tail of a sofa are generally on the long side of the sofa, while the headboard and footboard of a single bed are generally on the wide side of the single bed.

[0041] At the same time, many furniture items in users' homes are placed against the wall. For example, the back of the sofa and the headboard of the single bed are usually against the wall.

[0042] Therefore, by considering the aspect ratio of the object and the distance between the object and the wall, the orientation of the target object, that is, the coordinate direction of the target object in the robot's vision coordinate system, can be determined.

[0043] The embodiments of this application can determine the orientation of the target object, that is, the coordinate direction of the object in the robot's visual coordinate system, based on the object point cloud and the wall position. This process does not rely on the engineering drawing model of the object and can effectively identify the orientation of various objects, improving the convenience of obtaining the object orientation and thus improving the robot's pathfinding performance.

[0044] Based on any of the above embodiments, determining the target bounding rectangle of the object point cloud includes: obtaining the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate of each point in the object point cloud; determining the coordinates of at least two vertices of the target bounding rectangle based on the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate; and determining the target bounding rectangle based on the coordinates of the at least two vertices.

[0045] In the embodiments of this application, the object point cloud determined based on robot vision includes multiple points, and the coordinate structure of these points is (x, y, z), that is, it includes x-axis coordinates, y-axis coordinates and z-axis coordinates.

[0046] After obtaining the point cloud of the target object, the x-axis and y-axis coordinates of each point in the point cloud are traversed sequentially, and the maximum and minimum values ​​of the x-axis and y-axis coordinates are determined.

[0047] Let the maximum value of the x-axis coordinate be x. max The minimum value of the x-axis coordinate is x min The maximum value of the y-axis coordinate is y max The minimum value of the y-axis coordinate is y min Then we can obtain the target bounding rectangle of the object point cloud, that is, the vertex coordinates of the minimum bounding rectangle are: (x min y min ), (x min y max ), (x max y max ), (x max y min ).

[0048] Figure 2 A schematic diagram of a target enclosing rectangle according to an embodiment of this application is shown, such as... Figure 2 As shown, a rectangle 200 is defined by the above 4 vertices. This rectangle 200 is the minimum bounding rectangle of the object point cloud 202, which is also the target bounding rectangle.

[0049] This application determines the coordinate orientation of an object in the robot's visual coordinate system by using the minimum bounding rectangle of the object's point cloud, without relying on the object's engineering drawing model, thus improving the convenience of obtaining the object's orientation.

[0050] Based on any of the above embodiments, before determining the first distance value between the object point cloud and the target wall according to the wall position, the method further includes: determining the target wall, which is the wall closest to the target object, according to the wall position.

[0051] In this embodiment, the robot can determine the location information of each wall in the current indoor environment based on the environmental data obtained from scanning. Since there may be multiple walls indoors, each of these walls can be assigned a unique wall number.

[0052] After identifying the target object, based on the position of the target object and the position of the walls, find the wall that is closest to the target object among the multiple walls, that is, determine which wall the target object is "leaning" against.

[0053] The wall closest to the target object is identified as the target wall. The orientation of the target object is then determined based on the specific distance between the target object and the target wall.

[0054] It is understandable that when the target object is equidistant from multiple walls, such as when the target object is placed at the angle between two walls, either of those two walls can be designated as the target wall.

[0055] Based on any of the above embodiments, the step of determining a first distance value between the object point cloud and the target wall according to the wall position is performed, specifically including: determining the target wall point on the target wall, the target wall point including the projection point of the point in the object point cloud on the target wall and / or the endpoint of the target wall; determining the first distance value according to the minimum distance between each point in the object point cloud and the target wall point.

[0056] In this embodiment of the application, the target wall point is determined by the projection of the target object onto the target wall, and a first distance value is determined based on the distance between the target wall point and the point in the object point cloud.

[0057] Specifically, environmental data about the robot's environment is acquired through the robot's vision sensors. This environmental data includes wall locations and point clouds of target objects, such as sofas. The wall locations can be determined based on the wall point cloud data collected by the robot, or by inputting a room layout diagram into the robot's system.

[0058] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. When determining the distance between the object's point cloud and the target wall, each point in the object's point cloud can be traversed, and the distance between each point and the target wall can be determined one by one.

[0059] Specifically, the object point cloud includes point A. If the projection of point A onto the plane where the target wall is located is within the range of the target wall, then this projection is taken as the target wall point, and the length of the line segment between point A and the target wall point is determined. This line segment length is the distance between point A and the target wall.

[0060] If the projection of point A onto the plane containing the target wall is outside the range of the target wall, then the two endpoints of the line segment representing the target wall are taken as the target wall points, and the lengths of the line segments between point A and these two endpoints are determined respectively. The smaller of the two line segment lengths is determined as the distance between point A and the target wall.

[0061] After obtaining the distance values ​​between all points of the object's point cloud and the target wall, the minimum value among these distance values ​​is determined as the first distance value between the target point cloud and the target wall. This first distance value can be used to determine the minimum distance between the target object and the target wall, thereby determining whether a target object is placed against the wall.

[0062] It is understandable that the midpoints of the four sides of the minimum bounding rectangle of the object's point cloud can be determined separately, the distances between the four midpoints and the target wall points can be determined separately, and the minimum value among the four distances between the four midpoints and the target wall points can be determined as the first distance value.

[0063] This application embodiment determines a first distance value between the object point cloud and the target wall, uses the first distance value to determine whether the target object is against the wall, and determines the orientation of the target object based on the determination result, that is, the coordinate direction of the target object in the robot's visual coordinate system, which can improve the robot's pathfinding performance.

[0064] Based on any of the above embodiments, the step of determining the first distance value between the object point cloud and the target wall according to the wall position is performed, specifically including: generating N wall points according to the wall position in the robot vision coordinate system, the N wall points being distributed along the extension direction of the wall at preset intervals; and determining the first distance value according to the minimum value of the distance between each point in the object point cloud and the N wall points.

[0065] In this embodiment of the application, the wall points are obtained by fitting the point cloud of the wall, and a first distance value is determined based on the distance between the target wall points and the points in the object point cloud.

[0066] Specifically, environmental data about the robot's environment is acquired through the robot's vision sensors. This environmental data includes wall locations and point clouds of target objects, such as sofas. The wall locations can be determined based on the wall point cloud data collected by the robot, or by inputting a room layout diagram into the robot's system.

[0067] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. After obtaining the line segment corresponding to each wall, a wall point is generated at preset intervals, starting from one endpoint of the wall line segment. Thus, N wall points can be obtained on the line segment corresponding to one wall.

[0068] These wall points form a simplified point cloud of the wall. When determining the distance between the object point cloud and the target wall, the distance between each point in the object point cloud and each of the N wall points can be determined separately. The minimum value among these distances is determined as the first distance between the object point cloud and the target wall, thereby determining whether a target object is placed against the wall.

[0069] It is understandable that the midpoints of the four sides of the minimum bounding rectangle of the object's point cloud can be determined separately, the distances between the four midpoints and the N wall points can be determined separately, and the minimum value among all the distances between the four midpoints and the N wall points can be determined as the first distance value.

[0070] This application embodiment determines a first distance value between the object point cloud and the target wall, uses the first distance value to determine whether the target object is against the wall, and determines the orientation of the target object based on the determination result, that is, the coordinate direction of the target object in the robot's visual coordinate system, which can improve the robot's pathfinding performance.

[0071] Based on any of the above embodiments, before determining the first distance value between the object point cloud and the target wall according to the wall position, the method further includes:

[0072] Get the number of walls;

[0073] When the number of walls is less than the number threshold, the target wall points on the target wall are determined. The target wall points include the projection points of the points in the object point cloud onto the target wall and / or the endpoints of the target wall.

[0074] The first distance value is determined based on the minimum distance between each point in the object point cloud and the point on the target wall.

[0075] When the number of walls is greater than or equal to the threshold, N wall points are generated according to the wall positions in the robot vision coordinate system. The N wall points are distributed along the extension direction of the wall at preset intervals, where N is a positive integer.

[0076] The first distance value is determined based on the minimum distance between each point in the object point cloud and N wall points.

[0077] In this embodiment, the room point cloud may contain multiple walls. When there are many walls, the target object is more likely to be close to multiple walls simultaneously; conversely, when there are few walls, the target object may only be close to one wall. Therefore, selecting a corresponding method to determine the first distance value between the object point cloud and the target wall based on a comparison between the number of walls and a threshold value can improve the algorithm's speed and effectively ensure the accuracy of the distance value, ultimately improving the reliability of the object's coordinate orientation.

[0078] It is understandable that the quantity threshold can be a preset fixed value, or it can be dynamically adjusted based on factors such as room area and room layout. For example, the quantity threshold is 6.

[0079] Specifically, when the number of walls is small, the target wall points can be determined by projecting the target object onto the target wall, and a first distance value can be determined based on the distance between the target wall points and the points in the object's point cloud.

[0080] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. When determining the distance between the object's point cloud and the target wall, each point in the object's point cloud can be traversed, and the distance between each point and the target wall can be determined one by one.

[0081] After obtaining the distance values ​​between all points of the object's point cloud and the target wall, the minimum value among these distance values ​​is determined as the first distance value between the target point cloud and the target wall. This first distance value can be used to determine the minimum distance between the target object and the target wall, thereby determining whether a target object is placed against the wall.

[0082] When there are many walls, the wall points can be obtained by fitting the point cloud of the walls. Based on the distance between the target wall points and the points in the object point cloud, the first distance value is determined.

[0083] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. After obtaining the line segment corresponding to each wall, a wall point is generated at preset intervals, starting from one endpoint of the wall line segment. Thus, N wall points can be obtained on the line segment corresponding to one wall.

[0084] These wall points form a simplified point cloud of the wall. When determining the distance between the object point cloud and the target wall, the distance between each point in the object point cloud and each of the N wall points can be determined separately. The minimum value among these distances is determined as the first distance between the object point cloud and the target wall, thereby determining whether a target object is placed against the wall.

[0085] Based on any of the above embodiments, determining the coordinate direction of the target object according to the ratio of the length of the first side to the length of the second side and the first distance value includes: when the first distance value is less than a distance threshold, determining the head coordinates and tail coordinates of the target object according to the second distance between each side of the target enclosing rectangle and the target wall; when the first distance value is greater than or equal to the distance threshold, determining the head coordinates and tail coordinates of the target object according to the ratio and the coordinate values ​​of each point in the object point cloud; and determining the coordinate direction of the target object according to the head coordinates and tail coordinates.

[0086] In this embodiment of the application, if the first distance value, that is, the minimum distance between the object point cloud and the target wall, is less than the distance threshold, it indicates that the target object is close to the target wall and the target object is placed against the target wall.

[0087] At this point, the four sides of the minimum bounding rectangle of the object point cloud are determined, which is the second distance between the four sides of the target rectangle and the target wall. Based on the second distance, the "head point" representing the head of the target object and the "tail point" representing the tail of the target object are determined on the object point cloud. The coordinates of the head point (head coordinates) and the coordinates of the tail point (tail coordinates) are then used to determine the coordinates of the tail point.

[0088] If the first distance value is greater than or equal to the distance threshold, it indicates that the distance between the target object and the target wall is relatively far, and the target object is not placed against the wall. In this case, the coordinate direction of the target object can be determined based on the minimum bounding rectangle of the object's point cloud, which is the ratio of the first side to the second side of the target's bounding rectangle, i.e., the aspect ratio of the target object.

[0089] Specifically, based on prior knowledge, for some objects with significant characteristics in terms of aspect ratio, after defining the direction from the head to the tail of the object as the object's orientation, the object's orientation is often directly related to its aspect ratio. For example, the head and tail of a sofa are generally on the long side of the sofa, while the head and foot of a single bed are generally on the wide side of the single bed.

[0090] Furthermore, these objects also have distinct shapes. For example, the rear of a sofa is defined as the seat cushion, and the head of the sofa as the backrest. The backrest is higher than the seat cushion. Similarly, the head of a single bed is defined as the headboard, and the rear of the single bed is defined as the footboard. The headboard is generally higher than the footboard. This height information can be expressed using the point coordinates in the object's point cloud, specifically the z-axis coordinates.

[0091] Therefore, for objects that are not against a wall, the head and tail positions of the target object can be determined by the aspect ratio of the target object and the coordinate values ​​of the points in the object's point cloud, thereby determining the head and tail coordinates.

[0092] After obtaining the head and tail coordinates, let the head coordinates be (x1, y1) and the tail coordinates be (x2, y2). Then, the coordinate direction of the target object can be calculated using the following formula:

[0093]

[0094] Where θ is the angle of the target object's coordinate direction, (x1, y1) is the head coordinate, and (x2, y2) is the tail coordinate.

[0095] This application embodiment determines the head and tail coordinates of the target object, determines the object's coordinate direction based on the head and tail coordinates, and then determines the object's orientation, thereby improving the pathfinding performance based on robot vision.

[0096] Based on any of the above embodiments, the head coordinates and tail coordinates of the target object are determined according to the second distance between each side of the target enclosing rectangle and the target wall, including: determining the coordinates of the midpoint of the third side corresponding to the minimum value of the second distance as the head coordinates; and determining the coordinates of the midpoint of the opposite side of the third side in the target enclosing rectangle as the tail coordinates.

[0097] In this embodiment of the application, the minimum bounding rectangle of the object point cloud, that is, the target bounding rectangle, includes 4 sides. If the first distance value is less than the distance threshold, it means that the distance between the target object and the target wall is relatively close, and the target object is placed against the target wall. At this time, the side of the target object against the wall is defined as the head. Then, the midpoint of the side closest to the target wall among the 4 sides of the target bounding rectangle can be defined as the head point. Therefore, the coordinates of the midpoint of the third side closest to the second distance from the target wall are determined as the head coordinates and used to mark the head of the target object.

[0098] Meanwhile, if the side of the target object furthest from the wall is defined as the tail, then the midpoint of the second side furthest from the target wall among the four sides of the target's enclosing rectangle, opposite to the third side, is defined as the aforementioned tail point, and the coordinates of the midpoint of the opposite side of the third side are determined as the aforementioned tail coordinates.

[0099] After obtaining the head and tail coordinates, the coordinate direction of the target object is calculated based on the coordinate values ​​of the head and tail coordinates.

[0100] This application embodiment determines the object's orientation, i.e., the object's coordinate direction in the robot's visual coordinate system, by using the distances between the four sides of the minimum bounding rectangle of the object's point cloud and the wall. This method can easily and quickly determine the object's orientation without needing to obtain the object's engineering drawing model, thus improving the robot's pathfinding performance.

[0101] Based on any of the above embodiments, the head coordinates and tail coordinates of the target object are determined according to the ratio and the coordinate values ​​of each point in the object point cloud, including: when the ratio is less than a ratio threshold, dividing the target enclosing rectangle into multiple sub-rectangles with the intersection of the first and second sides, the midpoint of the first side, the midpoint of the second side, and the center point of the target enclosing rectangle as vertices; determining the average value of the z-axis coordinates of each point in every two adjacent sub-rectangles; determining the adjacent first and second sub-rectangles corresponding to the maximum value of the average value; determining the head coordinates according to the coordinates of the first intersection point of the common side of the first and second sub-rectangles and the side of the target enclosing rectangle; and determining the tail coordinates according to the coordinates of the midpoint of the side opposite the side where the first intersection point is located in the side of the target enclosing rectangle.

[0102] In the embodiments of this application, based on prior knowledge, the orientation of some common indoor objects is strongly related to their shape. For example, in a sofa, the head of the sofa, which is the backrest, is significantly higher than the back of the sofa, i.e., the seat. Similarly, in a bed, the headboard is significantly higher than the bed frame.

[0103] Therefore, in practical application scenarios, such as giving a TV remote control to someone sitting on a sofa, the orientation of the object can be determined based on the height of each part of the sofa, that is, the z-axis coordinate of each point.

[0104] Specifically, the ratio threshold, which is a threshold used to determine the shape of a target object, is greater than or equal to 1 in some embodiments. In some embodiments, the first side can be preset as the long side and the second side as the short side. If the ratio of the length of the first side of the target object to the length of the second side of the target object, that is, the aspect ratio of the target object, is less than the ratio threshold, it indicates that the shape of the target object is relatively square, and the long side and the short side of the minimum bounding rectangle of the target object are relatively close. At this time, based on prior knowledge, all four sides of the minimum bounding rectangle of the target object may be the "head" or "tail" of the target object.

[0105] The minimum bounding rectangle of the target object, also known as the target bounding rectangle, is divided into multiple sub-rectangles of equal area. The target bounding rectangle, or the minimum bounding rectangle of the target object, includes an intersecting first side and a second side. The intersection of the first and second sides is a vertex of the minimum bounding rectangle.

[0106] Taking the first side as the long side of the target enclosing rectangle and the second side as the wide side of the target enclosing rectangle as an example, the first line segment is determined by taking the midpoint of the two long sides of the target enclosing rectangle as the endpoint, and the second line segment is determined by taking the midpoint of the two wide sides of the target enclosing rectangle as the endpoint. The intersection of the first line segment and the second line segment is the midpoint of the target enclosing rectangle.

[0107] Figure 3 A schematic diagram of a sub-rectangle according to an embodiment of this application is shown, such as... Figure 3 As shown, a sub-rectangle is defined using one vertex of the target-enclosing rectangle, the midpoint of the adjacent long and short sides, and the midpoint of the target-enclosing rectangle as vertices. This results in four sub-rectangles of equal area, effectively dividing the target-enclosing rectangle into four equal sub-rectangles. Each of these four sub-rectangles has an adjacent sub-rectangle on the short side and a adjacent sub-rectangle on the long side of the target-enclosing rectangle.

[0108] After obtaining the four sub-rectangles, the z-axis coordinates of each point within the area of ​​each sub-rectangle are determined, and the average value of these z-axis coordinates is calculated. This average value can intuitively reflect the height of the target object within the area of ​​a sub-rectangle.

[0109] The height of the target object on a certain side can be determined by averaging the z-axis coordinates within two adjacent sub-rectangles. Therefore, after obtaining the average z-axis coordinates of each point in every two adjacent sub-matrices, the object orientation, i.e., the coordinate direction of the target object in the robot's vision coordinate system, can be determined by the aspect ratio of the target object (i.e., the ratio of the length of the first side to the length of the second side) and the height of the target object on each side (i.e., the first and second sides).

[0110] Specifically, the target enclosing rectangle is divided into 4 sub-rectangles. These 4 sub-rectangles consist of 2 adjacent sub-rectangles on the long side of the target enclosing rectangle and 2 adjacent sub-rectangles on the wide side of the target enclosing rectangle, for a total of 4 sets of adjacent sub-rectangles.

[0111] Among these four groups of adjacent sub-rectangles, the group with the highest average z-axis coordinate value is determined. This group of adjacent sub-rectangles specifically includes the first sub-rectangle and the second sub-rectangle. The first intersection point of the common side of the first and second sub-rectangles with its corresponding long or wide side is defined as the "head point" of the target object, and its coordinates are recorded as the head coordinates of the target object. The midpoint of the side opposite the first intersection point among the four sides of the target's enclosing rectangle is defined as the "tail point" of the target object, and its coordinates are recorded as the tail coordinates of the target object.

[0112] This application determines the orientation, or coordinate direction, of the target object by dividing the minimum bounding rectangle of the target object into multiple adjacent sub-rectangles and combining the aspect ratio of the minimum bounding rectangle. This process does not rely on the engineering drawing model of the object and can determine the object's orientation more quickly.

[0113] Based on any of the above embodiments, when the ratio is greater than or equal to the ratio threshold, the fourth side of the target enclosing rectangle is determined according to the object type of the target object; the adjacent third and fourth sub-rectangles corresponding to the maximum value of the average value are determined, wherein the third and fourth sub-rectangles are adjacent on the fourth side; the head coordinates are determined according to the coordinates of the second intersection point of the common side of the third and fourth sub-rectangles and the fourth side; and the tail coordinates are determined according to the coordinates of the midpoint of the side opposite the side where the second intersection point is located in the target enclosing rectangle.

[0114] In this application embodiment, a preset threshold, which is a threshold used to determine the shape of the target object, is a constant greater than or equal to 1 in some embodiments. In some embodiments, the first side can be preset as the long side and the second side as the short side. If the ratio of the length of the first side of the target object to the length of the second side of the target object, that is, the aspect ratio of the target object, is greater than or equal to the preset threshold, it indicates that the difference between the length of the long side and the length of the short side of the minimum enclosing rectangle of the target object is large.

[0115] Based on prior knowledge, for scenarios where the target object is a sofa, the two longest sides of the minimum bounding rectangle of the target object are more likely to be the "head" or "tail" of the target object. For scenarios where the target object is a single bed, the two shortest sides of the minimum bounding rectangle of the target object are more likely to be the "head" or "tail" of the target object.

[0116] At this point, based on the object type of the target object, it is determined that the head and tail of this type of object are generally on the long side or the wide side. Taking the target object as a sofa as an example, the head of the sofa generally refers to the back of the sofa, and the back of the sofa is generally on the long side of the minimum bounding rectangle of the object point cloud of the sofa. Therefore, the long side of the target bounding rectangle can be recorded as the fourth side, and in the four sub-rectangles divided by the minimum bounding rectangle, two sets of two adjacent sub-rectangles on the long side, that is, the fourth side, are determined.

[0117] It is understandable that if the target object is a single bed, and the headboard and bed frame of the single bed are on the wide side of the minimum bounding rectangle of the object point cloud of the single bed, then the wide side of the bounding rectangle of the target is denoted as the fourth side.

[0118] Among the two adjacent sub-rectangles on the fourth side, determine the pair of adjacent sub-rectangles with the highest average z-axis coordinate values. This pair of adjacent sub-rectangles specifically includes the third and fourth sub-rectangles. The third and fourth sub-rectangles are adjacent on the fourth side. The second intersection point of the common edge of the third and fourth sub-rectangles with the fourth side is determined as the "head point" of the target object, and the coordinates of this second intersection point are taken as the head coordinates of the target object. The midpoint of the side opposite the edge containing the second intersection point among the four sides of the target's enclosing rectangle is recorded as the "tail point" of the target object, and its coordinates are taken as the tail coordinates of the target object.

[0119] This application embodiment determines the head and tail coordinates of the target object, determines the object's coordinate direction based on the head and tail coordinates, and then determines the object's orientation, thereby improving the pathfinding performance based on robot vision.

[0120] In some embodiments of this application, the length, width, and height of the object's point cloud are aligned with the coordinate system, and the height direction is aligned with the Z-axis.

[0121] The orientation of an object is determined by the angle between the vector AB formed by the object's head point A and tail point B, and the X-axis, ranging from -180° to 180°. For some objects, the head is generally higher than the tail, such as beds and sofas. For objects with a large length-to-width ratio, such as beds, the headboard is usually on the shorter side, while the headboard of a sofa is usually on the longer side. Also, when an object is close to a wall, its tail is generally farther from the wall, while its head is closer. Therefore, combining the length-to-width ratio and the distance to the wall can improve the accuracy of determining the object's head and tail.

[0122] The specific method is as follows:

[0123] Find the minimum bounding rectangle of the object's point cloud to obtain the object's aspect ratio:

[0124] Iterate through the x and y coordinates of each point in the object's point cloud, and find the maximum and minimum values ​​of the x and y coordinates respectively, let's say x. max x min y max y min The coordinates of the four vertices of the minimum bounding rectangle of the object's point cloud are: (x... min y min ), (x min y max ), (x max y max ), (x max y min ).

[0125] Take x max -x min and y max-y min If the larger value in the equation is the length L of the object and the smaller value is the width W of the object, then the aspect ratio S of the object is S = L ÷ W.

[0126] Obtain wall data:

[0127] Wall data can be extracted from wall point clouds using relevant algorithms, or wall data can be labeled using relevant software based on wall point clouds. The wall data contains many walls, each represented by a line segment.

[0128] Calculate the shortest distance between the midpoint of each of the four sides of the rectangle and the wall, and the corresponding wall number:

[0129] There are two methods to calculate the shortest distance between the midpoint of the four sides of a rectangle and the wall.

[0130] Method 1: Assign a unique number to each wall as the wall number. Find the shortest distance from the midpoint of the four sides of the minimum bounding rectangle of the object's point cloud to each wall. Then take the minimum value as the shortest distance between the midpoint and the wall. It is easy to obtain the wall number of the wall that is closest to the midpoint.

[0131] The method for calculating the shortest distance from the midpoint to each wall is as follows: If the vertical projection of the midpoint onto the wall is located inside the wall, then the shortest distance is the vertical distance from the midpoint to the wall; otherwise, calculate the distance from the midpoint to the two endpoints of the wall, and take the minimum of the two endpoints as the shortest distance.

[0132] Method 2: Assign a unique number to each wall as the wall number. For each wall, generate equally spaced points at certain intervals and record which wall each point belongs to. These equally spaced points form a wall point cloud. A kd-tree can be built using the wall point cloud. Based on the kd-tree, the equally spaced point closest to the midpoint and its index in the point cloud can be found.

[0133] Based on the distance between the nearest equally spaced point and the midpoint, which is taken as the closest distance between the midpoint and the wall, and based on its index in the point cloud and the previously recorded wall to which the equally spaced point belongs, the wall number of the wall closest to the midpoint can be obtained.

[0134] The minimum distance is obtained from the previously calculated shortest distances between the midpoints of the four sides of the rectangle and the wall. If the minimum distance is less than a threshold, the coordinates of the head and tail points of the object are calculated by combining the object's aspect ratio and the distance from the midpoint to the wall.

[0135] Prior knowledge: such as Figure 3 As shown, the head and tail of the object are located at the midpoints of the four sides of the smallest bounding rectangle of the object's point cloud. If the head of the object is known to be located at the midpoint of one of the sides, then the tail of the object is located at the midpoint of the opposite side of that side. For example, if the head of the object is point 3, then the tail of the object is point 1.

[0136] When all the wall numbers are the same, the orientation of the object is determined only based on the walls:

[0137] Among the shortest distances from the midpoints of the four sides of the previously calculated rectangle to the walls, the midpoint with the largest distance is the tail of the object, and the head of the object can be obtained based on prior knowledge.

[0138] When all the wall numbers are different, the orientation of the object is determined by combining the aspect ratio of the object and the walls:

[0139] When the aspect ratio of the object is less than k (1 < k):

[0140] It means the object is relatively square. At this time, among the shortest distances from the midpoints of the four sides of the previously calculated rectangle to the walls, the midpoint with the smallest distance is the head of the object, and the tail of the object can be obtained based on prior knowledge.

[0141] When the aspect ratio of the object is greater than k:

[0142] If the head of the object is generally located on the long side of the minimum bounding rectangle of the object point cloud, such as a sofa. Then determine which midpoint corresponding to the long side has a smaller distance to the wall, and this midpoint is the head of the object, and the tail of the object can be obtained based on prior knowledge.

[0143] If the head of the object is generally located on the short side of the minimum bounding rectangle of the object point cloud, such as a bed, then determine which midpoint corresponding to the short side has a smaller distance to the wall, and this midpoint is the head of the object, and the tail of the object can be obtained based on prior knowledge.

[0144] Among the shortest distances from the midpoints of the four sides of the previously calculated rectangle to the walls, obtain the minimum distance. If the minimum distance is greater than the threshold, use the method based on the object point cloud to calculate the head and tail of the object:

[0145] Find the minimum bounding rectangle of the object point cloud to obtain the aspect ratio of the object:

[0146] Traverse the x - coordinate and y - coordinate of each point in the object point cloud in turn, and find the maximum and minimum values of the x - coordinate and y - coordinate respectively. Assume they are x max 、x min 、y max 、y min . Then the four vertex coordinates of the minimum bounding rectangle of the object point cloud are: (x min , y min ), (x min , y​​​​​​​​​​​

[0147] Take x max -x min and y max -y min If the larger value in the equation is the length L of the object and the smaller value is the width W of the object, then the aspect ratio S of the object is S = L ÷ W.

[0148] Divide the rectangle into four equal regions:

[0149] Assume the midpoints of the four sides of a rectangle are point 1, point 2, point 3, and point 4. Based on the coordinates of the four vertices of the rectangle, calculate the coordinates of the midpoints of the four sides. Connect the midpoints of opposite sides to divide the rectangle into four equal regions A, B, C, and D. Figure 3 As shown. Each region is a rectangle, and based on the previous information, it is easy to obtain the vertex coordinates of the rectangle corresponding to each region.

[0150] Each point in the point cloud is divided into four regions:

[0151] For each region, find the maximum and minimum values ​​of the coordinates of the four vertices of its corresponding rectangle along the x-axis and y-axis, respectively. Let's assume they are x... max x min y max y min If the coordinates of a point in a point cloud are (x, y, z), and its coordinates satisfy x min <x<x max And y min <y<y max If the point is in the specified region, then that point should be assigned to that region. For each point in the point cloud, assign it to the corresponding region using the method described above.

[0152] Sort the points in each region by their z-coordinate from largest to smallest, and keep the first N points:

[0153] When the number of points in each region is greater than N, sort the points in each region by their z-coordinate from largest to smallest and keep the first N points. When the number of points in each region is less than N, do not perform any processing.

[0154] Find the average z-coordinate of all points in the adjacent region:

[0155] like Figure 3 As shown, there are a total of 4 adjacent regions: region A and region B, region B and region C, region C and region D, and region D and region A. For ease of description later, these adjacent regions are abbreviated as: adjacent region AB, adjacent region BC, adjacent region CD, and adjacent region DA. Find the average z-coordinate of all points in the adjacent regions.

[0156] Calculate the coordinates of the corresponding points at the head and tail of the object by combining the aspect ratio and the average z-coordinate.

[0157] Conclusion 1: The head and tail of an object are among the midpoints of the four sides of the minimum bounding rectangle of the object's point cloud. If it is known that the head of the object is at the midpoint of one of the sides, then the tail of the object is the midpoint of the opposite side. For example, if the head of the object is point 3, then the tail of the object is point 1.

[0158] Each adjacent region is associated with the midpoint of one of the sides of the minimum bounding rectangle of the object's point cloud. The associated points of adjacent regions AB, BC, CD, and DA are: point 2, point 3, point 4, and point 1 respectively.

[0159] Method 1:

[0160] When the aspect ratio of the object is less than k (1 < k), the possible number of positions of the object's head is 4. Obtain the adjacent region corresponding to the maximum average Z coordinate, and the associated point of this adjacent region is the object's head. The object's tail can be obtained according to Conclusion 1.

[0161] When the aspect ratio of the object is greater than k, combine the aspect ratio of the object and the possible positions of the head to determine the possible positions of the object's head. At this time, the possible number of positions of the object's head is 2.

[0162] If the head of the object is generally located on the long side of the minimum bounding rectangle of the object's point cloud, such as a sofa, then select the adjacent region whose associated point is on the long side of the minimum bounding rectangle of the object's point cloud from all adjacent regions, and obtain the adjacent region corresponding to the maximum average z coordinate. The associated point of this adjacent region is the object's head. The object's tail can be obtained according to Conclusion 1.

[0163] If the head of the object is generally located on the short side of the minimum bounding rectangle of the object's point cloud, such as a bed, then select the adjacent region whose associated point is on the short side of the minimum bounding rectangle of the object's point cloud from all adjacent regions, and obtain the adjacent region corresponding to the maximum average Z coordinate. The associated point of this adjacent region is the object's head. The object's tail can be obtained according to Conclusion 1.

[0164] Method 2:

[0165] Objects in general families are relatively symmetric. If the object is symmetric, then the average Z coordinates of the two adjacent regions obtained by being divided by the line connecting the head and tail points should not differ much. Combining this prior knowledge, calculate the absolute value of the difference between the average Z coordinates of adjacent regions AB and CD and the absolute value of the difference between the average Z coordinates of adjacent regions BC and DA, and determine which absolute value is larger. For the two corresponding adjacent regions with the larger absolute value, determine which adjacent region has the largest average Z coordinate. The associated point of this adjacent region is the object's head. The object's tail can be obtained according to Conclusion 1.

[0166] After obtaining the head and tail coordinates, let the head coordinates be (x1, y1) and the tail coordinates be (x2, y2). Then, the coordinate direction of the target object can be calculated using the following formula:

[0167]

[0168] Where θ is the angle of the target object's coordinate direction, (x1, y1) is the head coordinate, and (x2, y2) is the tail coordinate.

[0169] In some embodiments of this application, a device for determining the coordinate orientation of an object is provided. Figure 4 A structural block diagram of an object coordinate orientation determination device according to an embodiment of this application is shown, such as... Figure 4 As shown, the object's coordinate orientation determination device 400 includes:

[0170] The acquisition module 402 is used to acquire the wall position and the object point cloud of the target object;

[0171] The determination module 404 is used to determine the target bounding rectangle of the object point cloud, the target bounding rectangle including the intersecting first side and second side; and to determine the first distance value between the object point cloud and the target wall based on the wall position; and to determine the coordinate direction of the target object based on the ratio of the length of the first side to the length of the second side and the first distance value.

[0172] In the embodiments of this application, the robot includes a home robot, such as a robot vacuum cleaner or a service robot. The home robot can automatically perform certain tasks according to user instructions, such as delivering a TV remote control to a family member watching TV on the living room sofa. At this time, the user can place the TV remote control on the storage compartment of the home robot and give the instruction "deliver the TV remote control to the living room sofa". The robot will then automatically navigate to the "sofa" in the "living room" according to the robot's vision system.

[0173] During the robot's movement, it relies on the robot's vision system to identify routes and target objects. In the process, the robot uses image sensors and microwave radar sensors to build indoor point cloud data. By recognizing and processing the point cloud data, it plans its walking route and finds its destination.

[0174] Since furniture such as sofas and chairs have orientations, taking the scenario of the robot delivering a remote control as an example, in order to accurately deliver the remote control to the person sitting on the sofa, the robot needs to move to the front of the sofa, rather than the side or back. Therefore, the robot needs to identify the orientation of the sofa when delivering the remote control.

[0175] Specifically, in this embodiment of the application, robot vision sensors, such as image sensors and microwave radar sensors, are used to acquire environmental data of the robot's environment. The environmental data includes the location of walls and target objects, such as the point cloud of a sofa.

[0176] After identifying the object point cloud of the sofa, the robot further determines the target bounding rectangle of the object point cloud. Specifically, the target bounding rectangle is the minimum bounding rectangle of the object point cloud. The target bounding rectangle includes a first side and a second side that intersects with the first side. If the lengths of the first side and the second side are not the same, then the first side and the second side correspond to the long side and the wide side of the minimum bounding rectangle, respectively.

[0177] After obtaining the target bounding rectangle of the target object, determine the ratio between the length of the first side of the target bounding rectangle and the length of the second side of the target bounding rectangle. This ratio is also the aspect ratio of the target bounding rectangle, and therefore can also express the aspect ratio of the target object.

[0178] According to prior knowledge, for some objects with significant characteristics in terms of length-to-width ratio, after defining the direction from the head to the tail of the object as the object's orientation, the object's orientation is often directly related to its length-to-width ratio. For example, the head and tail of a sofa are generally on the long side of the sofa, while the headboard and footboard of a single bed are generally on the wide side of the single bed.

[0179] At the same time, many furniture items in users' homes are placed against the wall. For example, the back of the sofa and the headboard of the single bed are usually against the wall.

[0180] Therefore, by considering the aspect ratio of the object and the distance between the object and the wall, the orientation of the target object, that is, the coordinate direction of the target object in the robot's vision coordinate system, can be determined.

[0181] The embodiments of this application can determine the orientation of the target object, that is, the coordinate direction of the object in the robot's visual coordinate system, based on the object point cloud and the wall position. This process does not rely on the engineering drawing model of the object and can effectively identify the orientation of various objects, improving the convenience of obtaining the object orientation and thus improving the robot's pathfinding performance.

[0182] Based on any of the above embodiments, the determining module is further configured to: obtain the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate of each point in the object point cloud; determine the coordinates of at least two vertices of the target enclosing rectangle based on the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate; and determine the target enclosing rectangle based on the coordinates of the at least two vertices.

[0183] In the embodiments of this application, the object point cloud determined based on robot vision includes multiple points, and the coordinate structure of these points is (x, y, z), that is, it includes x-axis coordinates, y-axis coordinates and z-axis coordinates.

[0184] After obtaining the object point cloud of the target object, the x-axis coordinates and y-axis coordinates of each point in the object point cloud are traversed sequentially, and the maximum and minimum values ​​of the x-axis and y-axis coordinates are determined.

[0185] Let the maximum value of the x-axis coordinate be x. max The minimum value of the x-axis coordinate is x min The maximum value of the y-axis coordinate is y max The minimum value of the y-axis coordinate is y min Then we can obtain the target bounding rectangle of the object point cloud, that is, the vertex coordinates of the minimum bounding rectangle are: (x min y min ), (x min y max ), (x max y max ), (x max y min ).

[0186] Figure 2 A schematic diagram of a target enclosing rectangle according to an embodiment of this application is shown, such as... Figure 2 As shown, a rectangle 200 is defined by the above 4 vertices. This rectangle 200 is the minimum bounding rectangle of the object point cloud 202, which is also the target bounding rectangle.

[0187] This application determines the coordinate orientation of an object in the robot's visual coordinate system by using the minimum bounding rectangle of the object's point cloud, without relying on the object's engineering drawing model, thus improving the convenience of obtaining the object's orientation.

[0188] Based on any of the above embodiments, the determining module is further configured to: determine the target wall based on the wall position, wherein the target wall is the wall closest to the target object.

[0189] In this embodiment, the robot can determine the location information of each wall in the current indoor environment based on the environmental data obtained from scanning. Since there may be multiple walls indoors, each of these walls can be assigned a unique wall number.

[0190] After identifying the target object, based on the position of the target object and the position of the walls, find the wall that is closest to the target object among the multiple walls, that is, determine which wall the target object is "leaning" against.

[0191] The wall closest to the target object is identified as the target wall. The orientation of the target object is then determined based on the specific distance between the target object and the target wall.

[0192] It is understandable that when the target object is equidistant from multiple walls, such as when the target object is placed at the angle between two walls, either of these two walls can be designated as the target wall.

[0193] Based on any of the above embodiments, the determining module is further configured to: determine the target wall points on the target wall, the target wall points including the projection points of points in the object point cloud onto the target wall and / or the endpoints of the target wall; and determine a first distance value based on the minimum distance between each point in the object point cloud and the target wall points.

[0194] In this embodiment of the application, the target wall point is determined by the projection of the target object onto the target wall, and a first distance value is determined based on the distance between the target wall point and the point in the object point cloud.

[0195] Specifically, environmental data about the robot's environment is acquired through the robot's vision sensors. This environmental data includes wall locations and point clouds of target objects, such as sofas. The wall locations can be determined based on the wall point cloud data collected by the robot, or by inputting a room layout diagram into the robot's system.

[0196] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. When determining the distance between the object's point cloud and the target wall, each point in the object's point cloud can be traversed, and the distance between each point and the target wall can be determined one by one.

[0197] Specifically, the object point cloud includes point A. If the projection of point A onto the plane where the target wall is located is within the range of the target wall, then this projection is taken as the target wall point, and the length of the line segment between point A and the target wall point is determined. This line segment length is the distance between point A and the target wall.

[0198] If the projection of point A onto the plane containing the target wall is outside the range of the target wall, then the two endpoints of the line segment representing the target wall are taken as the target wall points, and the lengths of the line segments between point A and these two endpoints are determined respectively. The smaller of the two line segment lengths is determined as the distance between point A and the target wall.

[0199] After obtaining the distance values ​​between all points of the object's point cloud and the target wall, the minimum value among these distance values ​​is determined as the first distance value between the target point cloud and the target wall. This first distance value can be used to determine the minimum distance between the target object and the target wall, thereby determining whether a target object is placed against the wall.

[0200] It is understandable that the midpoints of the four sides of the minimum bounding rectangle of the object's point cloud can be determined separately, the distances between the four midpoints and the target wall points can be determined separately, and the minimum value among the four distances between the four midpoints and the target wall points can be determined as the first distance value.

[0201] This application embodiment determines a first distance value between the object point cloud and the target wall, uses the first distance value to determine whether the target object is against the wall, and determines the orientation of the target object based on the determination result, that is, the coordinate direction of the target object in the robot's visual coordinate system, which can improve the robot's pathfinding performance.

[0202] Based on any of the above embodiments, the determining module is further configured to: generate N wall points according to the wall position in the robot vision coordinate system, wherein the N wall points are distributed along the extension direction of the wall at preset intervals; and determine a first distance value based on the minimum distance between each point in the object point cloud and the N wall points.

[0203] In this embodiment of the application, the wall points are obtained by fitting the point cloud of the wall, and a first distance value is determined based on the distance between the target wall points and the points in the object point cloud.

[0204] Specifically, environmental data about the robot's environment is acquired through the robot's vision sensors. This environmental data includes wall locations and point clouds of target objects, such as sofas. The wall locations can be determined based on the wall point cloud data collected by the robot, or by inputting a room layout diagram into the robot's system.

[0205] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. After obtaining the line segment corresponding to each wall, a wall point is generated at preset intervals, starting from one endpoint of the wall line segment. Thus, N wall points can be obtained on the line segment corresponding to one wall.

[0206] These wall points form a simplified point cloud of the wall. When determining the distance between the object point cloud and the target wall, the distance between each point in the object point cloud and each of the N wall points can be determined separately. The minimum value among these distances is determined as the first distance between the object point cloud and the target wall, thereby determining whether a target object is placed against the wall.

[0207] It is understandable that the midpoints of the four sides of the minimum bounding rectangle of the object's point cloud can be determined separately, the distances between the four midpoints and the N wall points can be determined separately, and the minimum value among all the distances between the four midpoints and the N wall points can be determined as the first distance value.

[0208] This application embodiment determines a first distance value between the object point cloud and the target wall, uses the first distance value to determine whether the target object is against the wall, and determines the orientation of the target object based on the determination result, that is, the coordinate direction of the target object in the robot's visual coordinate system, which can improve the robot's pathfinding performance.

[0209] Based on any of the above embodiments, the acquisition module is also used to acquire the number of walls;

[0210] The module is specifically used for:

[0211] If the number of walls is less than the number threshold, the target wall points on the target wall are determined. The target wall points include the projection points of the points in the object point cloud onto the target wall and / or the endpoints of the target wall. The first distance value is determined based on the minimum distance between each point in the object point cloud and the target wall points.

[0212] When the number of walls is greater than or equal to the threshold, N wall points are generated according to the wall positions in the robot vision coordinate system. The N wall points are distributed along the extension direction of the wall at preset intervals, where N is a positive integer. The first distance value is determined based on the minimum distance between each point in the object point cloud and the N wall points.

[0213] In this embodiment, the room point cloud may contain multiple walls. When there are many walls, the target object is more likely to be close to multiple walls simultaneously; conversely, when there are few walls, the target object may only be close to one wall. Therefore, selecting a corresponding method to determine the first distance value between the object point cloud and the target wall based on a comparison between the number of walls and a threshold value can improve the algorithm's speed and effectively ensure the accuracy of the distance value, ultimately improving the reliability of the object's coordinate orientation.

[0214] It is understandable that the quantity threshold can be a preset fixed value, or it can be dynamically adjusted based on factors such as room area and room layout. For example, the quantity threshold is 6.

[0215] Specifically, when the number of walls is small, the target wall points can be determined by projecting the target object onto the target wall, and a first distance value can be determined based on the distance between the target wall points and the points in the object's point cloud.

[0216] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. When determining the distance between the object's point cloud and the target wall, each point in the object's point cloud can be traversed, and the distance between each point and the target wall can be determined one by one.

[0217] After obtaining the distance values ​​between all points of the object's point cloud and the target wall, the minimum value among these distance values ​​is determined as the first distance value between the target point cloud and the target wall. This first distance value can be used to determine the minimum distance between the target object and the target wall, thereby determining whether a target object is placed against the wall.

[0218] When there are many walls, the wall points can be obtained by fitting the point cloud of the walls. Based on the distance between the target wall points and the points in the object point cloud, the first distance value is determined.

[0219] After the robot generates the floor plan data of the room, each wall can be labeled with a line segment. After obtaining the line segment corresponding to each wall, a wall point is generated at preset intervals, starting from one endpoint of the wall line segment. Thus, N wall points can be obtained on the line segment corresponding to one wall.

[0220] These wall points form a simplified point cloud of the wall. When determining the distance between the object point cloud and the target wall, the distance between each point in the object point cloud and each of the N wall points can be determined separately. The minimum value among these distances is determined as the first distance between the object point cloud and the target wall, thereby determining whether a target object is placed against the wall.

[0221] Based on any of the above embodiments, the determining module is further configured to: determine the head coordinates and tail coordinates of the target object based on the second distance between each side of the target enclosing rectangle and the target wall when the first distance value is less than the distance threshold; determine the head coordinates and tail coordinates of the target object based on the ratio and the coordinate values ​​of each point in the object point cloud when the first distance value is greater than or equal to the distance threshold; and determine the coordinate direction of the target object based on the head coordinates and tail coordinates.

[0222] In this embodiment of the application, if the first distance value, that is, the minimum distance between the object point cloud and the target wall, is less than the distance threshold, it indicates that the target object is close to the target wall and the target object is placed against the target wall.

[0223] At this point, the four sides of the minimum bounding rectangle of the object point cloud are determined, which is the second distance between the four sides of the target rectangle and the target wall. Based on the second distance, the "head point" representing the head of the target object and the "tail point" representing the tail of the target object are determined on the object point cloud. The coordinates of the head point (head coordinates) and the coordinates of the tail point (tail coordinates) are then used to determine the coordinates of the tail point.

[0224] If the first distance value is greater than or equal to the distance threshold, it indicates that the distance between the target object and the target wall is relatively far, and the target object is not placed against the wall. In this case, the coordinate direction of the target object can be determined based on the minimum bounding rectangle of the object's point cloud, which is the ratio of the first side to the second side of the target's bounding rectangle, i.e., the aspect ratio of the target object.

[0225] Specifically, based on prior knowledge, for some objects with significant characteristics in terms of aspect ratio, after defining the direction from the head to the tail of the object as the object's orientation, the object's orientation is often directly related to its aspect ratio. For example, the head and tail of a sofa are generally on the long side of the sofa, while the head and foot of a single bed are generally on the wide side of the single bed.

[0226] Furthermore, these objects also have distinct shapes. For example, the rear of a sofa is defined as the seat cushion, and the head of the sofa as the backrest. The backrest is higher than the seat cushion. Similarly, the head of a single bed is defined as the headboard, and the rear of the single bed is defined as the footboard. The headboard is generally higher than the footboard. This height information can be expressed using the point coordinates in the object's point cloud, specifically the z-axis coordinates.

[0227] Therefore, for objects that are not against a wall, the head and tail positions of the target object can be determined by the aspect ratio of the target object and the coordinate values ​​of the points in the object's point cloud, thereby determining the head and tail coordinates.

[0228] After obtaining the head and tail coordinates, let the head coordinates be (x1, y1) and the tail coordinates be (x2, y2). Then, the coordinate direction of the target object can be calculated using the following formula:

[0229]

[0230] Where θ is the angle of the target object's coordinate direction, (x1, y1) is the head coordinate, and (x2, y2) is the tail coordinate.

[0231] This application embodiment determines the head and tail coordinates of the target object, determines the object's coordinate direction based on the head and tail coordinates, and then determines the object's orientation, thereby improving the pathfinding performance based on robot vision.

[0232] Based on any of the above embodiments, the determining module is further configured to: determine the coordinates of the midpoint of the third side corresponding to the minimum value of the second distance as the head coordinates; and determine the coordinates of the midpoint of the opposite side of the third side in the target enclosing rectangle as the tail coordinates.

[0233] In this embodiment of the application, the minimum bounding rectangle of the object point cloud, that is, the target bounding rectangle, includes 4 sides. If the first distance value is less than the distance threshold, it means that the distance between the target object and the target wall is relatively close, and the target object is placed against the target wall. At this time, the side of the target object against the wall is defined as the head. Then, the midpoint of the side closest to the target wall among the 4 sides of the target bounding rectangle can be defined as the head point. Therefore, the coordinates of the midpoint of the third side closest to the second distance from the target wall are determined as the head coordinates and used to mark the head of the target object.

[0234] Meanwhile, if the side of the target object furthest from the wall is defined as the tail, then the midpoint of the second side furthest from the target wall among the four sides of the target's enclosing rectangle, opposite to the third side, is defined as the aforementioned tail point, and the coordinates of the midpoint of the opposite side of the third side are determined as the aforementioned tail coordinates.

[0235] After obtaining the head and tail coordinates, the coordinate direction of the target object is calculated based on the coordinate values ​​of the head and tail coordinates.

[0236] This application embodiment determines the object's orientation, i.e., the object's coordinate direction in the robot's visual coordinate system, by using the distances between the four sides of the minimum bounding rectangle of the object's point cloud and the wall. This method can easily and quickly determine the object's orientation without needing to obtain the object's engineering drawing model, thus improving the robot's pathfinding performance.

[0237] Based on any of the above embodiments, the determining module is specifically used for: dividing the target enclosing rectangle into multiple sub-rectangles using the intersection of the first and second sides, the midpoint of the first side, the midpoint of the second side, and the center point of the target enclosing rectangle as vertices when the ratio is less than a ratio threshold; determining the average value of the z-axis coordinates of each point within each pair of adjacent sub-rectangles; determining the adjacent first and second sub-rectangles corresponding to the maximum value of the average value; determining the head coordinates based on the coordinates of the first intersection point of the common side of the first and second sub-rectangles and the side of the target enclosing rectangle; and determining the tail coordinates based on the coordinates of the midpoint of the side opposite the side where the first intersection point is located in the side of the target enclosing rectangle.

[0238] In the embodiments of this application, based on prior knowledge, the orientation of some common indoor objects is strongly related to their shape. For example, in a sofa, the head of the sofa, which is the backrest, is significantly higher than the back of the sofa, i.e., the seat. Similarly, in a bed, the headboard is significantly higher than the bed frame.

[0239] Therefore, in practical application scenarios, such as giving a TV remote control to someone sitting on a sofa, the orientation of the object can be determined based on the height of each part of the sofa, that is, the z-axis coordinate of each point.

[0240] Specifically, the ratio threshold, which is the threshold used to determine the shape of the target object, is a constant less than 1 and greater than 0 in some implementations. If the ratio of the length of the first side of the target object to the length of the second side of the target object, that is, the aspect ratio of the target object, is less than the ratio threshold, it indicates that the shape of the target object is relatively square, and the long side and the wide side of the minimum enclosing rectangle of the target object are relatively close. At this time, based on prior knowledge, all four sides of the minimum enclosing rectangle of the target object may be the "head" or "tail" of the target object.

[0241] The minimum bounding rectangle of the target object, also known as the target bounding rectangle, is divided into multiple sub-rectangles of equal area. The target bounding rectangle, or the minimum bounding rectangle of the target object, includes an intersecting first side and a second side. The intersection of the first and second sides is a vertex of the minimum bounding rectangle.

[0242] Taking the first side as the long side of the target enclosing rectangle and the second side as the wide side of the target enclosing rectangle as an example, the first line segment is determined by taking the midpoint of the two long sides of the target enclosing rectangle as the endpoint, and the second line segment is determined by taking the midpoint of the two wide sides of the target enclosing rectangle as the endpoint. The intersection of the first line segment and the second line segment is the midpoint of the target enclosing rectangle.

[0243] Figure 3 A schematic diagram of a sub-rectangle according to an embodiment of this application is shown, such as... Figure 3 As shown, a sub-rectangle is defined using one vertex of the target-enclosing rectangle, the midpoint of the adjacent long and short sides, and the midpoint of the target-enclosing rectangle as vertices. This results in four sub-rectangles of equal area, effectively dividing the target-enclosing rectangle into four equal sub-rectangles. Each of these four sub-rectangles has an adjacent sub-rectangle on the short side and a adjacent sub-rectangle on the long side of the target-enclosing rectangle.

[0244] After obtaining the four sub-rectangles, the z-axis coordinates of each point within the area of ​​each sub-rectangle are determined, and the average value of these z-axis coordinates is calculated. This average value can intuitively reflect the height of the target object within the area of ​​a sub-rectangle.

[0245] The height of the target object on a certain side can be determined by averaging the z-axis coordinates within two adjacent sub-rectangles. Therefore, after obtaining the average z-axis coordinates of each point in every two adjacent sub-matrices, the object orientation, i.e., the coordinate direction of the target object in the robot's vision coordinate system, can be determined by the aspect ratio of the target object (i.e., the ratio of the length of the first side to the length of the second side) and the height of the target object on each side (i.e., the first and second sides).

[0246] Specifically, the target enclosing rectangle is divided into 4 sub-rectangles. These 4 sub-rectangles consist of 2 adjacent sub-rectangles on the long side of the target enclosing rectangle and 2 adjacent sub-rectangles on the wide side of the target enclosing rectangle, for a total of 4 sets of adjacent sub-rectangles.

[0247] Among these four groups of adjacent sub-rectangles, the group with the highest average z-axis coordinate value is determined. This group of adjacent sub-rectangles specifically includes the first sub-rectangle and the second sub-rectangle. The first intersection point of the common side of the first and second sub-rectangles with its corresponding long or wide side is defined as the "head point" of the target object, and its coordinates are recorded as the head coordinates of the target object. The midpoint of the side opposite the first intersection point among the four sides of the target's enclosing rectangle is defined as the "tail point" of the target object, and its coordinates are recorded as the tail coordinates of the target object.

[0248] This application determines the orientation, or coordinate direction, of the target object by dividing the minimum bounding rectangle of the target object into multiple adjacent sub-rectangles and combining the aspect ratio of the minimum bounding rectangle. This process does not rely on the engineering drawing model of the object and can determine the object's orientation more quickly.

[0249] Based on any of the above embodiments, the determining module is further configured to: determine the fourth side of the target enclosing rectangle according to the object type of the target object when the ratio is greater than or equal to the ratio threshold; determine the adjacent third and fourth sub-rectangles corresponding to the maximum value of the average value, wherein the third and fourth sub-rectangles are adjacent on the fourth side; determine the head coordinates according to the coordinates of the second intersection point of the common side of the third and fourth sub-rectangles and the fourth side; and determine the tail coordinates according to the coordinates of the midpoint of the side opposite the side where the second intersection point is located in the target enclosing rectangle.

[0250] In this application embodiment, a preset threshold, which is a threshold used to determine the shape of the target object, is, in some embodiments, a constant less than 1 and greater than 0. If the ratio of the length of the first side of the target object to the length of the second side of the target object, that is, the aspect ratio of the target object, is greater than or equal to the preset threshold, it indicates that the difference between the length of the long side and the length of the wide side of the minimum enclosing rectangle of the target object is large.

[0251] Based on prior knowledge, for scenarios where the target object is a sofa, the two longest sides of the minimum bounding rectangle of the target object are more likely to be the "head" or "tail" of the target object. For scenarios where the target object is a single bed, the two shortest sides of the minimum bounding rectangle of the target object are more likely to be the "head" or "tail" of the target object.

[0252] At this point, based on the object type of the target object, it is determined that the head and tail of this type of object are generally on the long side or the wide side. Taking the target object as a sofa as an example, the head of the sofa generally refers to the back of the sofa, and the back of the sofa is generally on the long side of the minimum bounding rectangle of the object point cloud of the sofa. Therefore, the long side of the target bounding rectangle can be recorded as the fourth side, and in the four sub-rectangles divided by the minimum bounding rectangle, two sets of two adjacent sub-rectangles on the long side, that is, the fourth side, are determined.

[0253] It is understandable that if the target object is a single bed, and the headboard and bed frame of the single bed are on the wide side of the minimum bounding rectangle of the object point cloud of the single bed, then the wide side of the bounding rectangle of the target is denoted as the fourth side.

[0254] Among the two adjacent sub-rectangles on the fourth side, determine the pair of adjacent sub-rectangles with the highest average z-axis coordinate values. This pair of adjacent sub-rectangles specifically includes the third and fourth sub-rectangles. The third and fourth sub-rectangles are adjacent on the fourth side. The second intersection point of the common edge of the third and fourth sub-rectangles with the fourth side is determined as the "head point" of the target object, and the coordinates of this second intersection point are taken as the head coordinates of the target object. The midpoint of the side opposite the edge containing the second intersection point among the four sides of the target's enclosing rectangle is recorded as the "tail point" of the target object, and its coordinates are taken as the tail coordinates of the target object.

[0255] This application embodiment determines the head and tail coordinates of the target object, determines the object's coordinate direction based on the head and tail coordinates, and then determines the object's orientation, thereby improving the pathfinding performance based on robot vision.

[0256] In some embodiments of the present invention, a device for determining the coordinate orientation of an object is provided. Figure 5 A structural block diagram of an object coordinate orientation determination device according to an embodiment of this application is shown, such as... Figure 5 As shown, the object coordinate orientation determination device 500 includes: a memory 502 for storing programs or instructions; and a processor 504 for executing the program or instructions to implement the steps of the object coordinate orientation determination method provided in any of the above embodiments. Therefore, the object coordinate orientation determination device also includes all the beneficial effects of the object coordinate orientation determination method provided in any of the above embodiments. To avoid repetition, these effects will not be described again here.

[0257] In some embodiments of the present invention, a readable storage medium is provided on which a program or instructions are stored. When the program or instructions are executed by a processor, they implement the steps of the method for determining the coordinate orientation of an object as provided in any of the above embodiments. Therefore, the readable storage medium also includes all the beneficial effects of the method for determining the coordinate orientation of an object as provided in any of the above embodiments. To avoid repetition, these effects will not be described again here.

[0258] In some embodiments of the present invention, a computer program product is provided. The computer program product is stored in a storage medium. When the computer program product is executed by at least one processor, it implements the steps of the method for determining the coordinate orientation of an object as provided in any of the above embodiments. Therefore, the computer program product also includes all the beneficial effects of the method for determining the coordinate orientation of an object as provided in any of the above embodiments. To avoid repetition, these effects will not be described again here.

[0259] In some embodiments of the present invention, an electronic device is provided, including a means for determining the coordinate direction of an object as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments; and / or a computer program product as provided in any of the above embodiments, thus also including the means for determining the coordinate direction of an object as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments; and / or all the beneficial effects of the computer program product as provided in any of the above embodiments will not be repeated here to avoid repetition.

[0260] In some embodiments of the present invention, a robot is provided, including a device for determining the coordinate direction of an object as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments; and / or a computer program product as provided in any of the above embodiments, thus also including the device for determining the coordinate direction of an object as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments; and / or all the beneficial effects of the computer program product as provided in any of the above embodiments will not be repeated here to avoid repetition.

[0261] Based on the above embodiments, the robot further includes: a navigation module, which, upon receiving a walking instruction to walk to a target object, determines the target navigation coordinates according to the coordinate direction of the target object and controls the robot to walk to the target navigation coordinates.

[0262] In this embodiment, the robot includes a navigation module for semantic navigation. For example, a first user places their phone on the robot and commands it to hand the phone to a second user sitting on a sofa. The robot, based on the recognized coordinates of the sofa, determines the rear of the sofa (i.e., one side of the seat cushion) as the target navigation coordinate and moves to that coordinate, thus accurately handing the phone to the second user. This avoids the awkward situation where the robot travels to the side or back of the sofa, requiring the second user to get up to retrieve the phone, and improves the robot's pathfinding performance. In the description of this invention, the term "multiple" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. The terms "connection," "installation," "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0263] In the description of this invention, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this invention, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0264] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining the coordinate orientation of an object, characterized in that, include: Obtain the wall location and the object point cloud of the target object; Determine the target bounding rectangle of the object point cloud, the target bounding rectangle including the intersecting first and second sides; Based on the location of the wall, determine the first distance value between the object point cloud and the target wall; The coordinate direction of the target object is determined based on the ratio of the length of the first side to the length of the second side and the first distance value. Determining the coordinate direction of the target object based on the ratio of the length of the first side to the length of the second side and the first distance value includes: If the first distance value is less than the distance threshold, the head coordinates and tail coordinates of the target object are determined based on the second distance between each side of the target enclosing rectangle and the target wall. If the first distance value is greater than or equal to the distance threshold, the head coordinates and tail coordinates of the target object are determined according to the ratio and the coordinates of each point in the object point cloud; The coordinate orientation of the target object is determined based on the head coordinates and the tail coordinates; Determining the head and tail coordinates of the target object based on the second distance between each side of the target enclosing rectangle and the target wall includes: The coordinates of the midpoint of the third side corresponding to the minimum value of the second distance are determined as the head coordinates; The coordinates of the midpoint of the opposite side of the third side within the target bounding rectangle are determined as the tail coordinates; Determining the head and tail coordinates of the target object based on the ratio and the coordinates of each point in the object point cloud includes: If the ratio is less than the ratio threshold, the target enclosing rectangle is divided into multiple sub-rectangles using the intersection of the first side and the second side, the midpoint of the first side, the midpoint of the second side, and the center point of the target enclosing rectangle as vertices. Determine the average value of the z-axis coordinates of each point within every two adjacent sub-rectangles; Determine the first and second sub-rectangles adjacent to the maximum value of the average value; The head coordinates are determined based on the coordinates of the first intersection point of the common side of the first sub-rectangle and the second sub-rectangle and the side of the target enclosing rectangle; The tail coordinates are determined based on the coordinates of the midpoint of the side opposite the side containing the first intersection point of the target enclosing rectangle. The method for determining the coordinate orientation of the object also includes: If the ratio is greater than or equal to the ratio threshold, the fourth side of the target enclosing rectangle is determined according to the object type of the target object; Determine the adjacent third and fourth sub-rectangles corresponding to the maximum value of the average value, wherein the third and fourth sub-rectangles are adjacent on the fourth side; The head coordinates are determined based on the coordinates of the second intersection point of the common side of the third sub-rectangle and the fourth sub-rectangle and the fourth side; The tail coordinates are determined based on the coordinates of the midpoint of the side opposite the side containing the second intersection point in the rectangle surrounding the target.

2. The method according to claim 1, characterized in that, Determining the target bounding rectangle of the object point cloud includes: Obtain the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate for each point within the point cloud of the object; Based on the maximum value of the x-axis coordinate and the maximum value of the y-axis coordinate, determine the coordinates of at least two vertices of the target enclosing rectangle; The target enclosing rectangle is determined based on the coordinates of at least two vertices.

3. The method according to claim 1, characterized in that, Before determining the first distance value between the object point cloud and the target wall based on the wall position, the method further includes: Based on the location of the wall, the target wall is determined, which is the wall closest to the target object.

4. The method according to claim 3, characterized in that, The step of determining the first distance value between the object point cloud and the target wall based on the wall position specifically includes: Determine the target wall points on the target wall, wherein the target wall points include the projection points of the points in the object point cloud onto the target wall and / or the endpoints of the target wall; The first distance value is determined based on the minimum distance between each point in the object point cloud and the point on the target wall.

5. The method according to claim 3, characterized in that, The step of determining the first distance value between the object point cloud and the target wall based on the wall position specifically includes: In the robot vision coordinate system, N wall points are generated based on the wall position. The N wall points are distributed along the extension direction of the wall at preset intervals, where N is a positive integer. The first distance value is determined based on the minimum distance between each point in the object point cloud and the N wall points.

6. The method according to claim 3, characterized in that, Before determining the first distance value between the object point cloud and the target wall based on the wall position, the method further includes: Obtain the quantity of the walls; If the number of walls is less than a threshold, the target wall points on the target wall are determined. The target wall points include the projection points of the points in the object point cloud onto the target wall and / or the endpoints of the target wall. The first distance value is determined based on the minimum distance between each point in the object point cloud and the point on the target wall. When the number of walls is greater than or equal to a threshold, N wall points are generated in the robot vision coordinate system based on the wall positions. The N wall points are distributed along the extension direction of the wall at preset intervals, where N is a positive integer. The first distance value is determined based on the minimum distance between each point in the object point cloud and the N wall points.

7. A device for determining the coordinate orientation of an object, characterized in that, include: Memory, used to store programs or instructions; A processor, configured to implement the steps of the method for determining the coordinate orientation of an object as described in any one of claims 1 to 6 when executing the program or instructions.

8. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the method for determining the coordinate orientation of an object as described in any one of claims 1 to 6.

9. A computer program product, said computer program product being stored in a storage medium, characterized in that, When the computer program product is executed by at least one processor, it implements the steps of the method for determining the coordinate orientation of an object as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, include: The device for determining the coordinate direction of an object as described in claim 7; and / or The readable storage medium as described in claim 8; and / or The computer program product as described in claim 9.

11. A robot, characterized in that, include: The device for determining the coordinate direction of an object as described in claim 7; and / or The readable storage medium as described in claim 8; and / or The computer program product as described in claim 9.

12. The robot according to claim 11, characterized in that, Also includes: The navigation module is used to determine the target navigation coordinates based on the coordinate direction of the target object when it receives a walking instruction to walk to the target object, and control the robot to walk to the target navigation coordinates.

Images