Object information detection method, device and storage medium

Abstract

The present application relates to an object information detection method, device, and storage medium. The method includes: obtaining an initial ground plane equation and initial pose information of a collection unit; the initial ground plane equation is determined based on surrounding point cloud information at an initial moment; obtaining current surrounding point cloud information based on the collection unit; the current surrounding point cloud information includes the three-dimensional position of each point; obtaining current pose information of the collection unit; when the similarity between the current pose information and the initial pose information is less than or equal to a preset value, determining a pose deviation based on the current pose information and the initial pose information; determining the current ground plane equation based on the pose deviation and the initial ground plane equation; and determining object information in the current surrounding point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes a category. In this way, the real-time, accuracy, and stability of object detection can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of object detection, and in particular to a method, device and storage medium for detecting object information. Background Art

[0002] During autonomous walking, mobile robots need to detect the path and obstacle location information in the forward direction in real time, and complete autonomous obstacle avoidance actions through path planning algorithms to ensure safe arrival at the destination.

[0003] Depth cameras can capture three-dimensional information about the surrounding environment. Common depth cameras include structured light cameras (such as Kinect V1), time-of-flight cameras (such as Helios), and binocular cameras (such as LeadSense). Depth cameras can obtain two-dimensional image information and its corresponding depth and distance information. With the development of imaging technology in recent years, these cameras have been widely used.

[0004] In the existing technology, a depth camera is used to detect the ground plane and obstacles to achieve obstacle avoidance. However, since the robot is easily affected by factors such as vibration and external forces during movement, the positional relationship between the depth camera and the ground changes, thereby destroying the accuracy and stability of the obstacle detection function. Summary of the Invention

[0005] The embodiments of the present application provide an object information detection method, device, and storage medium, which can improve the real-time, accuracy, and stability of object detection.

[0006] In one aspect, an embodiment of the present application provides an object information detection method, comprising:

[0007] Obtain the initial ground plane equation and the initial pose information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment;

[0008] Acquire the current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point;

[0009] Get the current posture information of the acquisition unit;

[0010] When the similarity value between the current posture information and the initial posture information is less than or equal to a preset value, determining a posture deviation based on the current posture information and the initial posture information;

[0011] Determine the current ground plane equation based on the posture deviation and the initial ground plane equation;

[0012] Based on the current ground plane equation and the three-dimensional position of each point, determine the information of the object in the current surrounding environment point cloud information; the object information includes the category.

[0013] Optionally, the acquisition unit includes a depth camera; obtaining point cloud information of the current surrounding environment based on the acquisition unit includes:

[0014] The depth camera collects a depth image of the current surrounding environment; the depth image includes the position information and depth information of each pixel;

[0015] Based on the position information and depth information of each pixel, the three-dimensional position of each pixel in the camera coordinate system is determined to obtain the current surrounding environment point cloud information.

[0016] Optionally, the pose deviation is in quaternion form;

[0017] Determine the current ground plane equation based on the pose deviation and the initial ground plane equation, including:

[0018] Convert the quaternion form of the pose deviation into the corresponding rotation matrix;

[0019] Establish the current ground plane equation to be solved;

[0020] Determine the normal vector of the current ground plane equation to be solved according to the normal vector of the initial ground plane equation and the rotation matrix;

[0021] Get the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system;

[0022] The solved current ground plane equation is obtained according to the normal vector and the intersection coordinates of the current ground plane equation to be solved.

[0023] Optionally, obtain the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system, including:

[0024] Determine the first intercept of the initial ground plane equation on the Y axis of the initial camera coordinate system;

[0025] Determine the second intercept of the current ground plane equation to be solved on the Y axis of the current camera coordinate system;

[0026] Obtain the distance difference between the current moment and the initial moment detected by the laser ranging unit;

[0027] Based on the normal vector, the first intercept, the second intercept, and the distance difference of the current ground plane equation to be solved, the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system are determined.

[0028] Optionally, based on the current ground plane equation and the three-dimensional position of each point, information about objects in the current surrounding point cloud information is determined, including:

[0029] Determine the value to be determined corresponding to each point based on the three-dimensional position of each point and the current ground plane equation;

[0030] The category of each point is determined according to the value to be determined and the obtained first preset distance.

[0031] Optionally, determining the category of each point based on the value to be determined and the obtained first preset distance includes:

[0032] If the value to be determined is greater than or equal to the first preset distance, the category of the corresponding point is determined to be a raised obstacle;

[0033] Or; if the value to be determined is less than the first preset distance and greater than the opposite number of the first preset distance, then the category of the corresponding point is determined to be the ground plane;

[0034] Or; if the value to be determined is less than or equal to the opposite number of the first preset distance, the category of the corresponding point is determined to be a concave obstacle.

[0035] Optionally, after determining that the corresponding point is a ground plane, the method further includes:

[0036] Determine the current drivable path based on all points whose category is ground plane.

[0037] Optionally, the object's information also includes a location;

[0038] Based on the current ground plane equation and the 3D position of each point, determine the information of the objects in the current surrounding point cloud information, including:

[0039] Perform clustering based on the three-dimensional position of each point to obtain multiple clustered first point clouds;

[0040] determining a center position of each first point cloud in the plurality of first point clouds;

[0041] Determining a value to be determined corresponding to each first point cloud based on a center position of each first point cloud;

[0042] The category of each first point cloud is determined according to the value to be determined and the obtained second preset distance.

[0043] On the other hand, an embodiment of the present application provides an object information detection device, including:

[0044] A first acquisition module is used to obtain an initial ground plane equation and initial pose information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment;

[0045] The second acquisition module is used to acquire the current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point;

[0046] The third acquisition module is used to obtain the current posture information of the acquisition unit;

[0047] A first determining module is configured to determine a posture deviation based on the current posture information and the initial posture information when a similarity value between the current posture information and the initial posture information is less than or equal to a preset value;

[0048] A second determination module is used to determine the current ground plane equation according to the posture deviation and the initial ground plane equation;

[0049] The third determination module is used to determine the information of the object in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes the category.

[0050] On the other hand, an embodiment of the present application provides a computer storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned object information detection method.

[0051] The object information detection method, device, and storage medium provided by the embodiments of the present application have the following beneficial effects:

[0052] The system obtains an initial ground plane equation and initial pose information of the acquisition unit; the initial ground plane equation is determined based on the surrounding point cloud information at the initial moment; the current surrounding point cloud information is obtained based on the acquisition unit; the current surrounding point cloud information includes the three-dimensional position of each point; the current pose information of the acquisition unit is obtained; when the similarity between the current pose information and the initial pose information is less than or equal to a preset value, the pose deviation is determined based on the current pose information and the initial pose information; the current ground plane equation is determined based on the pose deviation and the initial ground plane equation; and the object information in the current surrounding point cloud information is determined based on the current ground plane equation and the three-dimensional position of each point; the object information includes the category. This improves the real-time, accuracy, and stability of object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0055] Figure 2 is a schematic diagram of a camera coordinate system provided in an embodiment of the present application;

[0056] Figure 3This is a flow chart of an object information detection method provided in an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of another application scenario provided by an embodiment of the present application;

[0058] Figure 5 It is a structural diagram of an object information detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0061] See also Figure 1 , Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application, including a movable robot 100. The robot 100 is provided with a collection unit 101 and a processing unit 102. The collection unit 101 transmits the collected current surrounding environment information to the processing unit 102. The processing unit 102 processes and detects object information from the current surrounding environment information. The object information is used to assist the robot 100 in completing obstacle avoidance actions.

[0062] In the embodiment of the present application, the acquisition unit 101 includes a depth camera 1011 and an inertial measurement unit 1012 (Inertial Measurement Unit, IMU). The IMU unit 1012 is generally composed of a 3-axis acceleration sensor and a 3-axis angular velocity (gyroscope) sensor. The depth camera 1011 and the IMU unit 1012 are fixed on the same structure, the relative position does not change, and they work synchronously through the same external hardware clock. The depth camera 1011 is fixed on the body of the robot 100, the relative position does not change, and the direction is facing the direction of travel of the robot 100. The ground plane must be observable within the field of view of the camera 1011. The processing unit 102 receives data from the depth camera 1011 and the IMU unit 1012, interprets the data, and completes the object information detection function.

[0063] Specifically, the processing unit 102 obtains the initial ground plane equation and the initial posture information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment; the current surrounding environment point cloud information is obtained based on the depth camera 1011; the current surrounding environment point cloud information includes the three-dimensional position of each point; the processing unit 102 obtains the current posture information of the IMU unit 1012; when the similarity value between the current posture information and the initial posture information is less than or equal to a preset value, the processing unit 102 determines the posture deviation based on the current posture information and the initial posture information, and determines the current ground plane equation based on the posture deviation and the initial ground plane equation; then the processing unit 102 determines the information of the object in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes the category;

[0064] It should be noted that the coordinate system involved in the embodiments of the present application all adopts the coordinate system of the depth camera 1011, such as Figure 2 As shown, the optical center of the depth camera 1011 is taken as the origin, and according to the right-hand coordinate system, the X axis is toward the right of the camera, the Y axis is toward the bottom, and the Z axis is toward the front; the coordinates of the original point cloud data obtained from the depth camera 1011 are all in the camera coordinate system. Since the depth camera 1011, the IMU unit 1012 and the robot body 100 are fixed in position, the relevant data in the robot 100 coordinate system can be obtained through coordinate system conversion according to actual needs; it is assumed in the following that all data information has been converted to the camera coordinate system, and only the object information analysis and detection method in the camera coordinate system are considered.

[0065] In an embodiment of the present application, the object can be each point in the current surrounding environment point cloud information, and the category of the object represents the category of the corresponding point; the object can also be a point cloud composed of multiple points, and the point cloud corresponds to an entity, and the category of the object represents the category of the corresponding entity.

[0066] The following describes a specific embodiment of an object information detection method of the present application. Figure 3 It is a flowchart of an object information detection method provided by an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 3 As shown, the method may include:

[0067] S301: Acquire an initial ground plane equation and initial pose information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment.

[0068] In the embodiment of the present application, when the robot is at an initial moment, such as the moment of starting the robot, the surrounding environment point cloud information can be obtained by the depth camera detection of the acquisition unit, and the initial ground plane equation can be obtained based on the surrounding environment point cloud information; the method for calculating the initial ground plane equation can adopt any existing technology, and the present application does not limit the existing method. The final determined initial ground plane equation can be expressed as the following equation (1):

[0069] A0*X+B0*Y+C0*Z+D0=0 (1)

[0070] Among them, A0, B0, C0 satisfy A0 2 +B0 2 +C0 2 =1, the normal vector of the initial ground plane equation (1) is n0=(A0, B0, C0); at the same time, the initial posture information of the IMU unit at this time is recorded.

[0071] S303: Acquire current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point.

[0072] S305: Obtain the current posture information of the acquisition unit.

[0073] S307: When the similarity value between the current posture information and the initial posture information is less than or equal to a preset value, determine a posture deviation based on the current posture information and the initial posture information.

[0074] S309: Determine the current ground plane equation based on the posture deviation and the initial ground plane equation.

[0075] In an embodiment of the present application, during the movement, the robot can obtain the current surrounding environment point cloud information through the depth camera in real time or at a preset frequency. The current surrounding environment point cloud information includes the three-dimensional position of each point, and the current posture information is determined by the IMU unit; when the similarity value between the current posture information and the initial posture information is less than or equal to a preset value, the preset value is the threshold of the posture change, that is, when the similarity value between the current posture information and the initial posture information is less than or equal to the threshold, it indicates that the posture of the IMU unit has changed. Since the depth camera and the IMU unit are fixed to the same structure and the relative position remains unchanged, the posture of the depth camera has also changed. Therefore, at the current moment and subsequent moments, the object information in the surrounding environment point cloud information is calculated based on the initial ground plane equation, and the result is inaccurate; therefore, when the similarity value between the current posture information and the initial posture information is less than or equal to the preset value, the present application determines the posture deviation between the current moment t1 and the initial moment t0 based on the current posture information and the initial posture information, updates the initial ground plane equation according to the posture deviation, and determines the current ground plane equation. For situations where the robot's posture is tilted due to vibration or acceleration, compared with the existing technology, the above steps can quickly update the ground plane equation according to the posture deviation, with low computational intensity, which can improve real-time performance, thereby ensuring the accuracy and stability of the object information detection function.

[0076] In an optional embodiment, the acquisition unit includes a depth camera; step S303 may specifically include: acquiring a depth image of the current surrounding environment via the depth camera; the depth image includes position information and depth information for each pixel; and determining the three-dimensional position of each pixel in the camera coordinate system based on the position information and depth information of each pixel, thereby obtaining point cloud information of the current surrounding environment. This process involves converting from the pixel coordinate system to the camera coordinate system, determining the point in space corresponding to each pixel based on the conversion relationship, and calculating the three-dimensional position of the point, thereby obtaining point cloud information of the current surrounding environment.

[0077] In an optional embodiment, the posture deviation is in the form of quaternion; both rotation matrix and quaternion are ways to describe posture in three-dimensional space; the initial posture quaternion information at the initial time t0 is Q0, and the current posture quaternion information detected at the current time t1 is Q1, then the posture deviation can be Correspondingly, the above step S309 may specifically include:

[0078] First, the quaternion form of the pose deviation Q is converted into the corresponding rotation matrix R; then, the current ground plane equation to be solved is established, as shown in the following equation (2):

[0079] A1*X+B1*Y+C1*Z+D1=0 (2)

[0080] Obviously, the current ground plane equation can be obtained by calculating the values ​​of A1, B1, C1, and D1 in the above formula. There is no need to re-detect the ground plane. According to the properties of the plane equation, the normal vector of the current ground plane equation is n1 = (A1, B1, C1).

[0081] Secondly, the normal vector of the current ground plane equation to be solved is determined based on the normal vector n0 of the initial ground plane equation and the rotation matrix R. The specific method is as follows:

[0082] n1=R T *n0=R T *(A0, B0, C0) (3)

[0083] Secondly, based on the principle that the plane equation can be solved by the normal vector of a plane and a point on the plane, the coordinates of the intersection of the current ground plane equation to be solved and the Y-axis of the current camera coordinate system can be obtained; then, based on the normal vector of the current ground plane equation to be solved and the coordinates of the intersection, the solved current ground plane equation is obtained.

[0084] In a specific implementation of obtaining the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system, considering that the height change in the Y axis direction is small during the robot's driving process, the height deviation can be ignored, that is, the intersection of the initial ground plane equation and the Y axis of the initial camera coordinate system is the same as the intersection of the current ground plane equation and the Y axis of the current camera coordinate system, that is, their intercepts on the Y axis are equal, so the following formula (4) can be obtained:

[0085]

[0086] In this way, the values ​​of A1, B1, C1, and D1 can be solved by the above formulas (3) and (4), thereby obtaining the solved current ground plane equation.

[0087] In another specific implementation of obtaining the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system, in order to obtain the ground plane spatial position more accurately, such as Figure 4 As shown, a laser ranging unit 1013 can be fixed on the depth camera 1011 to detect the distance change between the depth camera 1011 and the ground plane; the laser ranging unit 1013 is fixed relative to the depth camera 1011 and is installed downward along the Y-axis direction of the camera coordinate system, and its data can be connected to the processing unit 102 for analysis;

[0088] First, determine the first intercept of the initial ground plane equation on the Y axis of the initial camera coordinate system as d0 = -D0 / B0; secondly, determine the second intercept of the current ground plane equation to be solved on the Y axis of the current camera coordinate system as d1 = -D1 / B1; thirdly, obtain the distance difference between the current moment and the initial moment detected by the laser ranging unit 1013 as δT; obviously, the difference between the first intercept and the second intercept is the distance difference detected by the laser ranging unit 1013, thereby obtaining the following formula (5):

[0089]

[0090] In this way, the values ​​of A1, B1, C1, and D1 can also be solved by the above formulas (3) and (5), thereby obtaining the solved current ground plane equation.

[0091] S311: Based on the current ground plane equation and the three-dimensional position of each point, determine the information of the object in the current surrounding environment point cloud information; the object information includes the category.

[0092] In an embodiment of the present application, the category of each point is determined by judging the relative position of the three-dimensional position of each point and the current ground plane equation; alternatively, the category of the entity can be determined by judging the relative position of a point cloud composed of an indefinite number of points and the current ground plane equation, with one point cloud corresponding to one entity.

[0093] In an optional implementation, step S311 may specifically include:

[0094] According to the three-dimensional position of each point and the current ground plane equation, the corresponding value to be judged for each point is determined; specifically, for the current point P i The three-dimensional position coordinates (X i , Y i , Z i ), X i , Y i , Z i Substitute into the current ground plane equation (2) to get the current point P i The value to be judged is:

[0095] H=A1*X i +B1*Y i +C1*Z i +D1

[0096] Among them, H represents the value to be judged;

[0097] Determine the category of each point based on the value to be determined and the first preset distance obtained; this step may specifically include: if the value to be determined is greater than or equal to the first preset distance (H≥tol), where tol represents the first preset distance and tol>0, then determine the corresponding point Pi The category is a raised obstacle; or; if the value to be judged is less than the first preset distance and greater than the opposite number of the first preset distance (|H|<tol), then determine the corresponding point P i The category is the ground plane; or; if the value to be judged is less than or equal to the first preset distance inverse number (H<-tol), then determine the corresponding point P i The category is a sunken obstacle.

[0098] In an optional embodiment, after determining that the corresponding point is classified as a ground plane, the method may further include: determining a currently drivable path based on all ground plane points. Specifically, the processing unit may use a path planning algorithm to calculate one or more currently drivable paths based on all ground plane points to assist the robot in autonomous obstacle avoidance.

[0099] In another optional embodiment, the object information also includes a location. In this case, the object refers to a point cloud composed of multiple points, and the point cloud corresponds to an entity. Step S311 may specifically include:

[0100] Clustering is performed based on the three-dimensional position of each point to obtain multiple clustered first point clouds; the center position of each first point cloud in the multiple first point clouds is determined; based on the center position of each first point cloud, a value to be determined corresponding to each first point cloud is determined; and the category of each first point cloud is determined based on the value to be determined and the obtained second preset distance. The center position of each first point cloud can be determined as the position of the object in the final output. The benefit of performing clustering in this embodiment is that it can reduce the repeated operation of determining each point.

[0101] Specifically, the clustering method can refer to the existing technology, such as clustering based on distance; the center position of each first point cloud can be the average or weighted average of the positions of all point clouds in the first point cloud. This application does not limit the specific clustering method and the method of determining the center position; according to the value to be judged of each first point cloud and the obtained second preset distance, the method of determining the category of each first point cloud can refer to the above implementation method, and the position of a certain point can be replaced by the center position of a certain first point cloud. It will not be repeated here.

[0102] The present application also provides an object information detection device. Figure 5 is a structural diagram of an object information detection device provided in an embodiment of the present application, such as Figure 5 As shown, the device includes:

[0103] The first acquisition module 501 is used to obtain an initial ground plane equation and initial pose information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment;

[0104] The second acquisition module 502 is used to acquire the current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point;

[0105] The third acquisition module 503 is used to obtain the current posture information of the acquisition unit;

[0106] A first determining module 504 is configured to determine a posture deviation based on the current posture information and the initial posture information when the similarity value between the current posture information and the initial posture information is less than or equal to a preset value;

[0107] A second determining module 505 is configured to determine a current ground plane equation based on the posture deviation and the initial ground plane equation;

[0108] The third determining module 506 is configured to determine information about objects in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes a category.

[0109] In an optional embodiment, the acquisition unit includes a depth camera; the second acquisition module 502 is specifically used to: acquire a depth image of the current surrounding environment through the depth camera; the depth image includes position information and depth information of each pixel; based on the position information and depth information of each pixel, determine the three-dimensional position of each pixel in the camera coordinate system to obtain the current surrounding environment point cloud information.

[0110] In an optional embodiment, the posture deviation is in the form of a quaternion; the second determination module 505 is specifically used to: convert the posture deviation in the form of a quaternion into a corresponding rotation matrix; establish the current ground plane equation to be solved; determine the normal vector of the current ground plane equation to be solved based on the normal vector of the initial ground plane equation and the rotation matrix; obtain the coordinates of the intersection of the current ground plane equation to be solved and the Y-axis of the current camera coordinate system; obtain the solved current ground plane equation based on the normal vector and intersection coordinates of the current ground plane equation to be solved.

[0111] In an optional implementation, the second determination module 505 is further specifically used to: determine the first intercept of the initial ground plane equation on the Y-axis of the initial camera coordinate system; determine the second intercept of the current ground plane equation to be solved on the Y-axis of the current camera coordinate system; obtain the distance difference between the current moment and the initial moment detected by the laser ranging unit; and determine the coordinates of the intersection of the current ground plane equation to be solved and the Y-axis of the current camera coordinate system based on the normal vector, first intercept, second intercept and distance difference of the current ground plane equation to be solved.

[0112] In an optional implementation, the third determination module 506 is specifically configured to: determine the value to be determined corresponding to each point based on the three-dimensional position of each point and the current ground plane equation; and determine the category of each point based on the value to be determined and the obtained first preset distance.

[0113] In an optional embodiment, the third determination module 506 is further specifically used to: if the value to be determined is greater than or equal to the first preset distance, determine that the category of the corresponding point is a raised obstacle; or; if the value to be determined is less than the first preset distance and greater than the opposite of the first preset distance, determine that the category of the corresponding point is a ground plane; or; if the value to be determined is less than or equal to the opposite of the first preset distance, determine that the category of the corresponding point is a sunken obstacle.

[0114] In an optional implementation, the third determination module 506 is specifically configured to determine the current drivable path based on all points that are classified as ground plane.

[0115] In an optional embodiment, the object information further includes a position; the third determining module 506 is specifically configured to: perform clustering based on the three-dimensional position of each point to obtain a plurality of clustered first point clouds;

[0116] Determine the center position of each first point cloud in the plurality of first point clouds; determine a value to be determined corresponding to each first point cloud based on the center position of each first point cloud; and determine the category of each first point cloud according to the value to be determined and the obtained second preset distance.

[0117] The device and method embodiments in the embodiments of this application are based on the same application concept.

[0118] The device provided in the embodiments of the present application can be applied to other movable electronic devices such as robots that need to complete obstacle avoidance actions by detecting object information.

[0119] An embodiment of the present application also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to an object information detection method in an embodiment of the method. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the above-mentioned object information detection method.

[0120] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0121] As can be seen from the embodiments of the object information detection method, device and storage medium provided by the present application, the present application obtains the initial ground plane equation and the initial posture information of the acquisition unit; the initial ground plane equation is determined based on the surrounding environment point cloud information at the initial moment; the current surrounding environment point cloud information is obtained based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point; the current posture information of the acquisition unit is obtained; when the similarity value between the current posture information and the initial posture information is less than or equal to a preset value, the posture deviation is determined based on the current posture information and the initial posture information; the current ground plane equation is determined based on the posture deviation and the initial ground plane equation; the object information in the current surrounding environment point cloud information is determined based on the current ground plane equation and the three-dimensional position of each point; the object information includes the category. In this way, the real-time, accuracy and stability of object detection can be improved.

[0122] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0124] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0125] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for detecting object information, characterized in that: include: Obtain the initial ground plane equation and the initial pose information of the acquisition unit; The initial ground plane equation is determined based on the surrounding point cloud information at the initial moment; Acquire current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes the three-dimensional position of each point; Obtaining current posture information of the acquisition unit; When the similarity between the current posture information and the initial posture information is less than or equal to a preset value, determining a posture deviation based on the current posture information and the initial posture information; the posture deviation is in the form of a quaternion; Convert the quaternion form of the pose deviation into the corresponding rotation matrix; Establish the current ground plane equation to be solved; Determining the normal vector of the current ground plane equation to be solved according to the normal vector of the initial ground plane equation and the rotation matrix; Obtain the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system; Obtaining a solved current ground plane equation according to the normal vector of the current ground plane equation to be solved and the intersection coordinates; Determining information about an object in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes a category; The determining, based on the current ground plane equation and the three-dimensional position of each point, information about the object in the current surrounding environment point cloud information includes: Determining a value to be determined corresponding to each point according to the three-dimensional position of each point and the current ground plane equation; If the value to be determined is greater than or equal to the first preset distance, determining that the category of the corresponding point is a raised obstacle; Or, if the value to be determined is less than the first preset distance and greater than the opposite of the first preset distance, then the category of the corresponding point is determined to be a ground plane; Or; if the value to be determined is less than or equal to the opposite number of the first preset distance, the category of the corresponding point is determined to be a sunken obstacle.

2. The method according to claim 1, characterized in that The acquisition unit includes a depth camera; and the step of acquiring point cloud information of the current surrounding environment based on the acquisition unit includes: The depth camera collects a depth image of the current surrounding environment; the depth image includes position information and depth information of each pixel; Based on the position information and depth information of each pixel, the three-dimensional position of each pixel in the camera coordinate system is determined to obtain the current surrounding environment point cloud information.

3. The method according to claim 1, characterized in that The obtaining of the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system includes: Determine a first intercept of the initial ground plane equation on the Y axis of the initial camera coordinate system; Determine the second intercept of the current ground plane equation to be solved on the Y axis of the current camera coordinate system; Obtain the distance difference between the current moment and the initial moment detected by the laser ranging unit; Based on the normal vector of the current ground plane equation to be solved, the first intercept, the second intercept and the distance difference, the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system are determined.

4. The method according to claim 1, wherein After determining that the category of the corresponding point is the ground plane, the method further includes: A current drivable path is determined based on all points whose category is the ground plane.

5. The method according to claim 1, wherein The object information also includes a position; and determining the object information in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point includes: Performing clustering processing based on the three-dimensional position of each point to obtain a plurality of clustered first point clouds; determining a center position of each first point cloud in the plurality of first point clouds; Determining a value to be determined corresponding to each first point cloud based on a center position of each first point cloud; The category of each first point cloud is determined according to the value to be determined and the obtained second preset distance.

6. An object information detection device, characterized in that: include: A first acquisition module is used to obtain an initial ground plane equation and initial pose information of the acquisition unit; The initial ground plane equation is determined based on the surrounding point cloud information at the initial moment; A second acquisition module is configured to acquire current surrounding environment point cloud information based on the acquisition unit; the current surrounding environment point cloud information includes a three-dimensional position of each point; A third acquisition module is used to obtain the current posture information of the acquisition unit; A first determining module is configured to determine a posture deviation based on the current posture information and the initial posture information when a similarity value between the current posture information and the initial posture information is less than or equal to a preset value; A second determining module is configured to determine a current ground plane equation based on the posture deviation and the initial ground plane equation; A third determining module is configured to determine information about an object in the current surrounding environment point cloud information based on the current ground plane equation and the three-dimensional position of each point; the object information includes a category; The pose deviation is in the form of a quaternion; the second determination module is used to convert the pose deviation in the form of a quaternion into a corresponding rotation matrix; establish a current ground plane equation to be solved; determine the normal vector of the current ground plane equation to be solved based on the normal vector of the initial ground plane equation and the rotation matrix; obtain the coordinates of the intersection of the current ground plane equation to be solved and the Y axis of the current camera coordinate system; and obtain the solved current ground plane equation based on the normal vector of the current ground plane equation to be solved and the coordinates of the intersection. The third determining module is configured to determine a value to be determined corresponding to each point based on the three-dimensional position of each point and the current ground plane equation; Determining the category of each point based on the value to be determined and the obtained first preset distance; and if the value to be determined is greater than or equal to the first preset distance, determining that the category of the corresponding point is a raised obstacle; or; If the value to be determined is less than the first preset distance and greater than the opposite of the first preset distance, then the category of the corresponding point is determined to be a ground plane; or; If the value to be determined is less than or equal to the inverse number of the first preset distance, the category of the corresponding point is determined to be a concave obstacle.

7. A computer storage medium, characterized in that The storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the object information detection method according to any one of claims 1 to 5.

Images