Object information extraction method and device, robot, and computer-readable storage medium

Abstract

The application discloses an object information extraction method and device, a robot and a computer readable storage medium, wherein the method comprises: acquiring environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, the target environment comprising at least one object to be extracted; performing three-dimensional reconstruction based on the environment point cloud data to obtain a plurality of triangular facets corresponding to the target environment; and clustering the triangular facets according to the object semantic information, and extracting object information corresponding to the object to be extracted from the clustered triangular facets. Thus, the method extracts object information corresponding to the object to be extracted from the clustered triangular facets by clustering the triangular facets based on the plurality of triangular facets obtained by three-dimensional reconstruction of the environment point cloud data and the object semantic information corresponding to the environment point cloud data, effectively solves the problem that object information cannot be extracted, and improves the accuracy of object information extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and in particular to an object information extraction method and device, a robot, and a computer readable storage medium. BACKGROUND

[0002] Environment structuring refers to a technology of extracting various object information (such as object models and object point clouds) from a cluttered environment, and is an indispensable process before a robot is positioned after perceiving an environment. In related technologies, a robot obtains object information directly from an environment design drawing or draws an object model based on computer aided design (CAD) through reverse engineering, thereby obtaining the object information. The method of obtaining the object information needs to be implemented on the premise of an existing environment model, and the robot cannot directly obtain object information in an environment. Moreover, the traditional CAD method needs to be manually drawn, is low in efficiency, and cannot guarantee the accuracy of the obtained object information. SUMMARY

[0003] The present application aims to at least partially solve one of the technical problems in the related art. To this end, the present application provides an object information extraction method, which effectively solves the problem of being unable to accurately extract object information.

[0004] The first aspect of the present application provides an object information extraction method, which comprises: obtaining environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, the target environment comprising at least one object to be extracted; performing three-dimensional reconstruction based on the environment point cloud data to obtain a plurality of triangular facets corresponding to the target environment; and clustering the triangular facets according to the object semantic information, and extracting object information corresponding to the object to be extracted from the clustered triangular facets.

[0005] The object information extraction method according to the present application first obtains environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, the target environment comprising at least one object to be extracted, then performs three-dimensional reconstruction based on the environment point cloud data to obtain a plurality of triangular facets corresponding to the target environment, and finally clusters the triangular facets according to the object semantic information and extracts object information corresponding to the object to be extracted from the clustered triangular facets. Thus, the method clusters the plurality of triangular facets obtained by three-dimensional reconstruction based on the environment point cloud data according to the object semantic information corresponding to the environment point cloud data, extracts object information corresponding to the object to be extracted from the clustered triangular facets, effectively solves the problem of being unable to extract object information, and improves the accuracy of object information extraction.

[0006] In addition, the object information extraction method according to the above-mentioned embodiments of the present application can further have the following additional technical features:

[0007] According to an embodiment of the present application, the clustering of the triangular facets according to the object semantic information and the extraction of the object information corresponding to the object to be extracted from the clustered triangular facets include: obtaining object category information from the object semantic information; clustering the triangular facets according to the object category information to obtain a plurality of sets of facets corresponding to the object categories respectively; and performing connectivity clustering processing on the plurality of sets of facets respectively to obtain the object information corresponding to the object to be extracted.

[0008] According to an embodiment of the present application, the connectivity clustering processing on the plurality of sets of facets respectively to obtain the object information corresponding to the object to be extracted includes: constructing an undirected graph corresponding to each of the sets of facets by taking the triangular facets as nodes and the connections between the nodes with common edges as edges; performing graph search on the undirected graph to obtain connected triangular facets in the sets of facets; and extracting the object information corresponding to the object to be extracted based on the connected triangular facets.

[0009] According to an embodiment of the present application, after the graph search on the undirected graph to obtain the connected triangular facets in the sets of facets, the object information extraction method further includes: performing distance clustering on the connected triangular facets and unconnected triangular facets; and extracting the object information corresponding to the object to be extracted according to the distance clustered triangular facets.

[0010] According to an embodiment of the present application, before the three-dimensional reconstruction based on the environmental point cloud data to obtain a plurality of triangular facets corresponding to the target environment, the object information extraction method further includes: performing point cloud extraction on the environmental point cloud data according to the object semantic information; when the object point cloud data is extracted, marking the environmental point cloud data other than the object point cloud data as to-be-reconstructed point cloud data; and performing three-dimensional reconstruction based on the to-be-reconstructed point cloud data to obtain the plurality of triangular facets.

[0011] According to an embodiment of the present application, the three-dimensional reconstruction based on the environmental point cloud data to obtain a plurality of triangular facets corresponding to the target environment includes: performing voxel division on the environmental point cloud data to obtain a plurality of voxels; determining an isosurface and an edge intersection point where the isosurface intersects with the voxels; and connecting the edge intersection points according to the relative positions of the vertices of the voxels and the isosurface to obtain the plurality of triangular facets.

[0012] According to an embodiment of the present application, after obtaining the environmental point cloud data corresponding to the target environment, the object information extraction method further includes: performing voxel division on the environmental point cloud data to obtain a plurality of voxels; obtaining a signed distance corresponding to the voxels; fusing the environmental point cloud data according to the signed distance to obtain a surface point cloud; and performing three-dimensional reconstruction on the surface point cloud to obtain a plurality of triangular facets.

[0013] To achieve the above object, the second aspect of the present application provides an object information extraction device, comprising: a data acquisition module, configured to acquire environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, wherein the target environment comprises at least one object to be extracted; a reconstruction module, configured to perform three-dimensional reconstruction on the environment point cloud data to obtain a plurality of triangular facets; and an identification module, configured to cluster the triangular facets according to the object semantic information and extract object information corresponding to the object to be extracted from the clustered triangular facets.

[0014] According to the object information extraction device provided by the embodiment of the present application, the data acquisition module is configured to acquire environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, wherein the target environment comprises at least one object to be extracted; the reconstruction module is configured to perform three-dimensional reconstruction on the environment point cloud data to obtain a plurality of triangular facets; and the identification module is configured to cluster the triangular facets according to the object semantic information and extract object information corresponding to the object to be extracted from the clustered triangular facets. Thus, the device clusters the triangular facets obtained by three-dimensional reconstruction of the environment point cloud data according to the object semantic information corresponding to the environment point cloud data, extracts object information corresponding to the object to be extracted from the clustered triangular facets, effectively solves the problem that the object information cannot be extracted, and improves the accuracy of object information extraction.

[0015] To achieve the above object, the third aspect of the present application provides a robot, comprising: a memory, a processor, and an object information extraction program stored in the memory and executable on the processor, wherein the processor implements the object information extraction method when executing the object information extraction program.

[0016] According to the robot provided by the embodiment of the present application, the object information extraction method is used, effectively solves the problem that the object information cannot be extracted, and improves the accuracy of object information extraction.

[0017] To achieve the above object, the fourth aspect of the present application provides a computer readable storage medium, which stores an object information extraction program, and the object information extraction program is executable on a processor to implement the object information extraction method.

[0018] According to the computer readable storage medium provided by the embodiment of the present application, the object information extraction method is used, effectively solves the problem that the object information cannot be extracted, and improves the accuracy of object information extraction.

[0019] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A schematic diagram of a hardware structure of a multi-legged robot according to an embodiment of the present application;

[0021] Figure 2 A perspective schematic diagram of a multi-legged robot according to an embodiment of the present application;

[0022] Figure 3 A flowchart of an object information extraction method according to an embodiment of the present application;

[0023] Figure 4 A schematic diagram of an isosurface acquisition according to an embodiment of the present application;

[0024] Figure 5 A schematic diagram of modeling and clustering of triangular facets according to an embodiment of the present application;

[0025] Figure 6 A flowchart of an object information extraction method according to an embodiment of the present application;

[0026] Figure 7 A block schematic diagram of an object information extraction apparatus according to an embodiment of the present application;

[0027] Figure 8 A block schematic diagram of a robot according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below with reference to the attached drawings, wherein the same or like reference numerals in different drawings denote the same or like components or components having the same or similar functions. It is to be understood that the embodiments described below are exemplary, and are intended to explain the present application, and are not to be understood as limiting the present application.

[0029] In the following description, the suffixes used for components, such as "module", "part", or "unit", are used only for convenience of explanation of the present application, and have no specific meaning by themselves. Thus, "module", "part", or "unit" can be mixedly used.

[0030] Referring to Figure 1 , Figure 1 A schematic diagram of a hardware structure of a multi-legged robot 100 according to an embodiment of the present application. In Figure 1 In the embodiment shown, the multi-legged robot 100 includes a mechanical unit 101, a communication unit 102, a sensing unit 103, an interface unit 104, a storage unit 105, a control module 110, and a power supply 111. The various components of the multi-legged robot 100 can be connected in any manner, including wired or wireless connection, etc. Those skilled in the art can understand that Figure 1The specific structure of the multi-legged robot 100 shown does not constitute a limitation on the multi-legged robot 100. The multi-legged robot 100 may include more or fewer parts than shown. Some parts are not essential components of the multi-legged robot 100 and may be omitted or combined as needed without changing the nature of the invention.

[0031] The following is combined Figure 1 A detailed description of each component of the multi-legged robot 100:

[0032] Mechanical unit 101 is the hardware of multi-legged robot 100. For example... Figure 1 As shown, the mechanical unit 101 may include a drive board 1011, a motor 1012, and a mechanical structure 1013, such as... Figure 2 As shown, the mechanical structure 1013 may include a main body 1014, extendable legs 1015, and feet 1016. In other embodiments, the mechanical structure 1013 may also include an extendable robotic arm (not shown), a rotatable head structure 1017, a rocking tail structure 1018, a cargo-carrying structure 1019, a saddle structure 1020, a camera structure 1021, etc. It should be noted that the various component modules of the mechanical unit 101 can be one or multiple, depending on the specific situation. For example, there may be four legs 1015, and each leg 1015 may be equipped with three motors 1012, resulting in a total of twelve motors 1012.

[0033] The communication unit 102 can be used for receiving and sending signals, and can also communicate with networks and other devices. For example, it can receive instructions from a remote control or other multi-legged robot 100 to move in a specific direction at a specific speed according to a specific gait, and then transmit these instructions to the control module 110 for processing. The communication unit 102 includes modules such as WiFi, 4G, 5G, Bluetooth, and infrared modules.

[0034] The sensing unit 103 is configured to acquire information data of the environment around the multi-legged robot 100 and monitor parameter data of components inside the multi-legged robot 100, and transmit the acquired information data and the monitored parameter data to the control module 110. The sensing unit 103 includes various sensors, such as sensors configured to acquire information of the environment around the multi-legged robot 100, such as a laser radar (for long-range object detection, distance determination, and / or speed value determination), a millimeter wave radar (for short-range object detection, distance determination, and / or speed value determination), a camera, an infrared camera, a Global Navigation Satellite System (GNSS), and the like. The sensing unit 103 also includes sensors configured to monitor components inside the multi-legged robot 100, such as an Inertial Measurement Unit (IMU) (for measuring values of speed, acceleration, and angular velocity), a plantar sensor (for monitoring plantar force point position, plantar posture, ground contact force, and direction), and a temperature sensor (for detecting component temperature). The multi-legged robot 100 can also be equipped with other sensors, such as a load sensor, a touch sensor, a motor angle sensor, a torque sensor, and the like, which are not described herein.

[0035] The interface unit 104 can be configured to receive input (e.g., data information, power, and the like) from an external device and transmit the received input to one or more components inside the multi-legged robot 100, or can be configured to output (e.g., data information, power, and the like) to an external device. The interface unit 104 can include a power supply port, a data port (e.g., a USB port), a memory card port, a port for connecting a device having an identification module, an audio input / output (I / O) port, a video I / O port, and the like.

[0036] The storage unit 105 is configured to store software programs and various data. The storage unit 105 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system program, a motion control program, an application program (such as a text editor), and the like; and the data storage area can store data generated by the multi-legged robot 100 during use (such as various sensing data acquired by the sensing unit 103, log file data), and the like. In addition, the storage unit 105 can include a high-speed random access memory, and can also include a non-volatile memory, such as a disk memory, a flash memory, or other volatile solid-state memory.

[0037] The display unit 106 is configured to display information input by a user or information provided to a user. The display unit 106 can include a display panel 1061, which can be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), and the like.

[0038] The input unit 107 can be configured to receive input digital or character information. Specifically, the input unit 107 can include a touch panel 1071 and other input devices 1072. The touch panel 1071, also called a touch screen, can collect touch operations of a user (such as the user's operation on or near the touch panel 1071 using a palm, a finger, or a suitable accessory) and drive the corresponding connection device according to a pre-set program. The touch panel 1071 can include two parts, a touch detection device 1073 and a touch controller 1074. The touch detection device 1073 detects the touch position of the user and detects the signal generated by the touch operation, and transmits the signal to the touch controller 1074; the touch controller 1074 receives the touch information from the touch detection device 1073, converts it into touch coordinates, and sends it to the control module 110, and can also receive the command sent by the control module 110 and execute it. In addition to the touch panel 1071, the input unit 107 can also include other input devices 1072. Specifically, the other input devices 1072 can include one or more of a remote control handle and the like, and the specific embodiments are not limited here.

[0039] Further, the touch panel 1071 can cover the display panel 1061, and when the touch panel 1071 detects a touch operation on or near it, it transmits to the control module 110 to determine the type of touch event, and then the control module 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in the Figure 1 In the above embodiment, the touch panel 1071 and the display panel 1061 are realized as two independent components to realize the input and output functions respectively, but in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions, and the specific embodiments are not limited here.

[0040] The control module 110 is the control center of the multi-legged robot 100, which connects all components of the multi-legged robot 100 through various interfaces and lines, and controls the multi-legged robot 100 as a whole by running or executing the software programs stored in the storage unit 105 and calling the data stored in the storage unit 105.

[0041] The power supply 111 is used to supply power to each component. The power supply 111 can include a battery and a power control board, and the power control board is used to control the functions such as battery charging, discharging, and power consumption management. In the embodiment shown in Figure 1 The power supply 111 is electrically connected to the control module 110, and in other embodiments, the power supply 111 can also be electrically connected to the sensing unit 103 (such as a camera, a radar, a sound box, etc.) and the motor 1012. It should be noted that each component can be connected to a different power supply 111, or powered by the same power supply 111.

[0042] On the basis of the above-mentioned embodiments, specifically, in some embodiments, the terminal device can be connected in communication with the multi-legged robot 100, and when the terminal device communicates with the multi-legged robot 100, the terminal device can send instruction information to the multi-legged robot 100, the multi-legged robot 100 can receive the instruction information through the communication unit 102, and can transmit the instruction information to the control module 110, so that the control module 110 can process the target speed value according to the instruction information. The terminal device includes but is not limited to: a mobile phone with image shooting function, a tablet computer, a server, a personal computer, a wearable smart device, and other electrical appliances.

[0043] The instruction information can be determined according to a preset condition. In an embodiment, the multi-legged robot 100 can include a sensing unit 103, which can generate instruction information according to the current environment of the multi-legged robot 100. The control module 110 can determine whether the current speed value of the multi-legged robot 100 meets the corresponding preset condition according to the instruction information. If it meets, the current speed value and the current gait of the multi-legged robot 100 are maintained; if it does not meet, the target speed value and the corresponding target gait are determined according to the corresponding preset condition, so that the multi-legged robot 100 can be controlled to move at the target speed value and the corresponding target gait. The environmental sensor can include a temperature sensor, a barometric pressure sensor, a visual sensor, and a sound sensor. The instruction information can include temperature information, barometric pressure information, image information, and sound information. The communication mode between the environmental sensor and the control module 110 can be wired communication or wireless communication. The wireless communication mode includes but is not limited to: wireless network, mobile communication network (3G, 4G, 5G, etc.), Bluetooth, and infrared.

[0044] The hardware structure and mechanical structure of the multi-legged robot provided by the present application are described above, and the object information extraction method, object information extraction device, robot and computer readable storage medium provided by the embodiment of the present application are described below with reference to the accompanying drawings.

[0045] Please refer to Figure 3 , Figure 3 The flowchart of the object information extraction method according to the embodiment of the present application. The object information extraction method can include the following steps:

[0046] S301, obtaining environment point cloud data corresponding to a target environment, and object semantic information corresponding to the environment point cloud data, the target environment including at least one object to be extracted.

[0047] The target environment is the surrounding environment required to be perceived by the robot. The robot extracts object models in the surrounding environment, and the objects in the surrounding environment are the objects to be extracted. The robot can scan the surrounding environment through devices such as a laser radar and an RGB-D camera to obtain environment point cloud data, which includes horizontal coordinate values, vertical coordinate values, and vertical coordinate values. If a laser radar is used to obtain the environment point cloud data, the environment point cloud data can also include radar echo power values. It can be understood that the horizontal coordinates, vertical coordinates, and vertical coordinates included in the environment point cloud data are coordinate values in the robot coordinate system. The robot coordinate system can have the robot travel direction as the horizontal axis, the robot center as the origin, the line perpendicular to the robot travel direction and parallel to the ground as the vertical axis, and the line perpendicular to the ground as the vertical axis. The environment point cloud of the target environment is a collection of multiple data points. The object semantic information can be an object category. The object categories of the environment point cloud data corresponding to each data point are not the same. Based on deep learning, the data points are classified to determine the object semantic information corresponding to the environment point cloud data.

[0048] Specifically, taking the acquisition of data point cloud data by an RGB-D camera as an example, the environment point cloud data are coordinate values in the RGB-D camera coordinate system. First, the robot takes a photo of the target environment where the object to be extracted is located through the RGB-D camera to obtain a color image and a depth image of the target environment. The depth information of the target environment is determined according to the depth image, and then the environment point cloud data of the target environment are recovered based on the RGB-D camera intrinsic parameters, and the object semantic information corresponding to the environment point cloud data is obtained from the depth information and the color image based on a neural network. The object semantic information is attribute information of the object, which can include the outer contour of the object, the volume of the object, the area of the object, and the like.

[0049] It should be noted that the RGB-D camera can realize depth measurement by adding a depth measurement function to the function of an RGB ordinary camera, thereby obtaining a depth image of the object and determining the depth information of the target environment. The method for obtaining the depth information includes but is not limited to the acquisition technology of depth point cloud. The specific method can be set according to actual conditions. When the depth information is obtained by the acquisition technology of depth point cloud, the RGB-D camera can obtain the depth information through a single or multiple fusion sensors. For example, the depth information can be obtained by calculating binocular disparity through a binocular ranging method, or the depth information can be directly obtained by a laser range finder, a TOF (Time Of Flight) sensor, or the like. The specific method can be applied according to actual conditions. In addition, the above depth information acquisition process includes but is not limited to one or more methods such as a neural network, deep learning, and reinforcement learning.

[0050] In addition, the method for obtaining semantic information of the object through the color image and the depth information includes, but is not limited to, using one or more methods such as a neural network, deep learning, reinforcement learning, and the like to identify one or more information of the object, such as a volume, an area, a complete or partial surface contour of the object, and even spatial relationship information between different objects, such as a boundary line intersecting between the objects.

[0051] S302, three-dimensional reconstruction is performed based on the environment point cloud data, and a plurality of triangular facets corresponding to the target environment are obtained.

[0052] The three-dimensional reconstruction refers to establishing a mathematical model suitable for computer representation and processing for a three-dimensional object, and the triangular facet is a data structure used for modeling various irregular objects in computer graphics. In a facet of a polygonal mesh, the triangular facet is the smallest unit of segmentation, that is, the mathematical model obtained through the three-dimensional reconstruction is used to approximate the actual continuous surface by using the triangular facet. Therefore, according to the coordinate data corresponding to the environment point cloud, the three-dimensional reconstruction is performed according to a preset three-dimensional modeling rule, the mathematical model corresponding to the target environment is established, and a plurality of triangular facets corresponding to the data model are obtained. It should be noted that the method for obtaining the triangular facet can be applied according to actual conditions. For example, the Marching cubes (isosurface extraction) algorithm can be used to extract an isosurface, the triangular facet is used as an approximate representation of the isosurface, and the plurality of triangular facets corresponding to the target environment are determined.

[0053] In an embodiment of the present application, the three-dimensional reconstruction is performed based on the environment point cloud data, and a plurality of triangular facets corresponding to the target environment are obtained, including: voxel division is performed on the environment point cloud data to obtain a plurality of voxels; an isosurface and an edge intersection point of the isosurface and the voxel are determined; and the edge intersection points are connected according to the relative positions of the vertices of the voxel and the isosurface to obtain the plurality of triangular facets.

[0054] Specifically, the environment point cloud data obtained according to the depth information and the color image is voxel-divided, and the environment point cloud data is divided into small cubes in space, that is, voxels. Figure 4 The voxels in the data field are processed one by one through the Marching cubes algorithm, and the plane inside the voxel, that is, the triangular facet, is used to approximate the isosurface.

[0055] As shown in Figure 4As shown, each voxel has eight vertices, and the eight vertices of each voxel are classified according to the threshold of the isosurface to be calculated, to determine whether the vertex is inside or outside the isosurface. That is, the vertices of each voxel are compared with the threshold of the isosurface, if the data value of the vertex is greater than the threshold of the isosurface, the vertex is defined to be inside the isosurface, and is marked as "+", if the data of the vertex is less than the threshold of the isosurface, the vertex is defined to be outside the isosurface, and is marked as "-". Thus, when all the vertices of a voxel are "+" or "-", it is considered that the isosurface does not intersect with the voxel, and when there are "+" and "-" vertices in a voxel, it is considered that the isosurface intersects with the voxel.

[0056] Based on the above method, the voxels intersecting with the isosurface are separated, the vertices greater than the threshold of the isosurface and the vertices less than the threshold of the isosurface in the voxel are separated by the triangular patches, and the triangular subdivision mode of the final voxel is determined. Specifically, it is assumed that the basic assumption of the Marching cubes algorithm is that the function along the edge data field of the voxel is linearly changed, and the intersection points of the isosurface and the voxel boundary are calculated by linear interpolation according to the basic assumption, so as to obtain the edge intersection points of the isosurface and the voxel. Finally, the edge intersection points determined by the above intersection are taken as the vertices of the triangular patches, and the triangular patches are connected to obtain a plurality of triangular patches, which are used as the approximation representation of the isosurface in the voxel. Thus, according to the relative position of each vertex of the voxel and the isosurface, the edge intersection points of the isosurface and the edge of the voxel are connected in a certain way to form a triangle, and a plurality of triangular patches are obtained, which are used for the approximation representation of the isosurface.

[0057] S303, clustering the triangular patches according to the object semantic information, and extracting the object information corresponding to the object to be extracted from the clustered triangular patches.

[0058] That is, according to the determined object attribute information, the above-mentioned obtained triangular patches are classified, the triangular patches with the same object attribute information are gathered together to form multiple different class clusters, and then the class clustered triangular patches are combined with the object semantic information to extract the object information. The object information can be an object point cloud, a triangular patch, or an object model generated based on the triangular patch, such as a geometric model, which is not limited here. Compared with the method of directly converting the object semantic information to the same coordinate system as the environment point cloud data and then extracting the object information, the present application effectively avoids the situation that the object information extraction uncertainty increases due to part of the environment point cloud data not being recognized, so that the obtained object information is more accurate. The object information extraction method extracts the object information based on the environment point cloud data obtained by using an RGB-D camera and the like, combines the object semantic information of deep learning and the triangular patches obtained by three-dimensional reconstruction, and effectively solves the problem that the object point cloud surface geometric features are not obvious due to depth uncertainty, which leads to the inability to extract the object.

[0059] In an embodiment of the present application, the clustering of the triangular patches according to the object semantic information, and the extraction of the object information corresponding to the object to be extracted from the clustered triangular patches can include: obtaining object category information from the object semantic information; clustering the triangular patches according to the object category information to obtain a plurality of patch sets corresponding to the respective object categories; and performing connectivity clustering processing on the plurality of patch sets respectively to obtain the object information corresponding to the object to be extracted.

[0060] That is, first, the triangular patches are classified according to the object category information, then the triangular patches within the same category are clustered to obtain multiple class clusters, i.e., patch sets, and then the triangular patches in each class cluster are respectively subjected to connectivity clustering to further extract the object information. Specifically, before clustering the plurality of triangular patches determined according to the object semantic information of the environment point cloud data, the triangular patches are classified according to the object semantic information of each environment point cloud data, different objects are separated, and a plurality of patch sets are obtained. Then, the triangular patches in each patch set are respectively subjected to connectivity clustering processing, and the object information is further extracted from the connectivity clustered triangular patches, so that the extracted object information is more accurate.

[0061] In an embodiment of the present application, the connectivity clustering processing is performed on the plurality of patch sets respectively to obtain the object information corresponding to the object to be extracted, which includes: constructing an undirected graph corresponding to each patch set by taking the triangular patches as nodes and the connection lines between the nodes with common edges as edges; performing graph search on the undirected graph to obtain connected triangular patches in the patch set; and extracting the object information corresponding to the object to be extracted based on the connected triangular patches.

[0062] Specifically, within a set of facets, all triangular facets are simplified to nodes. If two triangular facets share a common edge (i.e., they are connected), the two nodes are connected by a line. If two triangular facets do not share a common edge (i.e., they are not connected), no connection is made. This process is repeated for all triangular facets in the set, using the common edge method to cluster them based on connectivity, resulting in several corresponding undirected graphs. Then, object information is extracted based on the undirected graphs composed of nodes and edges. It should be noted that triangular facets that are connected by edges and used as nodes are considered connected triangular facets and are considered the same object. Triangular facets that are not connected by edges and used as nodes are considered disconnected triangular facets and are considered different objects. Connected triangular facets form an undirected graph. This embodiment extracts object information based on the connected triangular facets within the undirected graph, making the obtained objects more accurate.

[0063] by Figure 5 For example, suppose a set of facets consists of nine triangular facets as shown in Figure (a), which are denoted by 1, 2, 3, 4, 5, 6, 7, 8, and 9 respectively. Referring to Figure (a), triangle 1 shares the same vertex and edges with triangles 2 and 8. Therefore, after simplification, node 1 is connected to nodes 2 and 8 respectively. Similarly, triangle 2 shares the same vertex and edges with triangles 1, 4, and 7 respectively. Therefore, node 2 is not only connected to node 1, but also to nodes 4 and 7 respectively. By analogy, nodes 1, 2, 3, 4, 6, 7, 8, and 9 are connected. Triangle 5 does not share the same vertex and edges with any other triangle, so node 5 exists in isolation. Thus, triangles 1-9 in Figure (a) are simplified to nodes 1-9 in Figure (b). At this time, nodes 1, 2, 3, 4, 6, 7, 8, and 9 are connected, and node 5 exists in isolation. Triangles 1, 2, 3, 4, 6, 7, 8, and 9 are determined to be connected triangles in the set of triangles, and triangle 5 is an unconnected triangle in the set of triangles. Then, the connected triangular facets 1, 2, 3, 4, 6, 7, 8 and 9 are clustered and restored to obtain region A as shown in Figure (c). The unconnected triangular facet 5 is clustered and restored separately to obtain region B as shown in Figure (c). Finally, the object information corresponding to the object to be extracted is extracted based on the clustered and restored region A.

[0064] In addition to the above-mentioned method for extracting object information based solely on the connectivity of triangular facets, in one embodiment of the present invention, after performing graph search on an undirected graph to obtain connected triangular facets in the facet set, the object information extraction method further includes: performing distance clustering on connected and disconnected triangular facets; and extracting the object information corresponding to the object to be extracted based on the triangular facets after distance clustering.

[0065] In other words, when it is determined that an unconnected triangle is close to a connected triangle, the unconnected triangle can be added to the connected triangle for clustering and reconstruction, and this can be used to extract object information, further improving the accuracy of object information acquisition. For example, continuing with... Figure 5 For example, when it is determined that the distance between the aforementioned triangular facet 5 and at least one other triangular facet is less than a preset distance threshold, it is determined that the unconnected triangular facet is closer to the connected triangular facet, and triangular facet 5 is clustered together with the other triangular facets for reconstruction. That is, triangular facets 1-9 can be simultaneously clustered into region C, and then object information can be extracted based on region C. In addition, for the case where there are multiple unconnected triangular facets in the facet set, the relative distance between multiple unconnected triangular facets in the same facet set can also be judged. When it is determined that the distance between unconnected triangular facets is close, they can be clustered for reconstruction. Furthermore, the distance between multiple connected triangular facets in the facet set can also be judged, and multiple connected triangular facets with close distances can be clustered for reconstruction. Therefore, this object information extraction method can use a preset distance threshold as the evaluation criterion for distance clustering. After judging the connectivity of triangular facets, it further judges the relative distance between unconnected and connected triangular facets based on the preset distance threshold, and clusters and restores triangular facets whose relative distance is less than the preset distance threshold.

[0066] It should be noted that, in addition to connectivity clustering and distance clustering mentioned above, normal vector clustering can also be used to cluster triangular facets. The specific method can be set according to the actual situation.

[0067] In another embodiment of the present invention, before performing three-dimensional reconstruction based on environmental point cloud data to obtain multiple triangular facets corresponding to the target environment, the object information extraction method further includes: extracting point cloud data from environmental point cloud data according to object semantic information; when object point cloud data is extracted, marking environmental point cloud data other than object point cloud data as point cloud data to be reconstructed; and performing three-dimensional reconstruction based on the point cloud data to be reconstructed to obtain multiple triangular facets.

[0068] Specifically, the object point cloud data refers to the point cloud data corresponding to a single object determined according to object semantic information, and the point cloud data to be reconstructed is the environmental point cloud data that needs to be reconstructed by three-dimensional reconstruction. After obtaining the environmental point cloud data and the object semantic information according to the depth information of the object and the color image, first, the object information that can be directly extracted is extracted according to the object semantic information, and then the environmental point cloud data from which the object information cannot be directly extracted according to the object semantic information is reconstructed by three-dimensional modeling to determine the triangular facets, thereby saving a large amount of computing resources. Assuming that the environmental point cloud data includes 1000 data points, the object is first extracted from the 1000 data points based on the semantic information, if it is judged that the single object can be directly extracted from 900 data points according to the semantic information, the single object is directly extracted, at this time, only the remaining 100 data points are reconstructed by three-dimensional reconstruction. If the single object cannot be extracted from the 1000 data points based on the semantic information, the 1000 data points are directly reconstructed by three-dimensional reconstruction.

[0069] In an embodiment of the present application, after obtaining the environmental point cloud data corresponding to the target environment, the object information extraction method further includes: voxel division is performed on the environmental point cloud data to obtain a plurality of voxels; a signed distance corresponding to the voxel is obtained; the environmental point cloud data is fused according to the signed distance to obtain a surface point cloud; and the surface point cloud is reconstructed by three-dimensional reconstruction to obtain a plurality of triangular facets.

[0070] The surface point cloud refers to the environmental point cloud corresponding to the surface of the object. That is, the plurality of frames of environmental point cloud data of the same target environment are respectively voxel-divided, and the environmental point cloud data is fused according to the signed distance in the voxel, that is, the environmental point cloud data is superimposed into a data field, the surface point cloud used for object information extraction is determined based on the fused environmental point cloud data, the data point density of the surface point cloud is greatly increased, a smoother object surface can be obtained, and thus the geometric model of the object is obtained based on the surface point cloud determined by fusion, and the accuracy of the object information acquisition is further improved.

[0071] Specifically, taking the RGB-D camera that can output 30 frames of color images and depth images within 1s as an example, a color point cloud image can be obtained from each frame of color image and depth image, and therefore the final point cloud image needs to be determined for object information extraction. After obtaining the environmental point cloud data from the depth information determined according to the depth image and the color image, the TSDF (Truncated Signed Distance Function) algorithm is used to fuse the plurality of frames of environmental point cloud data determined, and the specific operation is as follows:

[0072] Since each frame of point cloud image is determined based on the RGB-D camera coordinate system, and the camera coordinate system corresponding to each frame of point cloud image is different, the camera position needs to be estimated first. Specifically, first, the camera position in the world coordinate system is estimated, then a cube is defined in the world coordinate system, and the cube is cut into a plurality of small cubes, i.e. voxels. For each small cube, first, a first coordinate in the world coordinate system is determined, then a camera transformation matrix is obtained according to the position of the camera in the world coordinate system and the position of the camera in the camera coordinate system, and a second coordinate of the small cube in the camera coordinate system is obtained according to the camera transformation matrix and the first coordinate. Then, a third coordinate in the environment point cloud image coordinate system is obtained according to the second coordinate and the camera internal and external parameters, which can reflect the depth information of each small cube.

[0073] When the depth information of each small cube is determined, the signed distance is obtained according to the depth information and the object surface point cloud. Specifically, the distance between the voxel and the nearest object surface in each voxel is stored. If the voxel is in front of the object surface, the sign of the stored distance is positive; if the voxel is behind the object surface, the sign of the stored distance is negative. Generally, the object surface is considered to have a thickness, so both the too large distance value and the too small distance value are set to 1 or -1, which is the truncated distance, i.e. the so-called TSDF model. In the TSDF model, the place with a signed distance of 0 is the object surface, and the voxel with a signed distance of 0 is obtained. In this way, the processing of 30 frames of environment point cloud data is completed in turn, and the voxel with a signed distance of 0 in each frame of environment point cloud data is obtained. The voxels with a signed distance of 0 in multiple frames are fused as surface point cloud. Thus, the surface point cloud determined after the fusion of multiple frames is used as the final point cloud image, and the surface point cloud is subjected to three-dimensional reconstruction to determine a plurality of triangular facets, thereby extracting the object information of the object to be extracted. Thus, in the processing of the surface point cloud of the object, the TSDF algorithm distinguishes the positive and negative values of the points at both ends of the object surface, then obtains the surface point cloud through the fusion of multiple frames, and extracts the object information based on the surface point cloud, thereby saving a large amount of memory and computing resources such as computing power.

[0074] It should be noted that, in addition to the above method of fusing multiple frames of environment point cloud data to determine the surface point cloud, a plurality of single frames of environment point cloud data can also be fused to determine the surface point cloud, and three-dimensional reconstruction is performed on the surface point cloud to obtain a plurality of triangular facets for object information extraction. In addition, for single frame environment point cloud data, the object surface point cloud data, i.e. the surface point cloud, can also be directly determined according to the signed distance in the single frame voxel, i.e. three-dimensional reconstruction is performed on the surface point cloud determined according to the single frame to obtain a plurality of triangular facets.

[0075] It should be further noted that when the semantic information corresponding to the environment point cloud data cannot be obtained according to the depth information and the color image, the environment point cloud data can be directly clustered to extract the object information corresponding to the environment point cloud data. When the triangular facets are not obtained through the three-dimensional reconstruction, the environment point cloud data without the triangular facets can also be clustered to extract the object information corresponding to the clustered environment point cloud data.

[0076] In one embodiment, as shown in FIG. 1, the object information extraction method comprises the following steps: Figure 6

[0077] S601, obtaining the depth information and the color image corresponding to a target environment.

[0078] S602, obtaining the environment point cloud data corresponding to the target environment and the object semantic information corresponding to the environment point cloud data.

[0079] S603, judging whether the environment point cloud data has the object semantic information. If not, step S604 is executed; if yes, step S605 is executed.

[0080] S604, clustering the environment point cloud data without the object semantic information to extract the object information from the environment point cloud data.

[0081] S605, judging whether a single object is recognized based on the object semantic information. If yes, step S606 is executed; if not, step S607 is executed.

[0082] S606, extracting the point cloud data from the environment point cloud data according to the object semantic information to obtain the object point cloud data.

[0083] S607, marking the environment point cloud data other than the object point cloud data as the to-be-reconstructed point cloud data, and performing the three-dimensional reconstruction on the to-be-reconstructed point cloud data.

[0084] S608, judging whether the to-be-reconstructed point cloud data obtains a plurality of triangular facets through the three-dimensional reconstruction. If not, step S609 is executed; if yes, step S610 is executed.

[0085] S609, clustering the to-be-reconstructed point cloud data to extract the object information from the to-be-reconstructed point cloud data.

[0086] S610, obtaining the object category information from the object semantic information.

[0087] S611, clustering the triangular facets according to the object category information to obtain the facet geometry corresponding to each of the object categories.

[0088] S612, constructing the undirected graph corresponding to each of the facet sets by taking the triangular facets as nodes and the connection lines between the nodes with common edges as edges.​

[0089] S613, performing graph search on the undirected graph to obtain connected triangular patches in the patch set.

[0090] S614, extracting object information corresponding to the to-be-extracted object based on the connected triangular patches.

[0091] Compared with a method of directly using spatial information or intensity information of an object surface point cloud, the object information extraction method combines rich object semantic information and three-dimensional reconstruction to assist object extraction, can extract an object by using object semantic information in combination with a traditional point cloud geometric space method, and can also extract an object by combining object semantic information and triangular patch structures after three-dimensional reconstruction, thereby improving the accuracy of object extraction.

[0092] To sum up, according to the object information extraction method provided in the embodiments of the present application, first, environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data are acquired, the target environment includes at least one to-be-extracted object, then three-dimensional reconstruction is performed based on the environment point cloud data to obtain a plurality of triangular patches corresponding to the target environment, and finally, the triangular patches are clustered according to the object semantic information, and object information corresponding to the to-be-extracted object is extracted from the clustered triangular patches. Therefore, the plurality of triangular patches obtained by three-dimensional reconstruction based on the environment point cloud data are clustered by using the object semantic information corresponding to the environment point cloud data, and object information corresponding to the to-be-extracted object is extracted from the clustered triangular patches, thereby effectively solving the problem of being unable to extract object information and improving the accuracy of object information extraction.

[0093] According to the above-mentioned embodiments, the present application further provides an object information extraction device.

[0094] Please refer to Figure 7 , Figure 7 The block schematic diagram of the object information extraction device according to the embodiments of the present application, the object information extraction device provided in the embodiments of the present application can include a data acquisition module 701, a reconstruction module 702 and an identification module 703.

[0095] The data acquisition module 701 is configured to acquire environment point cloud data corresponding to a target environment and object semantic information corresponding to the environment point cloud data, wherein the target environment includes at least one to-be-extracted object. The reconstruction module 702 is configured to perform three-dimensional reconstruction on the environment point cloud data to obtain a plurality of triangular patches. The identification module 703 is configured to cluster the triangular patches according to the object semantic information, and extract object information corresponding to the to-be-extracted object from the clustered triangular patches.

[0096] According to an embodiment of the present application, the identification module 703 clusters the triangular facets according to the object semantic information, extracts the object information corresponding to the object to be extracted from the clustered triangular facets, and specifically is configured to: obtain object category information from the object semantic information; cluster the triangular facets according to the object category information to obtain a plurality of facet sets corresponding to the object categories respectively; and perform connectivity clustering processing on the plurality of facet sets respectively to obtain the object information corresponding to the object to be extracted.

[0097] According to an embodiment of the present application, the identification module 703 performs connectivity clustering processing on the plurality of facet sets respectively to obtain the object information corresponding to the object to be extracted, and specifically is configured to: construct an undirected graph corresponding to each of the facet sets by taking the triangular facets as nodes and the connection lines between the nodes with common edges as edges; perform graph search on the undirected graph to obtain connected triangular facets in the facet sets; and extract the object information corresponding to the object to be extracted based on the connected triangular facets.

[0098] According to an embodiment of the present application, after the identification module 703 performs graph search on the undirected graph to obtain the connected triangular facets in the facet sets, the identification module 703 is further configured to: perform distance clustering on the connected triangular facets and unconnected triangular facets; and extract the object information corresponding to the object to be extracted according to the distance clustered triangular facets.

[0099] According to an embodiment of the present application, before the reconstruction module 702 performs three-dimensional reconstruction based on the environment point cloud data to obtain a plurality of triangular facets corresponding to the target environment, the reconstruction module 702 is further configured to: perform point cloud extraction on the environment point cloud data according to the object semantic information; when the object point cloud data is extracted, mark the environment point cloud data other than the object point cloud data as to-be-reconstructed point cloud data; and perform three-dimensional reconstruction based on the to-be-reconstructed point cloud data to obtain the plurality of triangular facets.

[0100] According to an embodiment of the present application, the reconstruction module 702 performs three-dimensional reconstruction based on the environment point cloud data to obtain a plurality of triangular facets corresponding to the target environment, and specifically is configured to: perform voxel division on the environment point cloud data to obtain a plurality of voxels; determine an isosurface and an edge intersection point where the isosurface intersects with the voxels; and connect the edge intersection points according to the relative positions of the vertices of the voxels and the isosurface to obtain the plurality of triangular facets.

[0101] According to an embodiment of the present application, after obtaining the environment point cloud data corresponding to the target environment, the reconstruction module 702 is further configured to: perform voxel division on the environment point cloud data to obtain a plurality of voxels; obtain a signed distance corresponding to the voxels; fuse the environment point cloud data according to the signed distance to obtain a surface point cloud; and perform three-dimensional reconstruction on the surface point cloud to obtain a plurality of triangular facets.

[0102] It should be noted that details not disclosed in the object information extraction device of the embodiments of the present application are referred to the details disclosed in the object information extraction method of the above-mentioned embodiments of the present application, and will not be described here.

[0103] In summary, according to the object information extraction device of the embodiments of the present application, the environment point cloud data corresponding to the target environment and the object semantic information corresponding to the environment point cloud data are obtained by the data acquisition module, wherein the target environment includes at least one object to be extracted, the environment point cloud data is three-dimensionally reconstructed by the reconstruction module to obtain a plurality of triangular facets, and the triangular facets are clustered according to the object semantic information by the identification module, and the object information corresponding to the object to be extracted is extracted from the clustered triangular facets. Therefore, based on the plurality of triangular facets obtained by three-dimensional reconstruction of the environment point cloud data, the triangular facets are clustered by the object semantic information corresponding to the environment point cloud data, and the object information corresponding to the object to be extracted is extracted from the clustered triangular facets, effectively solving the problem of being unable to extract object information, and improving the accuracy of object information extraction.

[0104] Corresponding to the above-mentioned embodiments, the present application also proposes a robot.

[0105] Please refer to Figure 8 , Figure 8 The block diagram of the robot according to the embodiments of the present application, the robot 800 of the embodiments of the present application, comprising: a memory 810, a processor 820, and an object information extraction program stored on the memory 810 and executable on the processor 820, and when the processor 820 executes the object information extraction program, the object information extraction method described above is realized.

[0106] According to the robot of the embodiments of the present application, based on the above-mentioned object information extraction method, the problem of being unable to extract object information is effectively solved, and the accuracy of object information extraction is improved.

[0107] Corresponding to the above-mentioned embodiments, the present application also proposes a computer readable storage medium.

[0108] The computer readable storage medium of the embodiments of the present application has an object information extraction program stored thereon, and the object information extraction program is executed by a processor to realize the above-mentioned object information extraction method.

[0109] According to the computer readable storage medium of the embodiments of the present application, based on the above-mentioned object information extraction method, the problem of being unable to extract object information is effectively solved, and the accuracy of object information extraction is improved.

[0110] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description, either functionally or chronologically, as well as changes being made concerning which elements of the description and / or examples are employed per se, all without departing from the spirit and scope of the application. It should be further appreciated that the logic and / or steps represented in the flow diagrams and / or otherwise described herein, for example, can be considered as a sequence of executable instructions executed by a logic processor, such as a processing system, including a processor, or other logic processor-based system, or in conjunction with such an instruction execution system. In this regard, the "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can comprise any one of the following: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0111] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or combinations thereof, can be used with the necessary hardware: a discrete logic circuit(s) having logic gates for implementing logic functions upon an application of data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), and / or the like.

[0112] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Descriptive expressions of the above terms in the present specification do not necessarily refer to the same embodiment or example. Also, the specific feature, structure, material or characteristic described can be combined in any one or more embodiments or examples in an appropriate manner.

[0113] In addition, the terms "first", "second", etc. are used only for the purpose of description, and should not be understood as indicating or implying relative importance or implying a number of the technical features indicated. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified and limited.

[0114] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0115] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for extracting object information, characterized in that, The method includes: Acquire environmental point cloud data corresponding to the target environment, and object semantic information corresponding to the environmental point cloud data. The target environment includes at least one object to be extracted, wherein the object semantic information is the object's attribute information. Based on the environmental point cloud data, a three-dimensional reconstruction is performed to obtain multiple triangular facets corresponding to the target environment; The triangular facets are clustered according to the semantic information of the objects, and the object information corresponding to the object to be extracted is extracted from the clustered triangular facets. The step of clustering the triangular facets based on the semantic information of the object, and extracting the object information corresponding to the object to be extracted from the clustered triangular facets, includes: Obtain object category information from the semantic information of the object; The triangular facets are clustered according to the object category information to obtain facet sets corresponding to each of the multiple object categories. Connectivity clustering is performed on each of the multiple facet sets to obtain the object information corresponding to the object to be extracted.

2. The object information extraction method according to claim 1, characterized in that, The process of performing connectivity clustering on multiple sets of patches to obtain object information corresponding to the object to be extracted includes: Using the triangular facets as nodes and the lines connecting nodes with shared edges as edges, construct the undirected graphs corresponding to each set of facets; Perform a graph search on the undirected graph to obtain connected triangular faces in the face set; Based on the connected triangular facets, the object information corresponding to the object to be extracted is obtained.

3. The object information extraction method according to claim 2, characterized in that, After performing a graph search on the undirected graph to obtain the connected triangular faces in the face set, the method further includes: Perform distance clustering on connected and disconnected triangular faces; The object information corresponding to the object to be extracted is obtained from the triangular facets after distance clustering.

4. The object information extraction method according to any one of claims 1-3, characterized in that, Before performing 3D reconstruction based on the environmental point cloud data to obtain multiple triangular facets corresponding to the target environment, the method further includes: Point cloud extraction is performed on the environmental point cloud data based on the semantic information of the object; When the object point cloud data is extracted, the environmental point cloud data other than the object point cloud data is marked as the point cloud data to be reconstructed. Based on the point cloud data to be reconstructed, a three-dimensional reconstruction is performed to obtain multiple triangular facets.

5. The object information extraction method according to claim 1, characterized in that, The 3D reconstruction based on the environmental point cloud data yields multiple triangular facets corresponding to the target environment, including: The environmental point cloud data is divided into voxels to obtain multiple voxels; Determine the isosurface and the intersection points of the isosurface and the voxel; Based on the relative positions of the vertices of the voxels and the isosurfaces, the intersection points of the edges are connected to obtain multiple triangular facets.

6. The object information extraction method according to claim 1, characterized in that, After acquiring the environmental point cloud data corresponding to the target environment, the method further includes: The environmental point cloud data is divided into voxels to obtain multiple voxels; Obtain the symbolic distance corresponding to the voxel; The environmental point cloud data is fused based on the symbolic distance to obtain a surface point cloud; The surface point cloud is reconstructed in three dimensions to obtain multiple triangular facets.

7. An object information extraction device, characterized in that, The apparatus for implementing the object information extraction method according to any one of claims 1-6 includes: The data acquisition module is used to acquire environmental point cloud data corresponding to the target environment and object semantic information corresponding to the environmental point cloud data, wherein the target environment includes at least one object to be extracted; The reconstruction module is used to perform three-dimensional reconstruction of the environmental point cloud data to obtain multiple triangular facets. The information extraction module is used to cluster the triangular facets according to the semantic information of the object, and extract the object information corresponding to the object to be extracted from the clustered triangular facets.

8. A robot, characterized in that, include: The device includes a memory, a processor, and an object information extraction program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the object information extraction method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores an object information extraction program, which, when executed by a processor, implements the object information extraction method according to any one of claims 1-6.

Images