Object three-dimensional information reconstruction method with body perception based on humanoid robot

Abstract

The invention relates to an object three-dimensional information reconstruction method based on humanoid robot body perception, and the method comprises the steps: 1, obtaining a single-visual-angle depth image collected by a humanoid robot body, carrying out the preprocessing, and obtaining a single-visual-angle three-dimensional point cloud of an object, and taking the single-visual-angle three-dimensional point cloud as an initial object surface point cloud; step 2, inputting the initial object surface point cloud into the visual touch fusion object reconstruction network to obtain an initial object shape estimation result; and step 3, performing active exploration on the object by adopting a double-arm active tactile exploration strategy to obtain tactile data and further refined object surface point cloud, adding the further refined object surface point cloud into an initial object shape estimation result, and jointly inputting the further refined object surface point cloud and the tactile data into a visual touch fusion object reconstruction network to perform further object reconstruction. And obtaining a further refined object shape estimation result, and repeating the step 3 until the reconstruction is finished. Compared with the prior art, the method has the advantages of realizing refined shape reconstruction of the object under the condition of single view angle of the robot in a complex scene and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and robotics technology, and in particular to a method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot. Background Art

[0002] In unknown and unstructured environments, robots use a range of perception methods to build and improve their understanding of the world. Vision is one of the main ways for robots to obtain information about objects. Through a depth camera, robots can obtain a 3D point cloud of an object or environment to build their relevant representations. Accurately estimating and characterizing the 3D shape of an unknown object or environment is one of the challenges in many engineering applications. However, the accuracy of object reconstruction using vision alone is not enough, especially in some key areas, such as the handle of a cup. Using tactile perception combined with visual methods to complete object reconstruction is an effective solution to improve the accuracy of object reconstruction.

[0003] In addition, most of the previous point cloud completion methods focused on using vision to reconstruct objects, or on this basis, using a single-arm robot with a tactile sensor for further refinement. Patent CN115937043B discloses a method for tactile-assisted point cloud completion, which uses a robotic arm connected to an electric gripper and a DIGIT tactile sensor to select the missing area of ​​the object point cloud for touch, and obtain the tactile image and posture information of the touch area; the tactile image is converted into a preliminary tactile point cloud, and then the preliminary tactile point cloud is converted from the world coordinate system to the target feature coordinate system, and spliced ​​with the missing point cloud of the object. This method only uses the surface point cloud obtained by touch, and does not fully extract the rich information contained in the tactile data captured when the tactile sensor contacts the object. In addition, the role of the tactile sensor in this method is only to obtain the spatial position of the surface of the object, and no further integration of vision and touch is achieved. In addition, for larger shapes of objects, the efficiency of single-arm exploration is limited, and risks such as collisions may occur during the exploration process, making the object reconstruction inefficient. Summary of the invention

[0004] The purpose of the present invention is to provide a method for reconstructing three-dimensional information of an object based on the embodied perception of a humanoid robot, which improves the accuracy of object reconstruction under a single perspective.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot comprises the following steps:

[0007] Step 1: Obtain a single-view depth image collected by the humanoid robot, perform preprocessing, and obtain a single-view three-dimensional point cloud of the object as the initial object surface point cloud;

[0008] Step 2: Input the initial object surface point cloud into the visual-tactile fusion object reconstruction network to obtain the initial object shape estimation result;

[0009] Step 3: Based on the initial object shape estimation result, a dual-arm active tactile exploration strategy is used to actively explore the object to obtain tactile data and a further refined object surface point cloud, and the further refined object surface point cloud is added to the initial object shape estimation result, and is input together with the tactile data into the visual-tactile fusion object reconstruction network in step 2 for further object reconstruction to obtain a further refined object shape estimation result, and step 3 is repeated until the reconstruction is completed.

[0010] Furthermore, the pre-processing step includes:

[0011] The depth image is subjected to coordinate transformation, object segmentation and downsampling processing in sequence to obtain a single-view three-dimensional point cloud of the object, where each point is represented as:

[0012] p i =(x,y,z)

[0013] In the formula, p i is the i-th point, x, y, z are the three-dimensional coordinates in space.

[0014] Furthermore, the visual-tactile fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module and a self-attention-based decoder.

[0015] Furthermore, in step 2, the step of obtaining the initial object shape estimation result includes:

[0016] Inputting the initial object surface point cloud into a point cloud channel encoder to extract point cloud features;

[0017] The point cloud features are input into the self-attention based decoder for decoding to obtain an initial object shape estimation result.

[0018] Furthermore, in step 3, the execution steps of the visual-tactile fusion object reconstruction network include:

[0019] Inputting the further refined object surface point cloud into a point cloud channel encoder to extract point cloud features;

[0020] Inputting the tactile data into a tactile channel encoder to extract tactile features;

[0021] The point cloud features and the tactile features are spliced ​​and input into the fusion module to obtain point-tactile fusion features;

[0022] The point-tactile fusion feature is input into the self-attention based decoder for decoding to obtain a further refined object shape estimation result.

[0023] Furthermore, the point cloud channel encoder and the tactile channel encoder have the same network structure, both including a convolution layer and a maximum pooling layer, wherein the formula of the convolution layer is:

[0024] Output = Cov(Input)

[0025] Cov(n)=∑x(k)×h(nk)

[0026] Where Output is the output of the convolutional network, Cov(·) is the convolution operation of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after the convolution kernel sequence is processed;

[0027] The maximum pooling layer formula is:

[0028] F pool =Pool max (F)

[0029] In the formula, F pool is the maximum pooling layer output, Pool max (·) is the maximum pooling operation, F is the input of the maximum pooling layer and the output of the last convolutional layer.

[0030] Furthermore, the self-attention based decoder includes a self-attention layer and a convolutional layer, and the self-attention layer is expressed as:

[0031]

[0032] Where Attention(Q,K,V) is the self-attention value, Q, K, V are query, key, and value respectively, softmax is the normalized exponential function, K T is the transpose of K, is the length of the sequence, x i is the i-th element of the input vector. The role of the softmax function is to convert each element in the input vector into its form as a probability value. N is the total number of input vectors. x for e i Second power, x for e k Power, x k is the i-th element of the input vector.

[0033] Furthermore, the step of actively exploring the object using a dual-arm active tactile exploration strategy includes:

[0034] According to the object shape estimation result, the distance between the two farthest points on the object surface is calculated and compared with the set capacity risk factor δ. If the farthest distance is greater than δ, the double-arm mode is determined to be adopted, otherwise the single-arm mode is determined to be adopted;

[0035] According to the determined pattern, active exploration is performed under the tactile exploration design criteria to obtain the exploration position and generate tactile data;

[0036] Based on the tactile data, a position calibration network is used to obtain a further refined point cloud of the object surface.

[0037] Furthermore, the design criteria include a curvature criterion and a safety criterion, and the curvature criterion is expressed as:

[0038]

[0039]

[0040] Where U1(·) is the curvature criterion, p i 、p j are points i and j, and λ is K cur The eigenvalue of K cur is the curvature covariance matrix, where the curvature covariance can be calculated based on the n point clouds adjacent to the current point cloud. In single-arm mode, only the curvature of one point is calculated;

[0041] The safety criteria are expressed as:

[0042] U2(p i ,p j )=d(p i ,p j )+(z i +z j )

[0043] Where U2(·) is the safety criterion, p i 、p j are points i and j, d(·) is the first term, which represents the distance between the two points to be explored in the dual-arm mode, and z i +z j is the second term, indicating the distance from the ground, z i 、z j are the heights of points i and j above the ground respectively. In single-arm mode, only the second item is calculated.

[0044] Furthermore, the step of obtaining the exploration location includes:

[0045] The curvature criterion and the safety criterion are weighted and added together, and the maximum value is solved as the exploration position, where the weighted addition expression is:

[0046] U=α1U1+α2U2

[0047] Where U is the active tactile exploration strategy of both arms, α1 and α2 are the weighted coefficients of U1 and U2, respectively, and U1 and U2 are the curvature criterion and safety criterion, respectively.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The vision-tactile fusion object reconstruction network of the present invention first performs an initial object estimation on the single-view point cloud, and then performs active exploration through a dual-arm active tactile exploration strategy to obtain tactile data and refined point cloud data. The point cloud data is then merged and input together with the tactile data into the vision-tactile fusion object reconstruction network for further reconstruction to obtain a refined object estimation. Through continuous exploration and reconstruction, the precise construction of objects in a single view is achieved.

[0050] (2) Point cloud data contains object surface information, and tactile data contains object features at the touch site through active exploration. The visual-tactile fusion object reconstruction network of the present invention includes a point cloud channel and a tactile channel, which can respectively realize their respective feature extraction and obtain object category features, which is conducive to extracting common features and unique features of similar objects, thereby improving reconstruction accuracy.

[0051] (3) The dual-arm active tactile exploration strategy of the present invention takes into account the size of the object and the robot's own capabilities. By calculating the distance between the two farthest points on the surface of the object, the single-arm mode or the dual-arm mode is adaptively selected. The design criteria include curvature criteria and safety criteria. The curvature criteria enable the robot to focus on exploring the parts of the object with high curvature, that is, the edges and corners of the object. The safety criteria represent the risk of collision with the ground and the risk of collision with the dual arms, which can reduce the probability of collision of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0053] Figure 2 It is a schematic flow chart of the object shape reconstruction method of the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0055] This embodiment provides a method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot. Figure 2As shown in the figure, this method uses a depth camera and a tactile sensor, and preprocesses to obtain a three-dimensional point cloud and tactile data on the surface of the object. It includes two parts: a visual-tactile fusion object reconstruction network and a dual-arm active exploration strategy. The visual-tactile fusion object reconstruction network contains two channels, point cloud and tactile. The visual channel encodes the point cloud data obtained by the depth camera to obtain point cloud features; the tactile channel encodes the tactile sensor data to obtain tactile features; in addition, the tactile data can be calibrated through the network to obtain the surface point cloud of the contact object. The visual-tactile fusion object reconstruction network uses a convolutional neural network to process the two features to obtain point-tactile fusion features; finally, a self-attention-based decoder is used to realize multi-scale point cloud generation to obtain the object shape estimation result. Based on the currently estimated object shape, a humanoid dual-arm robot is used to perform tactile active exploration of the object to further refine the object shape. Active tactile exploration mainly involves tactile contact with the parts of the object that are not captured by vision. The dual-arm active tactile exploration strategy can adaptively select single-arm or dual-arm mode according to the size of the object. Through active exploration, surface point cloud and tactile contact data are further acquired, and a more accurate object shape estimation result is obtained through the reconstruction network. The present invention can make full use of single-view point cloud information and tactile information; use the visual-tactile fusion reconstruction network and the dual-arm active exploration strategy to complete the refined shape reconstruction of the object. Specifically, combined with Figure 1 and Figure 2 , the method comprises the following steps:

[0056] S1: The depth camera obtains the single-view visual information of the target object.

[0057] The depth image of the scene where the robot is located is obtained through the camera, and the single-view three-dimensional point cloud P of the object is obtained through preprocessing as the surface point cloud of the object.

[0058] The preprocessing in step S1 is to perform coordinate transformation, object segmentation, and downsampling on the depth image acquired by the camera to obtain the required spatial information of the object, that is, the actual position of the point cloud in the real world; the object single-view 3D point cloud P contains a total of 2048 points, each of which is represented by: p i =(x,y,z), where x, y, and z are the three-dimensional coordinates of the space respectively; in particular, in the initial stage, all the points contained in P are the surface point clouds of objects collected from a single visual perspective. After subsequent active tactile exploration, P contains both visual point clouds and tactile point clouds.

[0059] S2: Use the visual-touch fusion object reconstruction network to perform preliminary object reconstruction (single channel of point cloud).

[0060] The visual-tactile fusion object reconstruction network in this step contains two channels, namely the point cloud channel and the tactile channel. The point cloud channel contains the surface information of the object, and the tactile data contains the object features of the contact part.

[0061] The point cloud channel initially contains only visual point cloud data. After tactile exploration, the point cloud channel contains both visual and tactile point clouds. The tactile channel is conducive to extracting contact site signs. After extraction, object category features can be obtained, which is conducive to extracting common and unique features of similar objects, thereby improving reconstruction accuracy.

[0062] The visual-tactile fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module, and a decoder based on self-attention. In the initial stage, the point cloud data is extracted from the point cloud channel encoder and decoded by the decoder to obtain the initial object shape estimation result.

[0063] In the reconstruction process of other stages, the point cloud channel encodes the point cloud data obtained by the depth camera to obtain point cloud features. The point cloud channel encoder is composed of five convolutional layers and one maximum pooling layer. The tactile channel encodes the tactile sensor data to obtain tactile features. The structure of the tactile channel encoder is consistent with that of the point cloud channel encoder. After the point cloud features and tactile features are spliced, they are processed using a fusion module (two convolutional neural networks) to obtain point-tactile fusion features. Finally, a self-attention-based decoder is used to realize multi-scale point cloud generation and obtain the object shape estimation result.

[0064] The encoder in the above-mentioned visual-tactile fusion object reconstruction network is composed of five convolutional layers. The outputs of the last four convolutional layers are respectively concatenated through the maximum pooling layer to obtain the encoded point cloud features.

[0065] The formula for the convolutional layer is as follows:

[0066] Output=Cov(Input) (1)

[0067] Cov(n)=Σx(k)×h(nk) (2)

[0068] Where Ouput is the output of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after the convolution kernel sequence is processed; the input of the first layer of the convolutional network is point cloud data or voxel data, and the inputs of the next five layers are the outputs of the previous layer of the convolutional network;

[0069] The maximum pooling layer formula is as follows:

[0070] F pool =Pool max (F) (3)

[0071] Poolmax is the maximum pooling layer, and the input of the pooling layer here is the output of the last four convolution layers;

[0072] The above self-attention-based decoder consists of 1 self-attention layer and 3 convolutional layers. The self-attention mechanism is the core part of Transformer and can be expressed as:

[0073]

[0074] Among them, Q, K, V are query, key and value respectively, softmax is the normalized exponential function, K T is the transpose of K, is the length of the sequence, x i is the i-th element of the input vector. The role of the softmax function is to convert each element in the input vector into its form as a probability value.

[0075] S3: Active tactile exploration of both arms to obtain tactile data.

[0076] The tactile active exploration in step S3 can firstly adaptively determine whether to use a single-arm mode or a dual-arm mode according to the size of the object and its own capabilities; this mode is determined by setting a capability risk factor δ, which represents the convenience of using dual arms to explore the object, and is an empirical value. After the shape of the object is estimated, the distance between the two farthest point clouds on the surface of the object is calculated. If it is greater than the capability risk factor δ, the dual-arm mode is used; if it is less than the capability risk factor δ, the single-arm mode is used.

[0077] Secondly, the design criteria for the tactile active exploration strategy are designed. The design criteria include curvature criteria and safety criteria. The curvature criteria means that the robot focuses on exploring the parts of the object with high curvature, that is, the edges and corners of the object; the safety criteria include the risk of collision with the ground and the risk of collision with the arms.

[0078] The curvature criterion is expressed as:

[0079]

[0080] λ is K cur The eigenvalue of K cur is the curvature covariance matrix; the curvature covariance can be calculated based on the n point clouds adjacent to the current point cloud. In single-arm mode, only the curvature of one point is calculated.

[0081] The safety criterion can be expressed as:

[0082] U2(p i ,p j )=d(p i ,p j )+(z i +z j ) (8)

[0083] The first term represents the distance between the two points to be explored in the dual-arm mode, and the second term represents the distance from the ground. In the single-arm mode, only the second term is calculated.

[0084] The final active two-arm exploration strategy is the weighted sum of two guarantees:

[0085] U=α1U1+α2U2 (9)

[0086] The corresponding exploration position is obtained by solving the maximum value of the above formula.

[0087] The forward exploration makes contact with the exploration position and generates tactile data.

[0088] S4. Update the object data and update the object shape by reconstructing it.

[0089] According to the tactile data, a position calibration network (which may be a fully connected network) is used for processing to obtain refined point cloud data.

[0090] In step S4, the object data is updated by adding the point cloud obtained by tactile sensation to the multi-scale point cloud, and the tactile data is used as the tactile channel input of the reconstruction network to obtain a new shape reconstruction result. By continuously actively exploring and reconstructing the network, a more refined three-dimensional object shape is finally obtained.

[0091] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0092] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0093] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot, characterized in that: The following steps are involved: Step 1: Obtain a single-view depth image collected by the humanoid robot, perform preprocessing, and obtain a single-view three-dimensional point cloud of the object as the initial object surface point cloud; Step 2: Input the initial object surface point cloud into the visual-tactile fusion object reconstruction network to obtain the initial object shape estimation result; Step 3: Based on the initial object shape estimation result, a dual-arm active tactile exploration strategy is used to actively explore the object to obtain tactile data and a further refined object surface point cloud, and the further refined object surface point cloud is added to the initial object shape estimation result, and is input together with the tactile data into the visual-tactile fusion object reconstruction network in step 2 for further object reconstruction to obtain a further refined object shape estimation result, and step 3 is repeated until the reconstruction is completed.

2. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that: The pre-processing steps include: The depth image is subjected to coordinate transformation, object segmentation and downsampling processing in sequence to obtain a single-view three-dimensional point cloud of the object, where each point is represented as: p i =(x,y,z) In the formula, p i is the i-th point, x, y, z are the three-dimensional coordinates in space.

3. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that: The visual-tactile fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module and a self-attention-based decoder. The point cloud channel encoder is used to extract point cloud features, the tactile channel encoder is used to extract tactile features, the fusion module is used to fuse the point cloud features and tactile features to obtain point-tactile fusion features, and the self-attention-based decoder is used to decode the point-tactile fusion features to achieve object shape estimation.

4. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 3, characterized in that: In step 2, the step of obtaining the initial object shape estimation result includes: Inputting the initial object surface point cloud into a point cloud channel encoder to extract point cloud features; The point cloud features are input into the self-attention based decoder for decoding to obtain an initial object shape estimation result.

5. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 3, characterized in that: In step 3, the execution steps of the visual-tactile fusion object reconstruction network include: Inputting the further refined object surface point cloud into a point cloud channel encoder to extract point cloud features; Inputting the tactile data into a tactile channel encoder to extract tactile features; The point cloud features and the tactile features are spliced ​​and input into the fusion module to obtain point-tactile fusion features; The point-tactile fusion feature is input into the self-attention based decoder for decoding to obtain a further refined object shape estimation result.

6. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 3, characterized in that: The point cloud channel encoder and the tactile channel encoder have the same network structure, both including a convolution layer and a maximum pooling layer, wherein the formula of the convolution layer is: Output = Cov(Input) Cov(n)=∑x(k)×h(nk) Where Output is the output of the convolutional network, Cov(·) is the convolution operation of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after the convolution kernel sequence is processed; The maximum pooling layer formula is: F pool =Pool max (F) In the formula, F pool is the maximum pooling layer output, Pool max (·) is the maximum pooling operation, F is the input of the maximum pooling layer and the output of the last convolutional layer.

7. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 3, characterized in that: The self-attention based decoder includes a self-attention layer and a convolutional layer, and the self-attention layer is expressed as: Where Attention(Q,K,V) is the self-attention value, Q, K, V are query, key, and value respectively, softmax is the normalized exponential function, K T is the transpose of K, is the length of the sequence, x i is the i-th element of the input vector. The role of the softmax function is to convert each element in the input vector into its form as a probability value. N is the total number of input vectors. x for e i Second power, x for e k Power, x k is the i-th element of the input vector.

8. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that: The step of actively exploring the object using a dual-arm active tactile exploration strategy comprises: According to the object shape estimation result, the distance between the two farthest points on the object surface is calculated and compared with the set capacity risk factor δ. If the farthest distance is greater than δ, the double-arm mode is determined to be adopted, otherwise the single-arm mode is determined to be adopted; According to the determined pattern, active exploration is performed under the tactile exploration design criteria to obtain the exploration position and generate tactile data; Based on the tactile data, a position calibration network is used to obtain a further refined point cloud of the object surface.

9. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 8, characterized in that: The design criteria include a curvature criterion and a safety criterion. The curvature criterion is expressed as: Where U1(·) is the curvature criterion, p i 、p j are points i and j, and λ is K cur The eigenvalue of K cur is the curvature covariance matrix, where the curvature covariance can be calculated based on the n point clouds adjacent to the current point cloud. In single-arm mode, only the curvature of one point is calculated; The safety criteria are expressed as: U2(p i ,p j )=d(p i ,p j )+(z i +z j ) Where U2(·) is the safety criterion, p i 、p j are points i and j, d(·) is the first term, which represents the distance between the two points to be explored in the dual-arm mode, and z i +z j is the second term, indicating the distance from the ground, z i 、z j are the heights of points i and j above the ground respectively. In single-arm mode, only the second item is calculated.

10. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 9, characterized in that: The step of obtaining the exploration position comprises: The curvature criterion and the safety criterion are weighted and added together, and the maximum value is solved as the exploration position, where the weighted addition expression is: U=α1U1+α2U2 Where U is the active tactile exploration strategy of both arms, α1 and α2 are the weighted coefficients of U1 and U2, respectively, and U1 and U2 are the curvature criterion and safety criterion, respectively.

Images