A method for reconstructing 3D information of objects based on embodied perception of humanoid robots

Abstract

This invention relates to a method for reconstructing 3D information of objects based on embodied perception of a humanoid robot, comprising: Step 1, acquiring a single-view depth image captured by the humanoid robot, performing preprocessing to obtain a single-view 3D point cloud of the object, which serves as the initial object surface point cloud; Step 2, inputting the initial object surface point cloud into a vision-touch fusion object reconstruction network to obtain an initial object shape estimation result; Step 3, employing a dual-arm active tactile exploration strategy to actively explore the object, obtaining tactile data and a further refined object surface point cloud, adding the further refined object surface point cloud to the initial object shape estimation result, and inputting it together with the tactile data into the vision-touch fusion object reconstruction network for further object reconstruction to obtain a further refined object shape estimation result, repeating Step 3 until reconstruction is complete. Compared with existing technologies, this invention has advantages such as achieving refined shape reconstruction of objects in complex scenes under single-view robot conditions.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and robotics, and in particular to a method for reconstructing three-dimensional information of objects based on embodied perception of humanoid robots. Background Technology

[0002] In unknown and unstructured environments, robots build and refine their understanding of the world through a series of perceptual means. Vision is one of the primary ways robots acquire information about objects. Using depth cameras, robots can acquire 3D point clouds of objects or environments, thus establishing their relevant representations. Accurately estimating and representing the 3D shape of an unknown object or environment is one of the challenges in many engineering applications. However, the accuracy of object reconstruction using only vision is insufficient, especially in critical areas such as the handle of a cup. Utilizing tactile perception in conjunction with visual methods to complete object reconstruction is an effective solution to improve the accuracy of object reconstruction.

[0003] Furthermore, previous point cloud completion methods mostly focused on using vision for object reconstruction, or further refining it using a single-arm robot with tactile sensors. Patent CN115937043B discloses a tactile-assisted point cloud completion method. This method uses a robotic arm connected to an electric gripper and a DIGIT tactile sensor to select missing areas of the object's point cloud for touch, acquiring tactile images and pose information of the touched areas. The tactile images are converted into preliminary tactile point clouds, which are then transformed from the world coordinate system to the target feature coordinate system and stitched together with the missing object point cloud. This method only utilizes the surface point cloud acquired through touch, failing to fully extract the rich information contained in the tactile data captured when the tactile sensor contacts the object. Moreover, the tactile sensor's role in this method is merely to acquire the spatial position of the object's surface, without achieving further fusion of vision and touch. Additionally, for larger objects, single-arm exploration efficiency is limited, and collisions and other risks may occur during exploration, leading to low object reconstruction efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a method for reconstructing three-dimensional information of objects based on embodied perception of humanoid robots to improve the accuracy of object reconstruction from a single perspective.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for reconstructing 3D information of objects based on embodied perception of humanoid robots includes the following steps:

[0007] Step 1: Acquire a single-view depth image captured by the humanoid robot, perform preprocessing to obtain a single-view 3D point cloud of the object, which serves as the initial point cloud of the object's surface.

[0008] Step 2: Input the initial object surface point cloud into the visual-touch fusion object reconstruction network to obtain the initial object shape estimation result;

[0009] Step 3: Based on the initial object shape estimation result, the object is actively explored using a dual-arm active tactile exploration strategy to obtain tactile data and a further refined object surface point cloud. The further refined object surface point cloud is added to the initial object shape estimation result and input together with the tactile data into the visual-tactile fusion object reconstruction network in Step 2 for further object reconstruction to obtain a further refined object shape estimation result. Step 3 is repeated until the reconstruction is completed.

[0010] Furthermore, the preprocessing steps include:

[0011] The depth image is sequentially subjected to coordinate transformation, object segmentation, and downsampling to obtain a single-view 3D point cloud of the object, where each point is represented as:

[0012] p i =(x,y,z)

[0013] In the formula, p i Let x, y, and z be the three-dimensional coordinates of the i-th point.

[0014] Furthermore, the visual-touch fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module, and a self-attention-based decoder.

[0015] Further, in step 2, the step of obtaining the initial object shape estimation result includes:

[0016] The initial object surface point cloud is input into the point cloud channel encoder to extract point cloud features;

[0017] The point cloud features are input into the self-attention-based decoder for decoding to obtain the initial object shape estimation result.

[0018] Further, in step 3, the execution steps of the view-touch fusion object reconstruction network include:

[0019] The further refined object surface point cloud is input into the point cloud channel encoder to extract point cloud features;

[0020] The tactile data is input into the tactile channel encoder to extract tactile features;

[0021] The point cloud features and tactile features are concatenated and input into the fusion module to obtain point-tactile fusion features;

[0022] The point-tactile fusion features are input into the self-attention-based decoder for decoding to obtain further refined object shape estimation results.

[0023] Furthermore, the point cloud channel encoder and the haptic channel encoder have the same network structure, both including convolutional layers and max pooling layers, wherein the formula for the convolutional layer is:

[0024] Output = Cov(Input)

[0025] Cov(n) = ∑x(k) × h(nk)

[0026] In the formula, Output is the output of the convolutional network, Cov(·) is the convolution operation of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after processing the convolution kernel sequence.

[0027] The formula for the maximum pooling layer is:

[0028] F pool =Pool max (F)

[0029] In the formula, F pool For the output of the max pooling layer, Pool max (·) represents the max pooling operation, where F is the input to the max pooling layer and the output of the last convolutional layer.

[0030] Furthermore, the self-attention-based decoder includes a self-attention layer and a convolutional layer, wherein the self-attention layer is represented as follows:

[0031]

[0032] In the formula, Attention(Q,K,V) represents the self-attention value, Q, K, and V are the query, key, and value, respectively, softmax is the normalized exponential function, and K... T It is the transpose of K. x is the length of the sequence. i is the i-th element of the input vector. The softmax function converts each element of the input vector into its probability value. N is the total number of input vectors. x for e i Power of 1 x for e k power, x k It is the i-th element of the input vector.

[0033] Furthermore, the steps of actively exploring objects using a dual-arm active tactile exploration strategy include:

[0034] Based on the object shape estimation results, calculate the distance between the two points on the object surface that are farthest apart, and compare it with the set capability risk factor δ. If the farthest distance is greater than δ, determine to use the dual-arm mode; otherwise, determine to use the single-arm mode.

[0035] Based on the established pattern, active exploration is conducted under the tactile exploration design principles to obtain the exploration location and generate tactile data;

[0036] Based on the tactile data, a position calibration network is used to obtain a further refined point cloud of the object surface.

[0037] Furthermore, the design criteria include curvature criteria and safety criteria, wherein the curvature criteria are expressed as:

[0038]

[0039]

[0040] In the formula, U1(·) is the curvature criterion, p i p j Let i and j be points, and λ be K. cur eigenvalues, K cur The curvature covariance matrix is ​​calculated based on the n neighboring point clouds of the current point cloud. In single-arm mode, the curvature of only one point is calculated.

[0041] The safety guidelines are expressed as follows:

[0042] U2(p i ,p j )=d(p i ,p j )+(z i +z j )

[0043] In the formula, U2(·) is the safety criterion, p i p j Let z be points i and j, d(·) be the first term, representing the distance between the two points to be explored in the two-arm mode, and z be the distance between the two points to be explored. i +z j The second term represents the distance from the ground, z. i z j are the ground heights of points i and j, respectively. In single-arm mode, only the second term is calculated.

[0044] Furthermore, the step of obtaining the explored location includes:

[0045] The curvature criterion and the safety criterion are weighted and summed, and the maximum value is used as the exploration position. The expression for the weighted sum is:

[0046] U = α1U1 + α2U2

[0047] In the formula, U represents the dual-arm active tactile exploration strategy, α1 and α2 are the weighting coefficients of U1 and U2 respectively, and U1 and U2 are the curvature criterion and safety criterion respectively.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The visual-touch fusion object reconstruction network of the present invention first performs initial object estimation on the single-view point cloud, and then actively explores through the dual-arm active tactile exploration strategy to obtain tactile data and refined point cloud data. Then, the point cloud is merged and input together with the tactile data into the visual-touch fusion object reconstruction network for reconstruction to obtain refined object estimation. Through continuous exploration and reconstruction, the accurate construction of objects under single view is achieved.

[0050] (2) Point cloud data contains object surface information, and tactile data obtains object features containing touch points through active exploration. The object reconstruction network of the present invention includes point cloud channels and tactile channels, which can respectively realize their respective feature extraction and obtain object category features. This is beneficial for extracting common features and unique features of similar objects, thereby improving reconstruction accuracy.

[0051] (3) The dual-arm active tactile exploration strategy of the present invention takes into account the size of the object and its own capabilities. By calculating the distance between the two points farthest from the object surface, it adaptively selects the single-arm mode or the dual-arm mode. The design criteria include curvature criteria and safety criteria. The curvature criteria enable the robot to focus on exploring the parts of the object with high curvature, such as the edges and corners of the object. The safety criteria represent the collision risk to the ground and the collision risk of the two arms, which can reduce the probability of the robot colliding. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0053] Figure 2 This is a schematic flowchart of the object shape reconstruction method of the present invention. Detailed Implementation

[0054] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0055] This embodiment provides a method for reconstructing the three-dimensional information of an object based on the embodied perception of a humanoid robot, such as... Figure 2As shown, this method utilizes a depth camera and a tactile sensor, and preprocesses to obtain 3D point cloud and tactile data of the object surface. It comprises two parts: a visual-tactile fusion object reconstruction network and a dual-arm active exploration strategy. The visual-tactile fusion object reconstruction network includes two channels: point cloud and tactile. The visual channel encodes the point cloud data acquired by the depth camera to obtain point cloud features; the tactile channel encodes the tactile sensor data to obtain tactile features. Additionally, the tactile data is processed by a calibration network to obtain the surface point cloud of the object. The visual-tactile fusion object reconstruction network uses a convolutional neural network to process the two types of features to obtain point-tactile fusion features. Finally, a self-attention-based decoder is used to generate multi-scale point clouds, obtaining the object shape estimation result. Based on the currently estimated object shape, a humanoid dual-arm robot is used to actively explore the object using tactile feedback, further refining the object shape. Active tactile exploration primarily involves tactile contact with parts of an object not captured by visual perception. The dual-arm active tactile exploration strategy adaptively selects between single-arm and dual-arm modes based on the object's size. Active exploration further acquires surface point cloud and tactile contact data, and a reconstruction network yields more accurate object shape estimation results. This invention fully utilizes single-viewpoint point cloud information and tactile information; it employs a visual-tactile fusion reconstruction network and a dual-arm active exploration strategy to achieve refined object shape reconstruction. Specifically, combined with... Figure 1 and Figure 2 The method includes the following steps:

[0056] S1: The depth camera acquires visual information of the target object from a single view angle.

[0057] Depth images of the scene where the robot is located are acquired by a camera, and a single-view 3D point cloud P of the object is obtained through preprocessing, which serves as the point cloud of the object's surface.

[0058] The preprocessing in step S1 involves performing coordinate transformation, object segmentation, and downsampling on the depth image acquired by the camera to obtain the required spatial information of the object, i.e., the true location of the point cloud in the real world. The single-view 3D point cloud P of the object contains the positions of 2048 points, each represented as: p i = (x, y, z), where x, y, and z are the three-dimensional coordinates in space. In particular, in the initial stage, P contains only point clouds of object surfaces collected from a single visual perspective. After subsequent active tactile exploration, P contains both visual and tactile point clouds.

[0059] S2: Use the view-touch fusion object reconstruction network to perform preliminary object reconstruction (point cloud single channel).

[0060] The visual-tactile fusion object reconstruction network in this step contains two channels: a point cloud channel and a tactile channel. The point cloud channel contains object surface information, while the tactile data contains object features at the contact points.

[0061] Initially, the point cloud channel contains only visual point cloud data. After tactile exploration, the point cloud channel includes both visual and tactile point clouds. The tactile channel is beneficial for extracting features of the contact area. After extraction, object category features can be obtained, which is helpful for extracting common and unique features of similar objects, thereby improving reconstruction accuracy.

[0062] The visual-touch fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module, and a self-attention-based decoder. In the initial stage, point cloud features are extracted from the point cloud data by the point cloud channel encoder and then decoded by the decoder to obtain initial object shape estimation results.

[0063] In the reconstruction process at other stages, the point cloud channel encodes the point cloud data acquired by the depth camera to obtain point cloud features. The point cloud channel encoder consists of five convolutional layers and one max pooling layer. The tactile channel encodes the tactile sensor data to obtain tactile features. The tactile channel encoder structure is consistent with that of the point cloud channel encoder. After the point cloud features and tactile features are concatenated, they are processed by a fusion module (two convolutional neural networks) to obtain point-tactile fusion features. Finally, a self-attention-based decoder is used to generate multi-scale point clouds and obtain the object shape estimation results.

[0064] The encoder in the above-mentioned visual-touch fusion object reconstruction network consists of five convolutional layers. The outputs of the last four convolutional layers are concatenated after passing through a max pooling layer to obtain the encoded point cloud features.

[0065] The formula for a convolutional layer is as follows:

[0066] Output = Cov(Input) (1)

[0067] Cov(n)=Σx(k)×h(nk) (2)

[0068] Where Ouput is the output of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after processing the convolution kernel sequence; the input of the first convolutional network is point cloud data or voxel data, and the inputs of the following five layers are the outputs of the previous convolutional network.

[0069] The formula for the max pooling layer is as follows:

[0070] F pool =Pool max (F) (3)

[0071] Poolmax is the max pooling layer, and the input to this pooling layer is the output of the last four convolutional layers.

[0072] The self-attention-based decoder described above consists of one self-attention layer and three convolutional layers. The self-attention mechanism is the core of the Transformer and can be represented as follows:

[0073]

[0074] Where Q, K, and V are the query, key, and value, respectively, softmax is the normalization exponential function, and K... T It is the transpose of K. x is the length of the sequence. i It is the i-th element of the input vector. The softmax function converts each element in the input vector into its probability value.

[0075] S3: Active tactile exploration with both arms to acquire tactile data.

[0076] In step S3, the active tactile exploration first adaptively determines whether to use single-arm or dual-arm mode based on the object's size and the user's own capabilities. This determination is made using a capability risk factor δ, which represents the ease of exploring the object using dual arms and is an empirical value. After estimating the object's shape, the distance between the two farthest point clouds on the object's surface is calculated. If the distance is greater than the capability risk factor δ, dual-arm mode is used; otherwise, single-arm mode is used.

[0077] Secondly, the design principles for the tactile active exploration strategy are designed. The design principles include curvature principle and safety principle. The curvature principle means that the robot focuses on exploring the parts of the object with high curvature, such as the edges and corners of the object. The safety principle includes the collision risk to the ground and the collision risk to both arms.

[0078] The curvature criterion is expressed as:

[0079]

[0080] λ is K cur eigenvalues, K cur It is the curvature covariance matrix; the curvature covariance can be calculated based on the n nearest points in the current point cloud. In single-arm mode, the curvature of only one point is calculated.

[0081] Safety guidelines can be expressed as:

[0082] U2(p i ,p j )=d(p i ,p j )+(z i +z j (8)

[0083] The first term represents the distance between the two points to be explored in dual-arm mode, and the second term represents the distance from the ground. In single-arm mode, only the second term is calculated.

[0084] The final active dual-arm exploration strategy is a weighted sum of two guaranteed outcomes:

[0085] U = α1U1 + α2U2 (9)

[0086] The corresponding exploration location is obtained by solving for the maximum value of the above formula.

[0087] Forward exploration involves making contact with the exploration location, generating tactile data.

[0088] S4. Update object data by reconstructing the object and updating its shape.

[0089] Based on the tactile data, a location calibration network (which can be a fully connected network) is used for processing to obtain refined point cloud data.

[0090] In step S4, updating the object data involves adding the point cloud obtained from tactile sensing to the multi-scale point cloud and using the tactile data as the input to the tactile channel of the reconstruction network to obtain a new shape reconstruction result. Through continuous active exploration and reconstruction of the network, a more refined 3D object shape is finally obtained.

[0091] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0093] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0097] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot, characterized in that, Includes the following steps: Step 1: Acquire a single-view depth image captured by the humanoid robot, perform preprocessing to obtain a single-view 3D point cloud of the object, which serves as the initial point cloud of the object's surface. Step 2: Input the initial object surface point cloud into the visual-touch fusion object reconstruction network to obtain the initial object shape estimation result; Step 3: Based on the initial object shape estimation result, the object is actively explored using a dual-arm active tactile exploration strategy to obtain tactile data and a further refined object surface point cloud. The further refined object surface point cloud is added to the initial object shape estimation result and input together with the tactile data into the visual-tactile fusion object reconstruction network in Step 2 for further object reconstruction to obtain a further refined object shape estimation result. Step 3 is repeated until the reconstruction is completed. The vision-touch fusion object reconstruction network includes a point cloud channel encoder, a tactile channel encoder, a fusion module, and a self-attention-based decoder. The point cloud channel encoder is used to extract point cloud features, the tactile channel encoder is used to extract tactile features, the fusion module is used to fuse the point cloud features and tactile features to obtain point-tactile fusion features, and the self-attention-based decoder is used to decode the point-tactile fusion features to achieve object shape estimation. Step 2, the step of obtaining the initial object shape estimation result includes: The initial object surface point cloud is input into the point cloud channel encoder to extract point cloud features; The point cloud features are input into the self-attention-based decoder for decoding to obtain the initial object shape estimation result; In step 3, the execution steps of the visual-touch fusion object reconstruction network include: The further refined object surface point cloud is input into the point cloud channel encoder to extract point cloud features; The tactile data is input into the tactile channel encoder to extract tactile features; The point cloud features and tactile features are concatenated and input into the fusion module to obtain point-tactile fusion features; The point-tactile fusion features are input into the self-attention-based decoder for decoding to obtain a further refined object shape estimation result; The steps for actively exploring objects using a dual-arm active tactile exploration strategy include: Based on the object shape estimation results, calculate the distance between the two points on the object surface that are farthest apart, and compare it with the set capability risk factor δ. If the farthest distance is greater than δ, determine to use the dual-arm mode; otherwise, determine to use the single-arm mode. Based on the established pattern, active exploration is conducted under the tactile exploration design principles to obtain the exploration location and generate tactile data; Based on the tactile data, a position calibration network is used to obtain a further refined point cloud of the object surface. The design criteria include curvature criteria and safety criteria, wherein the curvature criteria are expressed as follows: In the formula, U1(·) is the curvature criterion, p i p j Let i and j be points, and λ be K. cur eigenvalues, K cur The curvature covariance matrix is ​​calculated based on the n neighboring point clouds of the current point cloud. In single-arm mode, the curvature of only one point is calculated. The safety guidelines are expressed as follows: U2(p i ,p j )=d(p i ,p j )+(z i +z j ) In the formula, U2(·) is the safety criterion, p i p j Let z be points i and j, d(·) be the first term, representing the distance between the two points to be explored in the two-arm mode, and z be the distance between the two points to be explored. i +z j The second term represents the distance from the ground, z. i z j are the ground heights of points i and j, respectively. In single-arm mode, only the second term is calculated.

2. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that, The preprocessing steps include: The depth image is sequentially subjected to coordinate transformation, object segmentation, and downsampling to obtain a single-view 3D point cloud of the object, where each point is represented as: p i =(x,y,z) In the formula, p i Let x, y, and z be the three-dimensional coordinates of the i-th point.

3. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that, The point cloud channel encoder and the tactile channel encoder have the same network structure, both including convolutional layers and max pooling layers, wherein the formula for the convolutional layer is: Output = Cov(Input) Cov(n) = ∑x(k) × h(nk) In the formula, Output is the output of the convolutional network, Cov(·) is the convolution operation of the convolutional network, Input is the input of the convolutional network, Cov(n) is the nth element of the convolution result, x(k) is the kth element of the input sequence, and h(nk) is the nkth element after processing the convolution kernel sequence. The formula for the maximum pooling layer is: F pool =Pool max (F) In the formula, F pool For the output of the max pooling layer, Pool max (·) represents the max pooling operation, where F is the input to the max pooling layer and the output of the last convolutional layer.

4. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that, The self-attention-based decoder includes a self-attention layer and a convolutional layer, wherein the self-attention layer is represented as follows: In the formula, Attention(Q,K,V) represents the self-attention value, Q, K, and V are the query, key, and value, respectively, softmax is the normalized exponential function, and K... T It is the transpose of K. x is the length of the sequence. i It is the i-th element of the input vector. The softmax function converts each element of the input vector into its probability value. N is the total number of input vectors, and e is the probability value. xi x for e i The power, e xk x for e k power, x k It is the i-th element of the input vector.

5. The method for reconstructing three-dimensional information of an object based on embodied perception of a humanoid robot according to claim 1, characterized in that, The steps for obtaining the explored location include: The curvature criterion and the safety criterion are weighted and summed, and the maximum value is used as the exploration position. The expression for the weighted sum is: U = α1U1 + α2U2 In the formula, U represents the dual-arm active tactile exploration strategy, α1 and α2 are the weighting coefficients of U1 and U2 respectively, and U1 and U2 are the curvature criterion and safety criterion respectively.

Images