Object property perception method, device and storage medium based on dexterous hand tactile

Abstract

The application provides an object attribute perception method and device based on dexterous hand touch and a storage medium, and belongs to the technical field of touch perception. The method comprises the following steps: controlling a dexterous hand provided with a three-dimensional force sensor array to perform initial light touch scanning on a target object, and acquiring preliminary profile information of the target object; determining a key region of the surface of the target object based on the preliminary profile information, wherein the key region comprises a high-curvature region and / or a flat region; determining a corresponding perception strategy according to the type of the key region; and controlling the dexterous hand to perform the corresponding perception strategy in the key region, and acquiring a physical attribute perception result. The application can adapt to object characteristics and collaboratively perceive multiple attributes of the object.

Description

Technical Field

[0001] This application relates to the field of tactile sensing technology, and in particular to a method, device and storage medium for object attribute sensing based on dexterity hand tactile sense. Background Technology

[0002] With the increasing application of embodied intelligent robot technology in fields such as industrial precision assembly, hazardous environment operations, and especially minimally invasive surgical robots, robots often need to manipulate target objects in unfamiliar environments with no vision or limited vision. In such scenarios, tactile perception has become a core alternative means for robots to understand the key attributes of objects and achieve precise and dexterous manipulation.

[0003] Currently, existing tactile sensing methods typically rely on pre-set fixed procedures to sequentially measure one or more properties of an object. For example, one possible approach is to first control a tactile sensor to traverse the object's surface along a preset trajectory to reconstruct its shape and contour, and then perform fixed pressing or sliding actions at specific points to measure hardness or roughness, etc. While this sequential measurement method can acquire some properties of an object to a certain extent, its dispersed sensing process leads to low sensing efficiency and a lack of overall coordination.

[0004] Therefore, there is an urgent need to propose a new method that can adapt to object features and collaboratively perceive multiple attributes of an object. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, and storage medium for object attribute perception based on dexterous hand tactile sense, in order to solve the above-mentioned problems.

[0006] To achieve the above objectives, firstly, this application proposes a method for object attribute perception based on dexterous hand tactile perception, the method comprising:

[0007] A dexterous hand equipped with a three-dimensional force sensor array is controlled to perform an initial light touch scan on the target object to obtain preliminary contour information of the target object;

[0008] Based on the preliminary contour information, key regions on the surface of the target object are determined, including high curvature regions and / or flat regions.

[0009] Based on the type of the key area, determine the corresponding perception strategy;

[0010] The dexterous hand is controlled to execute the corresponding perception strategy in the key area to obtain the physical attribute perception results.

[0011] In some implementations, determining the corresponding perception strategy based on the type of the key region includes:

[0012] When the type of the key area includes a high curvature area, the corresponding perception strategy is determined to be multi-angle touch, and the target object is touched according to the curvature along the angle with the largest curvature to refine the contour perception.

[0013] When the type of the key area includes a flat area, the corresponding perception strategy is determined to be to apply a fixed normal force to the target object to press it, so as to perceive hardness and / or smoothness.

[0014] In some implementations, the key region includes a high curvature region, the physical property perception result includes a shape contour model, and controlling the dexterous hand to execute a corresponding perception strategy in the key region to obtain the physical property perception result includes:

[0015] The dexterous hand is controlled to make multi-angle touches in the high curvature area and acquire real-time sensing data collected by the three-dimensional force sensor array;

[0016] Based on the real-time sensing data, the real-time force area, real-time force direction, and real-time force magnitude are determined, and the real-time force area, real-time force direction, and real-time force magnitude are used as input to a preset neural network model to obtain the real-time curvature output by the preset neural network model.

[0017] The angle with the largest real-time curvature during multi-angle touch is determined as the target angle, and the dexterous hand is controlled to perform contour-following touch on the target object based on the target angle;

[0018] Upon detecting a complete global contour touch, a shape contour model is constructed based on the movement trajectory of the dexterous hand.

[0019] In some implementations, the key region includes a flat area, the physical property perception result includes a hardness level, and controlling the dexterous hand to execute a corresponding perception strategy in the key region to obtain the physical property perception result includes:

[0020] The dexterous hand is controlled to apply a first normal force and press against the flat area;

[0021] When the force signal sensed by the three-dimensional force sensor array is stable, determine the first number of sensing units in the three-dimensional force sensor array that sensed the force signal;

[0022] The area of ​​the force sensor is calculated based on the single force-bearing area of ​​the sensing unit and the number of the first points.

[0023] The hardness level is determined based on the hardness sensing area and the preset hardness calibration function.

[0024] In some implementations, the key region includes a flat region, and the physical property perception result further includes a smoothness coefficient. After determining the hardness level based on the hardness-sensing force area and a preset hardness calibration function, the following is also included:

[0025] The dexterous hand is controlled to slide in the flat area according to a preset direction and a preset speed;

[0026] When the force signal sensed by the three-dimensional force sensor array is stable, the total tangential force sensed in the three-dimensional force sensor array is determined;

[0027] The smoothness coefficient is determined based on the first normal force, the hardness grade, and the total tangential force.

[0028] In some implementations, the key region includes a flat region, the physical property perception result includes a smoothness coefficient, and controlling the dexterous hand to execute a corresponding perception strategy in the key region to obtain the physical property perception result includes:

[0029] The dexterous hand is controlled to apply a second normal force to press on the flat area, and slide on the flat area in a preset direction and at a preset speed;

[0030] When the force signal sensed by the three-dimensional force sensor array is stable, determine the second number of sensing units in the three-dimensional force sensor array that sensed the force signal, and the sensed tangential resultant force;

[0031] The smoothness sensing force area is calculated based on the single force area of ​​the sensing unit and the second number of points;

[0032] The smoothness coefficient is determined based on the second normal force, the tangential resultant force, and the smoothness-sensing force area.

[0033] In some implementations, determining the smoothness coefficient based on the second normal force, the tangential resultant force, and the smoothness-sensing force area includes:

[0034] When the smoothness sensing force area is less than or equal to a preset area threshold, a rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force and the tangential resultant force.

[0035] When the smoothness sensing force area is greater than a preset area threshold, a non-rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force, the tangential resultant force and the preset calibration formula.

[0036] In some embodiments, the physical property sensing result includes the elastic modulus, and the method further includes:

[0037] Control the first and second fingertips of the dexterous hand to perform a tightening action. When the three-dimensional force sensor array of the first and second fingertips detects that either finger is in contact with the target object, the tightening action of the contacting finger is stopped until both the first and second fingertips are in contact with the target object. The first distance between the first and second fingertips is recorded.

[0038] Control the first and second fingertips to clamp the target object with a third normal force;

[0039] When the third normal force sensed by the three-dimensional force sensor array of the first and second fingertips is stable, the third number of sensing units in the three-dimensional force sensor array that sense the third normal force, and the second distance between the first and second fingertips are determined.

[0040] The elastic sensing force area is calculated based on the single force-bearing area of ​​the sensing unit and the number of the third points.

[0041] The elastic modulus is calculated based on the third normal force, the elastic sensing area, the first spacing, and the second spacing.

[0042] Secondly, to achieve the above objectives, this application also proposes an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the object attribute perception method based on dexterous hand tactile sense as described above.

[0043] Thirdly, to achieve the above objectives, this application also proposes a computer storage medium storing executable instructions, which, when executed by a processor, cause the processor to perform the object attribute perception method based on dexterous hand tactile sense as described above.

[0044] Compared with the prior art, the beneficial effects of this application include:

[0045] Compared to existing technologies that employ fixed, preset perception programs that cannot be adjusted based on the characteristics of the object itself, resulting in numerous redundant operations or insufficient perception of key areas on complex objects, this application identifies key areas by using preliminary contour information obtained from an initial touch scan. This enables the system to automatically identify high-curvature areas requiring focused perception and flat areas suitable for material perception, and execute differentiated perception strategies to collaboratively perceive multiple object attributes. This not only avoids unnecessary redundant operations but also optimizes the perception path and efficiency, and effectively improves the accuracy and reliability of the perception results. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0047] Figure 1 This is a flowchart illustrating an object attribute perception method based on dexterous hand tactile feedback in one embodiment.

[0048] Figure 2 This is a schematic diagram of the arrangement of a three-dimensional force sensor array in one embodiment;

[0049] Figure 3 This is a flowchart illustrating the process of perceiving the external contour model in one embodiment;

[0050] Figure 4 This is a schematic diagram of the process for sensing hardness levels in one embodiment;

[0051] Figure 5 This is a schematic diagram of a dexterous hand applying a first normal force to a flat area in one embodiment;

[0052] Figure 6 This is a schematic diagram of a dexterous hand pressing a flat area with different hardness levels with the same first normal force in one embodiment;

[0053] Figure 7 This is a flowchart illustrating the process of sensing the smoothness coefficient in one embodiment;

[0054] Figure 8 This is a schematic diagram of a dexterous hand applying a second normal force to a flat area and sliding in the flat area according to a preset direction and a preset speed in one embodiment.

[0055] Figure 9 This is a schematic diagram of the process for sensing the elastic modulus in one embodiment;

[0056] Figure 10 This is a schematic diagram illustrating the operation of controlling a dexterous hand to sense the elastic modulus in one embodiment.

[0057] Figure 11 This is a schematic diagram of the electronic device involved in the object attribute perception method based on dexterous hand touch in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0060] For example, the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For instance, without departing from the scope of this application, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. Both the first element and the second element are elements, but they are not the same element.

[0061] For example, the terms "comprising" or "including" used in this application indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0062] As mentioned earlier, existing tactile sensing methods typically rely on a pre-set fixed procedure to sequentially measure one or more attributes of an object. For example, one possible implementation is to first control the tactile sensor to traverse the object's surface along a preset trajectory to reconstruct its shape contour, and then perform fixed pressing or sliding actions at specific points to measure hardness or roughness, etc. While this sequential measurement method can acquire some attributes of the object to a certain extent, for objects with complex shapes, the traversal of the fixed trajectory may result in insufficient sampling in key areas such as high curvature, while redundant measurements are performed in flat areas, leading to low sensing efficiency. Furthermore, this "one-size-fits-all" sensing mode makes it difficult to dynamically adjust the focus and sequence of sensing based on initial tactile feedback, lacking overall coordination, thus limiting its application in real-time, dynamic tasks. Therefore, there is an urgent need to propose a new method that can adaptively perceive multiple attributes of an object and collaboratively sense them. To this end, this application proposes an object attribute sensing method, device, and storage medium based on dexterous hand tactile senses, which can adaptively perceive multiple attributes of an object and collaboratively sense them.

[0063] like Figure 1 As shown in the figure, this application provides a method for object attribute perception based on dexterity hand tactile sense, the method including the following steps:

[0064] Step S10: Control the dexterous hand equipped with a three-dimensional force sensor array to perform an initial light touch scan on the target object to obtain preliminary contour information of the target object.

[0065] In this embodiment, as Figure 2The illustrated 3D force sensor array refers to a collection of sensing units arranged in an array on the surface of a dexterous fingertip. Each sensing unit can decompose the multidimensional force it receives into a normal force (perpendicular to the contact surface) and a tangential force (parallel to the contact surface), thus outputting a 3D force vector. A dexterous hand refers to a multi-fingered robotic hand that mimics a human hand, possessing multiple degrees of freedom and capable of performing fine manipulations such as grasping, pinching, sliding, and pressing. Preliminary contour information refers to a set of discrete, relatively low-precision spatial point cloud data obtained through an initial light touch scan, and the approximate 3D shape of the target object reconstructed based on this spatial point cloud data.

[0066] In some implementations, the initial touch scan can be based on an initial exploration strategy. The hand maintains slight contact with the surface of the target object with a preset normal force and moves along the surface. The initial exploration strategy can be a spiral scanning strategy, a grid scanning strategy, a boundary tracking strategy, a random sampling strategy, etc. Specifically, a spiral scanning strategy involves controlling the dexterous hand to start from a point on the target object's surface (e.g., the highest point) and gradually expand outwards along a spiral path, progressively covering more areas of the target object's surface. A grid scanning strategy involves controlling the dexterous hand to move along a preset grid path, touching each grid point to progressively build preliminary contour information of the target object's surface. A boundary tracking strategy involves controlling the dexterous hand to start from a point on the target object's surface and move along the object's boundary to progressively track the object's contour. A random sampling strategy involves controlling the dexterous hand to randomly select multiple points on the target object's surface for touch, obtaining multiple local information points on the target object's surface through random sampling, and then synthesizing them to construct preliminary contour information.

[0067] It should be noted that the purpose of step S10 is not high-precision modeling, but to quickly and cost-effectively obtain the global geometric overview of the target object, thereby providing spatial prior information for subsequent physical property perception.

[0068] Step S20: Based on the preliminary contour information, determine the key regions on the surface of the target object, including high curvature regions and / or flat regions.

[0069] In this embodiment, the critical region refers to the surface area of ​​the object that has a significant impact on the efficiency and accuracy of sensing the overall physical properties of the target object. This region can include high-curvature regions and flat regions. High-curvature regions refer to areas on the target object's surface with a large degree of curvature, such as edges, corners, grooves, and small-radius curved surfaces. Flat regions refer to large areas on the target object's surface where the curvature is close to zero.

[0070] In some implementations, the critical area may further include a gripping area, which refers to a localized area on the target object suitable for stable and safe gripping by a dexterous hand. The criteria for determining the gripping area include at least: having two relatively parallel contact surfaces, and the distance between the two relatively parallel contact surfaces matching the size and opening / closing range of the dexterous fingertip.

[0071] Step S30: Determine the corresponding perception strategy based on the type of the key area.

[0072] Specifically, when the key area type includes a high curvature area, the corresponding perception strategy is determined to be a fine contour perception strategy, which specifically includes controlling the dexterous hand to perform multi-angle touches in the high curvature area and performing contour-following touches on the high curvature area of ​​the target object along the angle of maximum curvature to refine contour perception; when the key area type includes a flat area, the corresponding perception strategy is determined to be a material property perception strategy, which specifically includes controlling the dexterous hand to apply a fixed normal force to the flat area of ​​the target object to perform hardness perception and / or smoothness perception.

[0073] By performing targeted multi-angle conformal touch on high-curvature areas, the system ensures the precise reconstruction of complex object contours (such as edges and grooves), improving the accuracy of shape contour modeling. Simultaneously, by sensing material properties (such as hardness and smoothness) in flat areas, the system reduces interference from surface curvature on force signals, thereby improving the accuracy of hardness and smoothness perception.

[0074] In some implementations, when the type of key area includes a grasping area, the corresponding sensing strategy is determined to be an elastic sensing strategy, which specifically includes controlling the dexterous hand to pinch the grasping area of ​​the target object with two fingers to perform elastic modulus sensing.

[0075] Step S40: Control the dexterous hand to execute the corresponding perception strategy in the key area and obtain the physical attribute perception result.

[0076] In this embodiment, the physical attribute perception result refers to the quantitative information that can be used for robot decision-making, which is finally obtained by executing the perception strategy, such as the high-precision 3D outline model of the target object, hardness level, smoothness coefficient, elastic modulus, etc.

[0077] When the key region type includes high curvature regions, as a feasible implementation of a perceptual shape contour model, such as Figure 3 As shown, step S40 includes:

[0078] Step S41: Control the dexterous hand to perform multi-angle touches in the high curvature area and acquire real-time sensing data collected by the three-dimensional force sensor array.

[0079] In this embodiment, multi-angle touch refers to controlling a dexterous fingertip to contact and slide in the same high-curvature region or near the same high-curvature point within a high-curvature region from multiple different spatial directions. Real-time sensing data refers to the data streamed in real time by the three-dimensional force sensor array during the touch process, including the real-time three-dimensional force vector measured by each sensing unit.

[0080] Step S42: Based on the real-time sensing data, determine the real-time force area, real-time force direction, and real-time force magnitude, and use the real-time force area, real-time force direction, and real-time force magnitude as input to a preset neural network model to obtain the real-time curvature output by the preset neural network model.

[0081] In this embodiment, the real-time force area is determined based on the real-time data of the sensing unit that outputs real-time sensing data and the single force area of ​​the sensing unit. Based on the real-time three-dimensional force vector measured by each sensing unit in the real-time sensing data, the real-time force direction and real-time force magnitude of the real-time resultant force are determined.

[0082] A pre-defined neural network model refers to a deep learning model trained offline. Its input includes tactile features such as real-time force area, real-time force direction, and real-time force magnitude, and its output is an estimate of the real-time curvature of the local geometry at the contact point. Real-time curvature is a mathematical quantity used to characterize the degree of curvature of the target object's surface at the current contact point, and can be Gaussian curvature, mean curvature, etc.

[0083] Step S43: Determine the angle with the largest real-time curvature during the multi-angle touch process as the target angle, and control the dexterous hand to perform contour touch on the target object based on the target angle.

[0084] In this embodiment, the target angle refers to the touch direction that maximizes the real-time curvature estimated by the preset neural network model during multi-angle exploration. Geometrically, the direction corresponding to the target angle is usually perpendicular to the contour lines of the target object's surface, i.e., along the steepest descending / ascending direction, which is the principal curvature direction.

[0085] Shape-following touch refers to the movement of nimble fingertips that no longer follow a preset fixed trajectory, but dynamically adjust their direction of movement according to the real-time curvature, so that they always slide along the direction of the target object's surface (i.e., along the target angle direction).

[0086] By touching along the target angle (i.e., surface orientation) with the greatest real-time curvature, the fingertip can cover the longest effective contour with minimal movement, just like a human finger sliding along the edge of a bowl, which is natural and efficient. This avoids the redundant movements caused by zigzag scanning and significantly improves the efficiency of contour perception.

[0087] Step S44: When the global contour touch is detected to be completed, a shape contour model is constructed based on the movement trajectory of the dexterous hand.

[0088] In this embodiment, global contour touch can be determined to be completed when all high curvature areas are touched in a conformal manner, the fingertip movement trajectory forms a closed loop, or no new contour features are found during continuous exploration.

[0089] Based on the continuously recorded coordinate sequence (point cloud data) of the fingertip center point in three-dimensional space throughout the entire conformal touch process, the discrete point cloud data and the preliminary contour information are connected into a continuous triangular mesh surface through point cloud processing algorithms (such as Poisson reconstruction, rolling sphere method or Delaunay triangulation), thus forming the final shape contour model.

[0090] When the key area type includes flat areas, as a feasible implementation method for sensing hardness levels, such as... Figure 4 As shown, step S40 includes:

[0091] Step S45: Control the dexterous hand to apply a first normal force and press it on the flat area.

[0092] In this embodiment, as Figure 5 As shown, the first normal force F1 is a pre-set constant pressure value used for hardness sensing. Pressing refers to controlling the dexterous fingertip to apply force in a direction perpendicular to the surface of the target object, and maintaining a stable force value without any tangential relative movement between the fingertip and the object surface.

[0093] Step S46: When the force signal sensed by the three-dimensional force sensor array is stable, determine the first number of sensing units in the three-dimensional force sensor array that sensed the force signal.

[0094] In this embodiment, it should be noted that when the fingertip first touches the target object, the force signal detected by the three-dimensional force sensor array (the number of sensing units involved in sensing and the detected force value) may experience a brief oscillation. Force signal stability means that within a preset time window (e.g., 0.5 seconds), the force signal sensed by the three-dimensional force sensor array reaches a dynamic equilibrium. For example, the fluctuation of the sensed normal force value is less than a preset fluctuation threshold, and the number of sensing units involved in sensing in the three-dimensional force sensor array no longer changes trend.

[0095] The first point count refers to the total number of sensing units in the three-dimensional force sensor array whose readings exceed a preset pressure threshold after the force signal stabilizes. Each sensing unit whose reading exceeds the preset pressure threshold represents a valid micro-contact point.

[0096] like Figure 6As shown, when a skillful hand presses the same first normal force onto flat areas of different hardness levels, the number of sensing units in the three-dimensional force sensor array that sense the force signal, i.e., the area of ​​hardness sensing, varies. The higher the hardness level, the smaller the area of ​​hardness sensing.

[0097] Step S47: Calculate the hardness sensing force area based on the single force-bearing area of ​​the sensing unit and the first number of points.

[0098] In this embodiment, the area of ​​the force-sensing unit can be calculated by multiplying the single force-receiving area of ​​the sensing unit by the first point number.

[0099] In some implementations, if the three-dimensional force sensor array is not a uniform grid or the cell shape is irregular, a preset lookup table can be used to map different contact distribution patterns to a more accurate hardness-sensing force area.

[0100] Step S48: Determine the hardness level based on the hardness sensing force area and the preset hardness calibration function.

[0101] In this embodiment, the preset hardness calibration function refers to a mathematical model established through prior experiments that describes the correspondence between the hardness-sensing force-bearing area S1 and the hardness grade H under a fixed first normal force. Specifically, the dexterous hand can be controlled to perform steps S45-S47 on a set of standard samples with known hardness (such as rubber, plastic, and metal blocks) to obtain (S1, H) data pairs for each sample. A function is fitted using these data pairs, for example, hardness grade H = a × S1² + b × S1 + c, where a, b, and c are calibration parameters measured in the experiment. Hardness grade refers to a dimensionless or quantified value with specific units (such as Shore hardness) that represents the relative softness or hardness of a target object.

[0102] This implementation method, by employing a three-dimensional force sensor array and using the force-bearing area as the core metric, elevates the robot's hardness perception capability from traditional, limited single-point pressure measurement to a biomimetic spatial distribution perception level. This not only simulates the realistic and robust perception of the human hand but also achieves technological breakthroughs in measurement range, accuracy, and adaptability.

[0103] Specifically, single-point sensors typically calculate pressure using the formula: pressure = force / area. For very soft objects (such as sponges), large deformation occurs under very little force, leading to a sharp increase in contact area and keeping the pressure consistently at a very low level, making it difficult to distinguish from objects of medium hardness. For extremely hard objects (such as diamonds), the deformation is negligible, the contact area is extremely small and insensitive to changes, resulting in extremely high pressure values ​​that are easily affected by microscopic surface defects, leading to high measurement noise and poor repeatability.

[0104] In this application, the area of ​​the hardness sensor is very large and sensitive to changes in hardness for ultrasoft materials, while it is very small and stable for ultrahard materials. This makes the area of ​​the hardness sensor a sensitive variable across a wide hardness spectrum, enabling the precise characterization of a broad range of hardness, from foam and rubber to ceramics and metals, through a unified preset calibration function—something difficult to achieve with single-point pressure measurement.

[0105] As an optional implementation of the smoothness coefficient sensing method, after step S48, the dexterous fingertip remains pressed against the surface of the target object, maintaining a first normal force, and the dexterous hand is controlled to slide in the flat area according to a preset direction and a preset speed. When the force signal sensed by the three-dimensional force sensor array stabilizes, the total tangential force sensed in the three-dimensional force sensor array is determined. Based on the first normal force, the hardness level, and the total tangential force, the smoothness coefficient is determined.

[0106] In this embodiment, the preset direction refers to a predefined sliding path direction. To measure pure frictional force, this direction can be defined along a principal axis of the sensor coordinate system (e.g., the X-axis) to simplify force decomposition. The preset speed refers to a constant and low sliding speed. Low speed avoids dynamic effects (such as inertial forces and rate-dependent responses of viscoelastic materials), making the measurement process approximately quasi-static, thereby ensuring that the frictional force measurement mainly reflects surface characteristics rather than dynamic effects. The smoothness coefficient K is a dimensionless parameter used to characterize the smoothness of the surface.

[0107] Furthermore, considering that for soft objects, under the same normal force F1, a larger area of ​​contact with the perceived hardness leads to an increase in the total tangential force F1'. If the smoothness coefficient K = F1' / F1 is directly calculated, it would incorrectly determine that the surface of the target object is rough, when in fact it is simply due to the object's softness. Therefore, this embodiment utilizes a known hardness grade H to correct for deviations caused by the material's inherent properties. The smoothness coefficient K = (F1 / F1') × g(H), where f(H) is a hardness-related correction function that can be determined through calibration experiments. For example, a set of standard samples with known smoothness but different hardnesses are tested, and the function form of g(H) in K = (F1 / F1') × g(H) is fitted. For high-hardness objects, f(H) ≈ 1; for low-hardness objects, f(H) > 1, used to offset the increased F1' due to their softness.

[0108] This embodiment integrates smoothness measurement with previously perceived hardness levels, eliminating the interference of material hardness on smoothness judgment and achieving a more intrinsic smoothness perception that is closer to human touch. For example, humans can easily distinguish between a "soft, sticky feeling" and a "rough, scratchy feeling," and this embodiment achieves a similar distinguishing ability by introducing hardness correction. It enables the determination of correct and reasonable smoothness coefficients for a smooth sponge and a rough piece of wood, greatly improving the accuracy and robustness of smoothness perception.

[0109] As another alternative implementation of the perceived smoothness coefficient, such as Figure 7 and Figure 8 As shown, step S40 includes:

[0110] Step S49: Control the dexterous hand to apply a second normal force to press on the flat area, and slide on the flat area in a preset direction and at a preset speed.

[0111] In this embodiment, the second normal force F2 is a preset constant pressure value used for smoothness sensing. It can be the same as the first normal force F1 in hardness sensing, or it can be set independently according to different sensing requirements.

[0112] Step S50: When the force signal sensed by the three-dimensional force sensor array is stable, determine the second number of sensing units in the three-dimensional force sensor array that sensed the force signal, and the sensed tangential resultant force.

[0113] In this embodiment, when the force signal sensed by the three-dimensional force sensor array stabilizes, that is, after the fingertip sliding reaches a stable phase, the resultant normal force and resultant tangential force sensed by the three-dimensional force sensor array reach dynamic equilibrium. The second point refers to the number of all effective sensing units in the three-dimensional force sensor array that sensed the force signal during the sliding stabilization phase.

[0114] Step S51: Calculate the smoothness sensing force area based on the single force area of ​​the sensing unit and the second number of points.

[0115] In this embodiment, the smoothness sensing force area S2 can be calculated by multiplying the single force-bearing area of ​​the sensing unit by the second point number.

[0116] In some implementations, if the three-dimensional force sensor array is not a uniform grid or the unit shape is irregular, a preset lookup table can be used to map different contact distribution patterns to a more accurate smoothness sensing force area S2.

[0117] Step S52: Determine the smoothness coefficient based on the second normal force, the tangential resultant force, and the smoothness sensing force area.

[0118] Specifically, when the smoothness-sensing force area S2 is less than or equal to a preset area threshold, a rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force and the tangential resultant force. When the smoothness-sensing force area S2 is greater than the preset area threshold, a non-rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force, the tangential resultant force, and a preset calibration formula.

[0119] It should be noted that the preset area threshold is a contact area value determined through pre-calibration experiments, used to distinguish between rigid and elastic bodies. When the smoothness-sensing force area S2 is less than or equal to the preset area threshold, the target object is determined to be a rigid object, and its deformation has a negligible impact on the force area. Based on the rigid body smoothness calculation strategy, the smoothness coefficient K = F2' / F2. When the smoothness-sensing force area S2 is greater than the preset area threshold, the target object is determined to be an elastic object, and the non-rigid body smoothness calculation strategy K = (F1 / F1') × f(S2) should be used, where f(S2) is a function f(S2) = n × S2² + p × S + q with the smoothness-sensing force area S2 as the variable, used to correct the increase in tangential resultant force caused by the increase in the smoothness-sensing force area S2 of the elastic object. Here, n, p, and q are calibration parameters that can be determined through calibration experiments.

[0120] In this embodiment, the target object is classified into "rigid" or "non-rigid" categories based on the force-bearing area perceived by smoothness, and different smoothness calculation strategies are applied to each category, which greatly improves the accuracy and scientific nature of smoothness perception.

[0121] The object attribute perception method based on dexterous hand touch proposed in this application differs from existing technologies that use fixed, preset perception programs that cannot be adjusted according to the characteristics of the object itself, resulting in a large number of redundant operations or insufficient perception of key areas on complex objects. This application identifies key areas by using preliminary contour information obtained from an initial light touch scan, enabling the system to automatically identify high-curvature areas requiring focused perception and flat areas suitable for material perception, and execute differentiated perception strategies to collaboratively perceive multiple object attributes. This not only avoids unnecessary redundant operations but also optimizes the perception path and efficiency, and effectively improves the accuracy and reliability of the perception results.

[0122] In one embodiment, the physical property perception result includes the elastic modulus, such as... Figure 9 As shown, the method further includes:

[0123] Step S61: Control the first and second fingertips of the dexterous hand to perform a tightening action.

[0124] In this embodiment, as Figure 10 As shown, when the three-dimensional force sensor array of the first and second fingertips detects that either finger is in contact with the target object, the tightening action of the contacting finger is stopped until both the first and second fingertips are in contact with the target object, and the first distance L between the first and second fingertips is recorded.

[0125] In this context, the first and second fingertips refer to two fingers on a dexterous hand that are positioned opposite each other, such as the thumb and index finger, forming a gripping pair. The tightening action refers to the movement of the first and second fingertips from an open state towards each other, preparing to grip an object. The first distance L is the straight-line distance between the two fingertips when they just simultaneously contact the surface of the target object. This represents the original thickness or diameter of the target object when it is not gripped.

[0126] Step S62: Control the first fingertip and the second fingertip to clamp the target object with a third normal force.

[0127] In this embodiment, the third normal force F3 is a preset clamping force used to generate measurable deformation. Furthermore, the third normal force F3 can be adaptively adjusted based on previously perceived hardness levels. A smaller F3 is used for soft objects, and a larger F3 is used for hard objects.

[0128] Step S63: When the third normal force sensed by the three-dimensional force sensor array of the first fingertip and the second fingertip is stable, determine the third number of sensing units in the three-dimensional force sensor array that sense the third normal force, and the second distance between the first fingertip and the second fingertip.

[0129] In this embodiment, the third point count is the total number of sensing units on the three-dimensional force sensor array of the two fingertips that sense the third normal force F3 under the force-stable state. The second distance L' is the straight-line distance between the two fingertips under the force-stable state. This is the current thickness or diameter of the target object under clamping.

[0130] Step S64: Calculate the elastic sensing force area based on the single force-bearing area of ​​the sensing unit and the number of the third points.

[0131] In this embodiment, the elastic sensing force area S3 can be calculated by multiplying the single force-bearing area of ​​the sensing unit by the third point number.

[0132] In some implementations, if the three-dimensional force sensor array is not a uniform grid or the unit shape is irregular, a preset lookup table can be used to map different contact distribution patterns to a more accurate elastic sensing force area S3.

[0133] Step S65: Calculate the elastic modulus based on the third normal force, the elastic sensing area, the first spacing, and the second spacing.

[0134] In this embodiment, the unit area pressure P = F3 / S3 is calculated based on the third normal force F3 and the elastic sensing force area S3. The deformation ΔL = L - L' and the strain ε = ΔL / L are calculated based on the first spacing L and the second spacing L'. Therefore, the elastic modulus E = P / ε = F3 × L / (S3 × ΔL). The larger the elastic modulus E, the greater the elasticity of the target object; conversely, the smaller the elastic modulus E, the smaller the elasticity of the target object.

[0135] In the object attribute perception method based on dexterous hand tactile sense proposed in this application embodiment, in the first aspect, by controlling the first and second fingertips of the dexterous hand to perform a tightening action, and using a three-dimensional force sensor array to detect the contact between the fingers and the target object, the contact point between the fingers and the object can be accurately determined, ensuring the accuracy and stability of the clamping action, and can adapt to target objects of different shapes and sizes.

[0136] Secondly, by controlling the finger to grip the object with a third normal force and waiting for the third normal force sensed by the force sensor array to stabilize, it is possible to ensure that the force applied during the measurement process is uniform and stable, thereby improving the accuracy of the measurement results.

[0137] Thirdly, by calculating the elastic modulus based on the third normal force, the elastic sensing area, the first spacing, and the second spacing, not only is quantitative data on the elastic properties of the target object provided, but the robot can also adjust its operational strategy based on this data. For example, for objects with a high elastic modulus, the robot can use a more stable gripping method; while for objects with a low elastic modulus, a gentler manipulation method can be used to prevent deformation or damage. This adaptive operation strategy based on the elastic modulus greatly improves the robot's operational flexibility and adaptability in various complex environments.

[0138] In one embodiment, a computer storage medium is provided that stores executable instructions that, when executed by a processor, cause the processor to perform the steps in the above method embodiments.

[0139] In one embodiment, an electronic device is also provided, including one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the above method embodiments.

[0140] In one embodiment, such as Figure 11The diagram illustrates the structure of an electronic device used to implement an embodiment of this application. The electronic device includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 102 or a program loaded from a storage portion 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0141] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to I / O interface 105 as needed. A removable medium 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 110 as needed so that computer programs read from it can be installed into storage section 108 as needed.

[0142] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer-readable medium carrying instructions that, in such embodiments, can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111. When the instructions are executed by central processing unit (CPU) 101, the various method steps described in this application are performed.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0144] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, any of the embodiments or implementations claimed above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for perceiving object attributes based on dexterous hand tactile sense, characterized in that, The method includes: A dexterous hand equipped with a three-dimensional force sensor array is controlled to perform an initial light touch scan on the target object to obtain preliminary contour information of the target object; Based on the preliminary contour information, key regions on the surface of the target object are determined, including high curvature regions and / or flat regions. When the type of the key area includes a high curvature area, the corresponding perception strategy is determined to be multi-angle touch, and the target object is touched along the angle with the largest curvature. When the type of the key area includes a flat area, the corresponding perception strategy is to apply a fixed normal force to the target object for pressing. The dexterous hand is controlled to execute a corresponding perception strategy in the key area to obtain physical property perception results. When the physical property perception results include the elastic modulus, the first and second fingertips of the dexterous hand are controlled to perform a tightening action until both the first and second fingertips are in contact with the target object. The first distance between the first and second fingertips is recorded. The first and second fingertips are controlled to clamp the target object with a third normal force. When the third normal force sensed by the three-dimensional force sensor array of the first and second fingertips is stable, the third number of sensing units in the three-dimensional force sensor array that sense the third normal force and the second distance between the first and second fingertips are determined. Based on the single force-bearing area of ​​the sensing unit and the third number of points, the elastic sensing force-bearing area is calculated. The elastic modulus is calculated based on the third normal force, the elastic sensing force-bearing area, the first distance, and the second distance.

2. The object attribute perception method based on dexterous hand tactile sensation according to claim 1, characterized in that, The key region includes a high curvature region, and the physical property perception result includes a shape contour model. Controlling the dexterous hand to execute a corresponding perception strategy in the key region to obtain the physical property perception result includes: The dexterous hand is controlled to perform multi-angle touches in the high curvature region and acquire real-time sensing data collected by the three-dimensional force sensor array; Based on the real-time sensing data, the real-time force area, real-time force direction, and real-time force magnitude are determined, and the real-time force area, real-time force direction, and real-time force magnitude are used as input to a preset neural network model to obtain the real-time curvature output by the preset neural network model. The angle with the largest real-time curvature during multi-angle touch is determined as the target angle, and the dexterous hand is controlled to perform contour-following touch on the target object based on the target angle; Upon detecting a complete global contour touch, a shape contour model is constructed based on the movement trajectory of the dexterous hand.

3. The object attribute perception method based on dexterous hand tactile sensation according to claim 1, characterized in that, The key area includes a flat area, and the physical property perception result includes a hardness level. Controlling the dexterous hand to execute a corresponding perception strategy in the key area to obtain the physical property perception result includes: The dexterous hand is controlled to apply a first normal force and press against the flat area; When the force signal sensed by the three-dimensional force sensor array is stable, determine the first number of sensing units in the three-dimensional force sensor array that sensed the force signal; The area of ​​the force sensor is calculated based on the single force-bearing area of ​​the sensing unit and the number of the first points. The hardness level is determined based on the hardness sensing area and the preset hardness calibration function.

4. The object attribute perception method based on dexterous hand tactile sensation according to claim 3, characterized in that, The key region includes a flat region, and the physical property perception result also includes a smoothness coefficient. After determining the hardness level based on the hardness-sensing force area and a preset hardness calibration function, the following is also included: The dexterous hand is controlled to slide in the flat area according to a preset direction and a preset speed; When the force signal sensed by the three-dimensional force sensor array is stable, the total tangential force sensed in the three-dimensional force sensor array is determined; The smoothness coefficient is determined based on the first normal force, the hardness grade, and the total tangential force.

5. The object attribute perception method based on dexterous hand tactile sensation according to claim 1, characterized in that, The key region includes a flat area, and the physical property perception result includes a smoothness coefficient. Controlling the dexterous hand to execute a corresponding perception strategy in the key region to obtain the physical property perception result includes: The dexterous hand is controlled to apply a second normal force to press on the flat area, and slide on the flat area in a preset direction and at a preset speed; When the force signal sensed by the three-dimensional force sensor array is stable, determine the second number of sensing units in the three-dimensional force sensor array that sensed the force signal, and the sensed tangential resultant force. The smoothness sensing force area is calculated based on the single force area of ​​the sensing unit and the second number of points; The smoothness coefficient is determined based on the second normal force, the tangential resultant force, and the smoothness-sensing force area.

6. The object attribute perception method based on dexterous hand tactile sensation according to claim 5, characterized in that, The step of determining the smoothness coefficient based on the second normal force, the tangential resultant force, and the smoothness-sensing force area includes: When the smoothness sensing force area is less than or equal to a preset area threshold, a rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force and the tangential resultant force. When the smoothness sensing force area is greater than a preset area threshold, a non-rigid body smoothness calculation strategy is adopted to determine the smoothness coefficient based on the second normal force, the tangential resultant force and the preset calibration formula.

7. The object attribute perception method based on dexterous hand tactile sense according to any one of claims 1 to 6, characterized in that, The dexterity hand is controlled to perform a tightening action with its first and second fingertips until both fingertips are in contact with the target object. A first distance between the first and second fingertips is recorded, including: When the three-dimensional force sensor array of the first and second fingertips detects that either finger is in contact with the target object, the tightening action of the contacting finger is stopped until both the first and second fingertips are in contact with the target object, and the first distance between the first and second fingertips is recorded.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the object attribute perception method based on dexterous hand tactile feedback as described in any one of claims 1 to 7.

9. A computer storage medium, characterized in that, The storage medium stores executable instructions, which, when executed by a processor, cause the processor to perform the object attribute perception method based on dexterous hand tactile sensation as described in any one of claims 1 to 7.

Images