Bionic hand control method and device, bionic hand and robot

Abstract

The application discloses a bionic hand control method and device, a bionic hand and a robot. The method comprises the following steps: acquiring three-dimensional image information of a target object; inputting the three-dimensional image information into an object recognition model to obtain an object recognition result; inputting the object recognition result into a weight estimation model to obtain the weight of the object; querying a grasping gesture database according to the weight of the object to obtain a target grasping gesture; and controlling the bionic hand to grasp the target object according to the target grasping gesture. According to the three-dimensional image information of the target object, object recognition and weight estimation are performed according to the three-dimensional image information, so that the weight of the object is determined, and then the target grasping gesture is selected according to the weight of the object to grasp the target object. The bionic hand can select a suitable target grasping gesture according to the target object, the situation that the target object cannot be stably grasped by the target grasping gesture and the target object falls is avoided, and the grasping stability is improved.

Description

Technical Field

[0001] This application relates to the field of bionic hand technology, and in particular to a bionic hand control method, device, bionic hand, and robot. Background Technology

[0002] A bionic hand is an artificial device that mimics the structure, motor function, and sensory abilities of the human hand, primarily used in human-machine collaboration, rehabilitation assistance, and intelligent prostheses. Current methods for controlling bionic hands mostly employ position control or preset motion trajectory control.

[0003] However, using position control or preset motion trajectory control to control the bionic hand will prevent the bionic hand from selecting the appropriate target grasping gesture according to the target object. This can easily lead to the target object being dropped due to the inability of the target grasping gesture to hold it firmly, resulting in poor grasping stability of the bionic hand.

[0004] Therefore, there is still an urgent need for a bionic hand control method that can improve grasping stability. Summary of the Invention

[0005] The main purpose of this application is to propose a bionic hand control method, device, bionic hand and robot to solve the problem of poor grasping stability of existing bionic hands.

[0006] To achieve the above objectives, this application proposes a bionic hand control method, which includes:

[0007] Obtain the three-dimensional image information of the target object;

[0008] The three-dimensional image information is input into the object recognition model to obtain the object recognition result;

[0009] The object recognition result is input into the weight estimation model to obtain the object weight;

[0010] The target grasping gesture is obtained by querying the grasping gesture database based on the weight of the object.

[0011] The bionic hand is controlled to grasp the target object based on the target grasping gesture.

[0012] In some embodiments, the bionic hand includes a vision module; acquiring the three-dimensional image information of the target object includes:

[0013] The vision module is controlled to scan the target object to be grasped, and scan data is obtained;

[0014] Analyzing the scanned data yields the three-dimensional shape, size data, surface texture, and color information of the target object;

[0015] The three-dimensional shape, the size data, the surface texture, and the color information are determined as the three-dimensional image information.

[0016] In some embodiments, inputting the three-dimensional image information into an object recognition model to obtain an object recognition result includes:

[0017] The three-dimensional shape, the size data, the surface texture, and the color information are input into the object recognition model;

[0018] Receive the object volume and object material output by the object recognition model based on the three-dimensional shape, the size data, the surface texture, and the color information;

[0019] The object's volume and material are determined as the object recognition result.

[0020] In some embodiments, inputting the object recognition result into a weight estimation model to obtain the object weight includes:

[0021] Input the object's volume and material into the weight estimation model;

[0022] Receive the object weight output by the weight estimation model based on the object's volume and material.

[0023] In some embodiments, the bionic hand includes five fingers, and the target grasping gesture includes a grasping gesture and a grasping force; controlling the bionic hand to grasp the target object according to the target grasping gesture includes:

[0024] Analyze the grasping gesture and determine the target finger set for performing the grasping gesture from the bionic hand, wherein the grasping gesture includes a flat grip, a hook grip, a two-finger pinch, and a three-finger pinch, and the target finger set includes at least two fingers;

[0025] The target finger set is controlled to grasp the target object based on the grasping gesture and the grasping force.

[0026] In some embodiments, after controlling the bionic hand to grasp the target object according to the target grasping gesture, the method further includes:

[0027] The vision module is controlled to continuously monitor the grasping process of the bionic hand grasping the target object, and monitoring data is obtained.

[0028] Based on the monitoring data, it is determined whether the target object slipped during the grasping process;

[0029] If the target object slips during the grasping process, the grasping force is gradually increased, and the step of determining whether the target object has slipped during the grasping process continues.

[0030] The grasping force remains constant as long as the target object does not slip during the grasping process.

[0031] This application also proposes a bionic hand control device, the bionic hand control device comprising:

[0032] The acquisition unit is used to acquire the three-dimensional image information of the target object;

[0033] The recognition unit is used to input the three-dimensional image information into the object recognition model to obtain the object recognition result;

[0034] An estimation unit is used to input the object recognition result into a weight estimation model to obtain the object weight.

[0035] The query unit is used to query the grasping gesture database based on the weight of the object to obtain the target grasping gesture;

[0036] The grasping unit is used to control the bionic hand to grasp the target object according to the target grasping gesture.

[0037] In some embodiments, the bionic hand includes a vision module; the acquisition unit is specifically used for:

[0038] The vision module is controlled to scan the target object to be grasped, and scan data is obtained;

[0039] Analyzing the scanned data yields the three-dimensional shape, size data, surface texture, and color information of the target object;

[0040] The three-dimensional shape, the size data, the surface texture, and the color information are determined as the three-dimensional image information.

[0041] This application also proposes a bionic hand, which includes a controller, a vision module, and multiple fingers, wherein the controller is capable of performing the bionic hand control method described in any of the above descriptions.

[0042] This application also proposes a robot comprising a robot body and the aforementioned bionic hand.

[0043] This application obtains three-dimensional image information of a target object; inputs the three-dimensional image information into an object recognition model to obtain object recognition results; inputs the object recognition results into a weight estimation model to obtain the object weight; queries a grasping gesture database based on the object weight to obtain a target grasping gesture; and controls a bionic hand to grasp the target object based on the target grasping gesture. By using the three-dimensional image information of the target object, and then performing object recognition and weight estimation based on the three-dimensional image information to determine the object weight, and then selecting a suitable target grasping gesture based on the object weight, the bionic hand can select an appropriate target grasping gesture based on the target object, avoiding the situation where the target grasping gesture cannot hold the target object securely and causes the target object to fall, thus improving grasping stability. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the bionic hand control method in the embodiments of this application;

[0045] Figure 2 This is another flowchart illustrating the bionic hand control method in the embodiments of this application;

[0046] Figure 3 This is another flowchart illustrating the bionic hand control method in the embodiments of this application;

[0047] Figure 4 This is another flowchart illustrating the bionic hand control method in the embodiments of this application;

[0048] Figure 5 This is another flowchart illustrating the bionic hand control method in the embodiments of this application;

[0049] Figure 6 This is another flowchart illustrating the bionic hand control method in the embodiments of this application;

[0050] Figure 7 This is a schematic diagram of the structure of the bionic hand control device according to the embodiments of this application;

[0051] Figure 8 This is a schematic diagram of the structure of the bionic hand involved in the embodiments of this application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] The solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments in this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0054] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0055] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0056] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0057] To achieve the above objectives, this application proposes a bionic hand control method, which includes:

[0058] Step S110: Obtain the three-dimensional image information of the target object;

[0059] Step S120: Input the 3D image information into the object recognition model to obtain the object recognition result;

[0060] Step S130: Input the object recognition result into the weight estimation model to obtain the object weight;

[0061] Step S140: Query the grasping gesture database based on the object's weight to obtain the target grasping gesture;

[0062] Step S150: Control the bionic hand to grasp the target object according to the target grasping gesture.

[0063] In this embodiment, refer to Figure 1 , Figure 7 and Figure 8 Bionic hand control methods can be applied to bionic hands. A bionic hand includes a controller, a vision module, and multiple fingers (of which, in...) Figure 8 (Illustrated with one finger). The controller connects to the vision module and each finger, and can control the operation of the vision module and each finger. The controller may include, for example... Figure 7 The bionic hand control device shown is an example. In this embodiment, the controller is the primary entity executing the method steps.

[0064] Understandably, each finger also includes a drive module, and the controller controls the fingers to perform their functions by controlling these drive modules. The vision module can be used to acquire image data. The controller can be pre-configured with an object recognition model, a weight estimation model, and a grasping gesture database. The object recognition model and weight estimation model can be pre-trained by the user and then configured onto the controller. Similarly, the grasping gesture database can be pre-configured by the user and then configured onto the controller.

[0065] The bionic hand may also include a communication model and / or a communication interface, with the controller connected to the communication model and / or communication interface. The communication model may also be wirelessly connected to a user terminal, and the communication interface may be wired to the user terminal. The user terminal can send commands to the controller via the communication model and / or communication interface to control the bionic hand's operation. When the user needs to control the bionic hand to grasp a target object, the user can send a grasping command to the controller via the user terminal. Upon receiving the grasping command, the controller can respond to the command and begin controlling the bionic hand to grasp the target object.

[0066] The controller first directs the vision module to collect data from the external environment, thereby identifying the target object to be grasped. Once the target object is identified, the controller can then control the vision module to scan the target object, thereby acquiring its 3D image information.

[0067] The controller can be pre-configured with an object recognition model, which can be pre-trained by the user and then configured onto the controller. For example, the user can configure an initial object recognition model based on their object recognition needs. Then, the user collects 3D image information of various objects and annotates this information, including object volume and material. Finally, the user uses the annotated 3D image information to train the initial object recognition model, thus obtaining the final object recognition model. After obtaining the object recognition model, the user can configure it onto the controller.

[0068] After acquiring the 3D image information of the target object, the controller can input this information into the object recognition model. Upon receiving the 3D image information, the object recognition model can perform recognition on it, obtaining the object recognition result, which is then output. The controller can then obtain the object recognition result. For example, after receiving the 3D image information, the object recognition model can perform recognition to obtain the object's volume and material, and then output these parameters. The controller can then obtain the object's volume and material as the object recognition result.

[0069] After receiving the object recognition result, the controller can estimate the weight of the target object. The controller can input the object recognition result into the weight estimation model. The weight estimation model, after receiving the object recognition result, can estimate the weight of the target object based on the recognition result, thus obtaining the object's weight, and then output the object's weight. At this point, the controller can obtain the object's weight. For example, the controller can input the object's volume and material into the weight estimation model. The weight estimation model can determine the object's density based on the material, calculate the object's weight based on the object's volume and density, and then output the object's weight. Then the controller can obtain the object's weight.

[0070] The grasping gesture database can include a mapping relationship between weights and grasping gestures. One weight can map to one grasping gesture, or one weight can map to multiple grasping gestures. After obtaining the object's weight, the controller can use the object's weight to query the grasping gesture database to find the grasping gesture that maps to the object's weight and identify that gesture as the target grasping gesture. For example, if querying the grasping gesture database using the object's weight yields only one grasping gesture that maps to the object's weight, the controller can directly identify that gesture as the target grasping gesture. If querying the database using the object's weight yields multiple grasping gestures that map to the object's weight, the controller can randomly select one of these gestures as the target grasping gesture; or the controller can compare the usage frequency of these gestures and determine the most frequently used gesture as the target grasping gesture.

[0071] Once the controller receives the target grasping gesture, it can control the bionic hand to grasp the target object based on the gesture. In other words, the controller controls the bionic hand to make it display the target grasping gesture and grasp the target object using that gesture.

[0072] This embodiment acquires three-dimensional image information of the target object; inputs the three-dimensional image information into an object recognition model to obtain the object recognition result; inputs the object recognition result into a weight estimation model to obtain the object weight; queries a grasping gesture database based on the object weight to obtain the target grasping gesture; controls the bionic hand to grasp the target object based on the target grasping gesture; by using the three-dimensional image information of the target object, and then performing object recognition and weight estimation based on the three-dimensional image information, the object weight is determined, and then the appropriate target grasping gesture is selected based on the object weight to grasp the target object. This allows the bionic hand to select the appropriate target grasping gesture based on the target object, avoiding the situation where the target grasping gesture cannot hold the target object firmly and causing the target object to fall, thus improving the grasping stability.

[0073] In some embodiments, the bionic hand includes a vision module; the aforementioned acquisition of three-dimensional image information of the target object includes:

[0074] Step S111: Control the vision module to scan the target object to be grasped and obtain scan data;

[0075] Step S112: Analyze the scan data to obtain the three-dimensional shape, size data, surface texture, and color information of the target object;

[0076] Step S113: The three-dimensional shape, size data, surface texture and color information are determined as three-dimensional image information.

[0077] In this embodiment, refer to Figure 2 When executing step S110, the controller needs to control the vision module to scan the target object to be grasped. The bionic hand may include a vision module. The controller can first control the vision module to recognize the external environment of the bionic hand, thereby identifying the target object to be grasped. After recognizing the target object, the controller can control the vision module to scan the target object to be grasped, thereby obtaining scan data.

[0078] After obtaining the scan data of the target object, the controller can analyze the scan data. The controller analyzes the scan data to obtain the target object's 3D shape, size data, surface texture, and color information. The 3D shape can be the solid shape of the target object, such as a cuboid, cylinder, cone, or sphere, or a combination of one or more of these shapes. The size data can be the length, width, and height of the target object. The surface texture can be the material texture of the target object; for example, different materials often have their own characteristic textures, such as metal textures, wood textures, and fabric textures. The color information can be the color of the target object, such as red, green, or blue.

[0079] After the controller obtains the three-dimensional shape, size data, surface texture, and color information, it can determine the three-dimensional shape, size data, surface texture, and color information as three-dimensional image information.

[0080] In some embodiments, the aforementioned input of 3D image information into an object recognition model to obtain object recognition results includes:

[0081] Step S121: Input the three-dimensional shape, size data, surface texture and color information into the object recognition model;

[0082] Step S122: Receive the object volume and material output by the object recognition model based on the three-dimensional shape, size data, surface texture, and color information;

[0083] Step S123: Determine the object volume and object material as the object recognition result.

[0084] In this embodiment, refer to Figure 3 When executing step S120, the controller can identify the object's volume and material. The controller can be pre-configured with an object recognition model, which can be pre-trained by the user and then configured onto the controller. For example, the user can configure an initial object recognition model based on their object recognition needs. Then, the user collects 3D image information of various objects and annotates this information, including 3D shape, size data, surface texture, color information, object volume, and object material. Finally, the user trains the initial object recognition model using the annotated 3D image information to obtain the final object recognition model. After obtaining the object recognition model, the user can configure it onto the controller.

[0085] The controller can input the 3D shape, size data, surface texture, and color information, which are determined as 3D image information, into the object recognition model. After receiving the 3D shape, size data, surface texture, and color information, the object recognition model can recognize the 3D shape, size data, surface texture, and color information to identify the object's volume and material, and then output the object's volume and material. At this point, the controller can receive the object volume and material output by the object recognition model based on the 3D shape, size data, surface texture, and color information.

[0086] After obtaining the object's volume and material, the controller can determine the object's volume and material as the object recognition result.

[0087] In some embodiments, the aforementioned inputting the object recognition result into the weight estimation model to obtain the object weight includes:

[0088] Step S131: Input the object volume and object material into the weight estimation model;

[0089] Step S132: Receive the object weight output by the weight estimation model based on the object volume and object material.

[0090] In this embodiment, refer to Figure 4 When the controller executes step S130, it inputs the object's volume and material into the weight estimation model. The controller can have a pre-configured weight estimation model, which can be pre-trained by the user and then configured onto the controller. The weight calculation formula is: weight equals density multiplied by volume multiplied by gravitational acceleration. In typical scenarios on the Earth's surface, gravitational acceleration can be constant; therefore, the weight estimation model only needs to estimate the object's density to calculate its weight. For example, the user can configure a raw density estimation model based on density estimation requirements. The user then collects and labels the material properties of various objects, including their density; the raw density estimation model is then trained using the labeled material properties to obtain the density estimation model; finally, the weight estimation model can be configured based on the density estimation model and the weight calculation formula.

[0091] After the controller obtains the object's volume and material, it can input these parameters into the weight estimation model. The weight estimation model, having obtained the object's volume and material, first estimates the object's density by calculating its density. Then, it substitutes the object's density, volume, and gravitational acceleration into the weight calculation formula to obtain the object's weight, which is then output. At this point, the controller can receive the object's weight output by the weight estimation model based on the object's volume and material.

[0092] In some embodiments, the bionic hand includes five fingers, and the target grasping gesture includes a grasping gesture and a grasping force; the aforementioned control of the bionic hand to grasp a target object based on the target grasping gesture includes:

[0093] Step S151: Analyze the grasping gesture and determine the target finger set for performing the grasping gesture from the bionic hand. The grasping gesture includes a flat grip, a hook grip, a two-finger pinch, and a three-finger pinch. The target finger set includes at least two fingers.

[0094] Step S152: Control the target finger set to grasp the target object based on the grasping gesture and grasping force.

[0095] In this embodiment, refer to Figure 5When executing step S150, the controller needs to analyze the grasping gesture. The bionic hand includes five fingers, and the target grasping gesture includes the grasping motion and the grasping force. Grasping gestures include a flat grip, a hook grip, a two-finger pinch, and a three-finger pinch. The controller can analyze the grasping gesture and determine the target finger set from the bionic hand to perform the grasping gesture. The target finger set includes at least two fingers. For example, when the grasping gesture is a three-finger pinch, the controller first determines the fingers from the bionic hand capable of performing a three-finger pinch, and then uses these fingers as the target finger set.

[0096] Once the controller identifies the target finger set, it can control the target finger set based on the grasping gesture and grasping force, causing the target finger set to use the grasping force and grasping gesture to grasp the target object. (Refer to...) Figure 8 The fingers are equipped with pressure sensors that are used to collect the current pressure on the fingers in real time. Figure 8 The diagram uses a single finger as an example, and each finger can be equipped with a pressure sensor. When the target finger set grasps the target object, the controller can control the pressure sensors of each finger in the target finger set to collect pressure data, and monitor the grasping force of the target finger set based on the pressure data.

[0097] In some embodiments, after controlling the bionic hand to grasp the target object based on the target grasping gesture, the method further includes:

[0098] Step S160: Control the vision module to continuously monitor the grasping process of the bionic hand grasping the target object and obtain monitoring data;

[0099] Step S161: Based on the monitoring data, determine whether the target object slips during the grasping process;

[0100] Step S162: When the target object slips during the grasping process, gradually increase the grasping force and continue to execute the step of determining whether the target object slips during the grasping process;

[0101] Step S163: If the target object does not slip during the grasping process, maintain the grasping force unchanged.

[0102] In this embodiment, refer to Figure 6 After executing step S150, the controller can also determine whether the target object slips during the grasping process. After controlling the bionic hand to grasp the target object, the controller can control the vision module to continuously monitor the grasping process of the bionic hand to obtain monitoring data.

[0103] After receiving the monitoring data, the controller can determine whether the target object slipped during the grasping process. For example, the controller can segment the monitoring data into multiple consecutive frames based on time sequence; then, it determines the relative position of the target object and the bionic hand based on the first frame, using this as the initial relative position; finally, it compares the relative positions of the target object and the bionic hand in each subsequent frame with the initial relative position to determine whether the target object slipped during the grasping process. If the relative position of the target object and the bionic hand in a particular frame is shifted compared to the initial relative position, it indicates that the target object slipped during the grasping process; if no shift is observed, it indicates that the target object did not slip during the grasping process.

[0104] If the target object slips during the grasping process, the controller can gradually increase the grasping force while continuing to determine if the object has slipped. If the object does not slip, the controller can maintain a constant grasping force to stably hold the target object. For example, while gradually increasing the grasping force, the controller continuously checks if the object is still slipping. If it does slip, the grasping force is increased further. If it does not slip, the grasping force is stopped. Gradually increasing the grasping force can involve increasing it by the same amount each time. For example, if the initial grasping force is 1 N (Newton), the gradual increase could be 0.1 N each time.

[0105] This application obtains three-dimensional image information of a target object; inputs the three-dimensional image information into an object recognition model to obtain object recognition results; inputs the object recognition results into a weight estimation model to obtain the object weight; queries a grasping gesture database based on the object weight to obtain a target grasping gesture; and controls a bionic hand to grasp the target object based on the target grasping gesture. By using the three-dimensional image information of the target object, and then performing object recognition and weight estimation based on the three-dimensional image information to determine the object weight, and then selecting a suitable target grasping gesture based on the object weight, the bionic hand can select an appropriate target grasping gesture based on the target object, avoiding the situation where the target grasping gesture cannot hold the target object securely and causes the target object to fall, thus improving grasping stability.

[0106] Reference Figure 7 This application also proposes a bionic hand control device 20, which includes:

[0107] Acquisition unit 201 is used to acquire three-dimensional image information of the target object;

[0108] The recognition unit 202 is used to input the three-dimensional image information into the object recognition model to obtain the object recognition result;

[0109] The estimation unit 203 is used to input the object recognition result into the weight estimation model to obtain the object weight;

[0110] The query unit 204 is used to query the grasping gesture database based on the weight of the object to obtain the target grasping gesture;

[0111] The grasping unit 205 is used to control the bionic hand to grasp the target object according to the target grasping gesture.

[0112] In some embodiments, the bionic hand includes a vision module; the acquisition unit 201 is specifically used for:

[0113] The vision module is controlled to scan the target object to be grasped, and scan data is obtained;

[0114] Analyzing the scanned data yields the three-dimensional shape, size data, surface texture, and color information of the target object;

[0115] The three-dimensional shape, the size data, the surface texture, and the color information are determined as the three-dimensional image information.

[0116] In some embodiments, the identification unit 202 is specifically used for:

[0117] The three-dimensional shape, the size data, the surface texture, and the color information are input into the object recognition model;

[0118] Receive the object volume and object material output by the object recognition model based on the three-dimensional shape, the size data, the surface texture, and the color information;

[0119] The object's volume and material are determined as the object recognition result.

[0120] In some embodiments, the estimation unit 203 is specifically used for:

[0121] Input the object's volume and material into the weight estimation model;

[0122] Receive the object weight output by the weight estimation model based on the object's volume and material.

[0123] In some embodiments, the bionic hand includes five fingers, and the target grasping gesture includes a grasping gesture and a grasping force; the grasping unit 205 is specifically used for:

[0124] Analyze the grasping gesture and determine the target finger set for performing the grasping gesture from the bionic hand, wherein the grasping gesture includes a flat grip, a hook grip, a two-finger pinch, and a three-finger pinch, and the target finger set includes at least two fingers;

[0125] The target finger set is controlled to grasp the target object based on the grasping gesture and the grasping force.

[0126] In some embodiments, the bionic hand control device 20 further includes:

[0127] The control unit is used to control the vision module to continuously monitor the grasping process of the bionic hand grasping the target object and obtain monitoring data;

[0128] The judgment unit is used to determine, based on the monitoring data, whether the target object slips during the grasping process;

[0129] An additional unit is provided to progressively increase the grasping force when the target object slips during the grasping process, and to continue executing the step of determining whether the target object has slipped during the grasping process;

[0130] The maintaining unit is used to maintain the grasping force unchanged when the target object does not slip during the grasping process.

[0131] Reference Figure 8 This application also proposes a bionic hand 30, which includes a controller 301, a vision module 302, and multiple fingers 303. The controller 301 is capable of executing the bionic hand control method described in any of the above-mentioned applications.

[0132] In this embodiment, refer to Figure 7 and Figure 8 The bionic hand 30 includes a controller 301, a vision module 302, and multiple fingers 303 (wherein, in Figure 8 (Illustrated with one finger). The controller 301 is connected to the vision module 302 and each finger 303, and can control the vision module 302 and each finger 303 to operate. The controller 301 may include, for example: Figure 7 The bionic hand control device 20 shown.

[0133] The bionic hand 30 may also include pressure tactile sensors 304, which can be configured on each finger 303. The controller 301 can also control the pressure tactile sensors 304 of each finger 303 to collect the current pressure of each finger 303 in real time. Each finger 303 may also include a drive module, and the controller 301 controls the operation of each finger 303 by controlling the drive module of each finger 303.

[0134] This application also proposes a robot, which includes a robot body and a bionic hand as described above.

[0135] The above description is only a part or preferred embodiment of this application. Neither the text nor the drawings should limit the scope of protection of this application. All equivalent structural transformations made using the content of this application's specification and drawings under the overall concept of this application, or direct / indirect applications in other related technical fields, are included within the scope of protection of this application.

Claims

1. A bionic hand control method, characterized in that, The bionic hand control method includes: Obtain the three-dimensional image information of the target object; The three-dimensional image information is input into the object recognition model to obtain the object recognition result; The object recognition result is input into the weight estimation model to obtain the object weight; The target grasping gesture is obtained by querying the grasping gesture database based on the weight of the object. The bionic hand is controlled to grasp the target object based on the target grasping gesture; The bionic hand includes a vision module; acquiring the three-dimensional image information of the target object includes: The vision module is controlled to scan the target object to be grasped, and scan data is obtained; Analyzing the scanned data yields the three-dimensional shape, size data, surface texture, and color information of the target object; The three-dimensional shape, the size data, the surface texture, and the color information are determined as the three-dimensional image information; The control vision module scans the target object to be grasped to obtain scan data, including: controlling the vision module to identify the external environment of the bionic hand and identify the target object to be grasped; controlling the vision module to scan the target object to be grasped to obtain scan data. The step of inputting the three-dimensional image information into the object recognition model to obtain the object recognition result includes: The three-dimensional shape, the size data, the surface texture, and the color information are input into the object recognition model; Receive the object volume and object material output by the object recognition model based on the three-dimensional shape, the size data, the surface texture, and the color information; The object's volume and material are determined as the object recognition result; The object recognition model is trained from the three-dimensional image information of various objects, and the three-dimensional image information of various objects is labeled with three-dimensional shape, size data, surface texture, color information, object volume and object material.

2. The bionic hand control method according to claim 1, characterized in that, The step of inputting the object recognition result into the weight estimation model to obtain the object weight includes: Input the object's volume and material into the weight estimation model; Receive the object weight output by the weight estimation model based on the object's volume and material.

3. The bionic hand control method according to claim 2, characterized in that, The bionic hand includes five fingers, and the target grasping gesture includes a grasping gesture and a grasping force; controlling the bionic hand to grasp the target object according to the target grasping gesture includes: Analyze the grasping gesture and determine the target finger set for performing the grasping gesture from the bionic hand, wherein the grasping gesture includes a flat grip, a hook grip, a two-finger pinch, and a three-finger pinch, and the target finger set includes at least two fingers; The target finger set is controlled to grasp the target object based on the grasping gesture and the grasping force.

4. The bionic hand control method according to claim 3, characterized in that, After controlling the bionic hand to grasp the target object according to the target grasping gesture, the method further includes: The vision module is controlled to continuously monitor the grasping process of the bionic hand grasping the target object, and monitoring data is obtained. Based on the monitoring data, it is determined whether the target object slipped during the grasping process; If the target object slips during the grasping process, the grasping force is gradually increased, and the step of determining whether the target object has slipped during the grasping process continues. The grasping force remains constant as long as the target object does not slip during the grasping process.

5. A bionic hand control device, characterized in that, The bionic hand control device includes: The acquisition unit is used to acquire the three-dimensional image information of the target object; The recognition unit is used to input the three-dimensional image information into the object recognition model to obtain the object recognition result; An estimation unit is used to input the object recognition result into a weight estimation model to obtain the object weight. The query unit is used to query the grasping gesture database based on the weight of the object to obtain the target grasping gesture; A grasping unit is used to control the bionic hand to grasp the target object according to the target grasping gesture; The bionic hand includes a vision module; the acquisition unit is specifically used for: The vision module is controlled to scan the target object to be grasped, and scan data is obtained; Analyzing the scanned data yields the three-dimensional shape, size data, surface texture, and color information of the target object; The three-dimensional shape, the size data, the surface texture, and the color information are determined as the three-dimensional image information; Specifically, when the acquisition unit executes the step of controlling the vision module to scan the target object to be grasped and obtain scanning data, it is used to: control the vision module to identify the external environment of the bionic hand and identify the target object to be grasped; control the vision module to scan the target object to be grasped and obtain scanning data. The identification unit is specifically used for: The three-dimensional shape, the size data, the surface texture, and the color information are input into the object recognition model; Receive the object volume and object material output by the object recognition model based on the three-dimensional shape, the size data, the surface texture, and the color information; The object's volume and material are determined as the object recognition result; The object recognition model is trained from the three-dimensional image information of various objects, and the three-dimensional image information of various objects is labeled with three-dimensional shape, size data, surface texture, color information, object volume and object material.

6. A bionic hand, characterized in that, The bionic hand includes a controller, a vision module, and multiple fingers, wherein the controller is capable of performing the bionic hand control method according to any one of claims 1-4.

7. A robot, characterized in that, The robot includes a robot body and the bionic hand as described in claim 6.

Images