Bimodal tactile sensor, fruit picking system and picking robot

Abstract

The invention relates to the technical field of fruit picking, and particularly provides a bimodal touch sensor, a fruit picking system and a picking robot. The lower electrode layer is fixed at the top of the base and comprises a first planar electrode and a plurality of first extension electrodes; the upper electrode layer comprises a second planar electrode and a plurality of second extension electrodes, the second extension electrodes are arranged above the first extension electrodes, and the number of the second extension electrodes is consistent with that of the first extension electrodes; the piezoresistive sensor is arranged between the first planar electrode and the second planar electrode; a plurality of triboelectric sensors which are arranged above the base in a circumferential array and are arranged between the first extension electrode and the second extension electrode; the flexible shell is connected with the top of the upper electrode layer; the sensor can effectively solve the problem that the reliability of the obtained dynamic information and the grabbing force is insufficient due to the fact that the dynamic information representing the fruit maturity and the grabbing force applied to the fruit cannot be detected at the same grabbing position point.

Description

Technical Field

[0001] The present application relates to the technical field of fruit picking, and in particular to a dual-modal tactile sensor, a fruit picking system and a picking robot. Background Art

[0002] Related tactile sensors can be divided into piezoresistive tactile sensors, piezoelectric tactile sensors and triboelectric tactile sensors. Piezoelectric tactile sensors and triboelectric tactile sensors can generate instantaneous voltage potential when the applied mechanical force changes. When the piezoelectric tactile sensor or triboelectric tactile sensor contacts the fruit, the mechanical force applied to the sensor will change. At this time, the sensor will generate an instantaneous voltage potential associated with the change in mechanical force, and the change in mechanical force is related to the maturity of the fruit. Therefore, the related technology can use piezoelectric tactile sensors or triboelectric tactile sensors to collect dynamic information for characterizing the maturity of the fruit. Since the piezoresistive tactile sensor has low sensitivity and long response time, that is, the piezoresistive tactile sensor cannot accurately measure the information for characterizing the maturity of the fruit based on the change in mechanical force when the sensor contacts the fruit, the piezoresistive tactile sensor can only be used to measure the grasping force applied by the robot to the fruit after the tactile sensor contacts the fruit. In order to realize automatic picking of fruits, the relevant technology needs to sense the maturity of the fruits and the grasping force applied by the robot to the fruits, that is, the relevant technology needs to set piezoresistive tactile sensors and piezoelectric tactile sensors or triboelectric tactile sensors at different positions at the end of the robot. Since the detection position of the piezoresistive tactile sensor is different from the detection position of the piezoelectric tactile sensor or the triboelectric tactile sensor, that is, the fruit maturity sensing position is different from the grasping force sensing position, the relevant technology has the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information of the fruit maturity and the grasping force applied to the fruit at the same grasping position point.

[0003] There is no effective technical solution to the above problems. It should be noted that the above information disclosed in this section is only used to understand the background of the present invention, and therefore may contain information that does not constitute prior art. Summary of the invention

[0004] The purpose of the present application is to provide a dual-modal tactile sensor, a fruit picking system and a picking robot, which can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the maturity of the fruit and the grasping force applied to the fruit at the same grasping position point.

[0005] In a first aspect, the present application provides a dual-modal tactile sensor, comprising: Base; The lower electrode layer is fixed on the top of the base, and includes a first planar electrode and a plurality of first extension electrodes, wherein the plurality of first extension electrodes are arranged in a circular array outside the first planar electrode and connected to the first planar electrode; An upper electrode layer, comprising a second planar electrode and a plurality of second extension electrodes, wherein the plurality of second extension electrodes are arranged in a circumferential array outside the second planar electrode and connected to the second planar electrode, and the second extension electrodes are arranged above the first extension electrodes and the number of the second extension electrodes is the same as the number of the first extension electrodes; A piezoresistive sensor is disposed between the first planar electrode and the second planar electrode and is used to collect normal force information representing a grasping force applied to the fruit; A plurality of triboelectric sensors, arranged in a circumferential array between the first extension electrode and the second extension electrode, for collecting triboelectric information indicative of the ripeness of the fruit; A flexible housing is connected to the top of the upper electrode layer.

[0006] The present application provides a dual-modal tactile sensor, which can sense the maturity of fruits and the grasping force applied to fruits using the same tactile sensor through the cooperation of a base, a lower electrode layer, a triboelectric sensor, a piezoresistive sensor, an upper electrode layer and a flexible shell. That is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can sense the maturity of fruits and the grasping force applied to fruits at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the maturity of fruits and the grasping force applied to fruits at the same grasping position point.

[0007] Optionally, the triboelectric sensor includes an upper sensor and a lower sensor, the cross-sectional shapes of the upper sensor and the lower sensor are both stepped, and the contact surfaces of the upper sensor and the lower sensor are perpendicular to the top surface of the flexible shell.

[0008] Optionally, the material of the upper sensor is polytetrafluoroethylene, and the material of the lower sensor is polyamide.

[0009] This technical solution selects polytetrafluoroethylene as the material of the upper sensor and polyamide as the material of the lower sensor. Since polytetrafluoroethylene and polyamide have large differences in their ability to attract electrons, this technical solution can increase the triboelectric information generated by the triboelectric sensor when the relative displacement of the upper sensor and the lower sensor is the same, so as to reduce the error of the triboelectric information, thereby effectively improving the accuracy of the fruit maturity obtained from the triboelectric information.

[0010] Optionally, the area between the upper electrode layer and the lower electrode layer except for the piezoresistive sensor and the triboelectric sensor is filled with an elastic adhesive.

[0011] In a second aspect, the present application also provides a fruit picking system, which includes a manipulator, a controller and a plurality of dual-modal tactile sensors provided in the first aspect above, wherein the plurality of dual-modal tactile sensors are respectively arranged at different ends of the manipulator, and the controller is used to control the movement of the manipulator and its ends so that all the dual-modal tactile sensors are in contact with the fruit, and then control the movement of the manipulator and its ends according to the normal force information to pick the fruit.

[0012] The present application provides a fruit picking system, which can sense the maturity of the fruit and the grasping force applied to the fruit using the same tactile sensor through the cooperation of a base, a lower electrode layer, a triboelectric sensor, a piezoresistive sensor, an upper electrode layer and a flexible shell. That is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can sense the maturity of the fruit and the grasping force applied to the fruit at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the maturity of the fruit and the grasping force applied to the fruit at the same grasping position point.

[0013] Optionally, the normal force information includes a normal vector, and the controller controls the manipulator and its end to move according to the normal force information to pick the fruit, and the process includes: A1, controlling the end of the manipulator to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches a preset force to grasp the fruit; A2. Obtaining a tangential vector based on all triboelectric information collected by at least one dual-modal tactile sensor when the end of the manipulator moves along the normal vector; A3. Control the robot to move along the tangential vector to pick the fruit from the plant.

[0014] Before picking the fruit from the plant, this technical solution can increase the clamping force on the fruit by controlling the end of the manipulator 8 to move along the normal vector collected by the dual-modal tactile sensor connected to the end, so as to hold the fruit firmly and avoid accidental falling of the fruit.

[0015] Optionally, step A1 includes: A11. Obtaining fruit type information, and obtaining a preset strength according to the fruit type information and a third preset conversion relationship; A12. Control the end of the manipulator to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches a preset force to grasp the fruit.

[0016] Optionally, each dual-modal tactile sensor includes a first triboelectric sensor group and a second triboelectric sensor group, and the first triboelectric sensor group and the second triboelectric sensor group each include two triboelectric sensors disposed opposite to each other, and step A2 includes: A21. Obtaining a first vector corresponding to each dual-modal tactile sensor according to a difference in triboelectric information of two triboelectric sensors in the first triboelectric sensor group and a first preset conversion relationship; A22. Obtaining a second vector corresponding to each dual-modal tactile sensor according to a difference in triboelectric information of two triboelectric sensors in the second triboelectric sensor group and a first preset conversion relationship; A23. Obtaining a three-dimensional force vector corresponding to each dual-modal tactile sensor according to the first vector and the corresponding second vector, and obtaining a preliminary tangential vector corresponding to each dual-modal tactile sensor according to the three-dimensional force vector; A24. Generate a tangent vector based on all preliminary tangent vectors.

[0017] Optionally, step A2 further includes the following steps performed before step A21: A25, obtaining fruit maturity information according to the triboelectric information collected by the triboelectric sensor when the dual-modal tactile sensor initially contacts the fruit and a second preset conversion relationship; A26, analyzing whether the fruit maturity information reaches the preset maturity, if so, executing step A21, if not, controlling the end of the manipulator to release the fruit.

[0018] This technical solution is equivalent to picking only fruits that have reached a preset degree of maturity. Therefore, this technical solution can effectively avoid the situation where the nutritional value of the fruit is reduced and the eating experience is reduced due to the fruit being picked being insufficiently mature, thereby effectively avoiding the situation where the fruit is unsalable due to the reduced eating experience of the fruit, causing additional economic losses.

[0019] In a third aspect, the present application provides a picking robot, which includes the fruit picking system provided in the second aspect above.

[0020] The present application provides a picking robot that can sense the maturity of fruits and the grasping force applied to fruits using the same tactile sensor through the cooperation of a base, a lower electrode layer, a triboelectric sensor, a piezoresistive sensor, an upper electrode layer and a flexible shell. That is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can sense the maturity of fruits and the grasping force applied to fruits at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the maturity of fruits and the grasping force applied to fruits at the same grasping position point.

[0021] From the above, it can be seen that the fruit picking system, method and picking robot provided by the present application can realize the perception of fruit maturity and the grasping force applied to the fruit using the same tactile sensor through the cooperation of the base, the lower electrode layer, the triboelectric sensor, the piezoresistive sensor, the upper electrode layer and the flexible shell, that is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can perceive the fruit maturity and the grasping force applied to the fruit at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the fruit maturity and the grasping force applied to the fruit at the same grasping position point. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the exploded structure of the dual-modal tactile sensor provided in an embodiment of the present application.

[0023] Figure 2 This is a schematic diagram of the cross-sectional structure of the dual-modal tactile sensor provided in an embodiment of the present application when not under pressure.

[0024] Figure 3 This is a schematic diagram of the cross-sectional structure of the dual-modal tactile sensor provided in an embodiment of the present application when under pressure.

[0025] Figure 4 A schematic diagram of the connection relationship of the fruit picking system provided in an embodiment of the present application.

[0026] Figure numerals: 1. base; 2. lower electrode layer; 21. first plane electrode; 22. first extended electrode; 3. triboelectric sensor; 31. upper sensor; 32. lower sensor; 4. piezoresistive sensor; 5. upper electrode layer; 51. second plane electrode; 52. second extended electrode; 6. flexible shell; 7. gap; 8. manipulator; 9. controller. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0028] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0029] First, as Figure 1-Figure 4 As shown, the present application provides a dual-modal tactile sensor, which includes: Base 1; The lower electrode layer 2 is fixed on the top of the base 1, and includes a first planar electrode 21 and a plurality of first extension electrodes 22. The plurality of first extension electrodes 22 are arranged in a circular array outside the first planar electrode 21 and connected to the first planar electrode 21; The upper electrode layer 5 includes a second planar electrode 51 and a plurality of second extended electrodes 52. The plurality of second extended electrodes 52 are arranged in a circumferential array outside the second planar electrode 51 and connected to the second planar electrode 51. The second extended electrodes 52 are arranged above the first extended electrodes 22 and the number of the second extended electrodes 52 is the same as the number of the first extended electrodes 22. The piezoresistive sensor 4 is disposed between the first planar electrode 21 and the second planar electrode 51 and is used to collect normal force information representing the grasping force applied to the fruit; A plurality of triboelectric sensors 3, arranged in a circular array between the first extension electrode 22 and the second extension electrode 52, for collecting triboelectric information representing the maturity of the fruit; The flexible shell 6 is connected to the top of the upper electrode layer 5 .

[0030] The lower electrode layer 2 and the upper electrode layer 5 of this embodiment are used to transmit the triboelectric information collected by the triboelectric sensor 3 and the normal force information collected by the piezoresistive sensor 4 to the outside of the dual-modal tactile sensor, so as to analyze and process the triboelectric information and the normal force information. The lower electrode layer 2 of this embodiment includes a first planar electrode 21 and a plurality of first extension electrodes 22, and the first extension electrodes 22 are preferably inclined toward the base 1. The number of the triboelectric sensors 3 of this embodiment is multiple, and the multiple triboelectric sensors 3 are respectively connected to different first extension electrodes 22, that is, each triboelectric sensor is connected to a first extension electrode 22. The triboelectric sensor 3 of this embodiment is a sensor that converts mechanical energy into electrical energy based on the coupling effect of friction electrification and electrostatic induction. When the flexible shell 6 contacts the fruit, the mechanical force exerted on the triboelectric sensor 3 will change. At this time, the triboelectric sensor 3 will generate an instantaneous voltage potential (triboelectric information), and the magnitude of the instantaneous voltage potential is related to the change in the mechanical force. Since the hardness of the fruit's skin is related to the maturity of the fruit, the change in the hardness of the fruit's skin will cause the change in the mechanical force exerted on the flexible shell 6 to change, so that the magnitude of the instantaneous voltage potential will change. That is, when the flexible shell 6 contacts the fruit, the magnitude of the instantaneous voltage potential generated by the triboelectric sensor 3 is associated with the maturity of the fruit. Therefore, the triboelectric information of this embodiment can characterize the maturity of the fruit. The piezoresistive sensor 4 of this embodiment is a sensor that converts mechanical energy into electrical signals based on piezoresistive characteristics. After the flexible shell 6 contacts the fruit and the position of the flexible shell 6 does not change, the grasping force applied to the fruit will not change. Since the grasping force applied to the fruit includes at least a normal force, which is perpendicular to the first planar electrode 21, the piezoresistive sensor 4 of this embodiment can measure the normal force in the grasping force applied to the fruit. Since the magnitude of the normal force is related to the magnitude of the grasping force, the normal force information of this embodiment can characterize the grasping force applied to the fruit. The structure of the upper electrode layer 5 of this embodiment is preferably the same as the structure of the lower electrode layer 2 of this embodiment, and will not be discussed in detail here. The flexible shell 6 of this embodiment is connected to the upper electrode layer 5. Therefore, when the flexible shell 6 contacts the fruit, the reaction force of the grasping force applied to the fruit will be transmitted to the triboelectric sensor 3 and the piezoresistive sensor 4 through the flexible shell 6, so that the triboelectric sensor 3 generates triboelectric information and the piezoresistive sensor 4 generates normal force information. The material of the flexible shell 6 of this embodiment is preferably rubber, which is elastic and can be deformed. The flexible shell 6 can play a role in stress dispersion during the contact between the dual-modal tactile sensor and the fruit. Therefore, this embodiment can minimize the damage to the fruit caused by excessive local force on the fruit.It should be understood that since the reaction force of the grasping force applied to the fruit by the flexible shell 6 of this embodiment will be transmitted to the triboelectric sensor 3 and the piezoresistive sensor 4 through the flexible shell 6 to obtain triboelectric information that can characterize the maturity of the fruit and normal force information that can characterize the grasping force applied to the fruit, this embodiment is equivalent to integrating the two functions of sensing the maturity of the fruit and sensing the grasping force applied to the fruit into the same tactile sensor, that is, the fruit maturity sensing position and the grasping force sensing position of this embodiment are the same.

[0031] The present application provides a dual-modal tactile sensor, which can sense the maturity of fruits and the grasping force applied to fruits using the same tactile sensor through the cooperation of a base 1, a lower electrode layer 2, a triboelectric sensor 3, a piezoresistive sensor 4, an upper electrode layer 5 and a flexible shell 6. That is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can sense the maturity of fruits and the grasping force applied to fruits at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the maturity of fruits and the grasping force applied to fruits at the same grasping position point.

[0032] In some preferred embodiments, the triboelectric sensor 3 includes an upper sensor 31 and a lower sensor 32, and the cross-sectional shapes of the upper sensor 31 and the lower sensor 32 are both stepped, and the contact surfaces of the upper sensor 31 and the lower sensor 32 are perpendicular to the top surface of the flexible housing 6. The triboelectric sensor 3 of this embodiment includes an upper sensor 31 and a lower sensor 32, and there is a difference between the selective electron attraction capability of the upper sensor 31 and the selective electron attraction capability of the lower sensor 32, so when the upper sensor 31 is displaced relative to the lower sensor 32, electrons will be transferred and an instantaneous voltage potential will be generated. Since the contact surfaces of the upper sensor 31 and the lower sensor 32 of this embodiment are perpendicular to the top surface of the flexible housing 6, when the top surface of the flexible housing 6 is compressed, the upper sensor 31 will be displaced relative to the lower sensor 32.

[0033] In some preferred embodiments, the material of the upper sensor 31 is polytetrafluoroethylene, and the material of the lower sensor 32 is polyamide. In this embodiment, polytetrafluoroethylene is selected as the material of the upper sensor 31 and polyamide is selected as the material of the lower sensor 32. Since polytetrafluoroethylene and polyamide have a large difference in the ability to attract electrons, this embodiment can increase the triboelectric information generated by the triboelectric sensor 3 when the relative displacement of the upper sensor 31 and the lower sensor 32 is the same, so as to reduce the error of the triboelectric information, thereby effectively improving the accuracy of the fruit maturity obtained by the triboelectric information.

[0034] In some preferred embodiments, the area between the upper electrode layer 5 and the lower electrode layer 2 except the piezoresistive sensor 4 and the triboelectric sensor 3 is filled with an elastic adhesive. Since the area between the upper electrode layer 5 and the lower electrode layer 2 except the piezoresistive sensor 4 and the triboelectric sensor 3 in this embodiment is filled with an elastic adhesive, when the flexible shell 6 is not under pressure, there is a gap 7 between the upper sensor 31 and the lower sensor 32 in this embodiment to ensure that when the flexible shell 6 is under pressure, the upper sensor 31 can be displaced toward the lower sensor 32, and when the flexible shell 6 is under pressure, the elastic adhesive in this embodiment is compressed, and the elastic force generated by the elastic adhesive can reset the flexible shell 6 when the flexible shell 6 changes from a compressed state to a non-compressed state.

[0035] As can be seen from the above, the dual-modal tactile sensor provided by the present application can realize the perception of fruit maturity and the grasping force applied to the fruit using the same tactile sensor through the cooperation of the base 1, the lower electrode layer 2, the triboelectric sensor 3, the piezoresistive sensor 4, the upper electrode layer 5 and the flexible shell 6, that is, the fruit maturity perception position of the present application is the same as the grasping force perception position, that is, the present application can perceive the fruit maturity and the grasping force applied to the fruit at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the fruit maturity and the grasping force applied to the fruit at the same grasping position point.

[0036] Second, as Figure 4 As shown, the present application also provides a fruit picking system, which includes a manipulator 8, a controller 9 and a plurality of dual-modal tactile sensors provided by the first aspect above, wherein the plurality of dual-modal tactile sensors are respectively arranged at different ends of the manipulator 8, and the controller 9 is used to control the movement of the manipulator 8 and its ends so that all the dual-modal tactile sensors are in contact with the fruit, and then control the movement of the manipulator 8 and its ends according to the normal force information to pick the fruit.

[0037] The fruit picking system provided in this embodiment can automatically pick fruits (such as strawberries and grapes). The manipulator 8 of this embodiment can be an existing manipulator 8, and the manipulator 8 has multiple ends. The manipulator 8 can use its ends to grab fruits and drive the grabbed fruits to move. The multiple bimodal tactile sensors of this embodiment are respectively arranged at different ends of the manipulator 8, that is, each end of the manipulator 8 corresponds to a bimodal tactile sensor. The controller 9 of this embodiment is electrically connected with the manipulator 8 and the bimodal tactile sensor. The controller 9 can identify and locate the fruit based on the existing visual recognition technology and control the manipulator 8 and its end to move according to the position of the fruit, so that all bimodal tactile sensors are in contact with the fruit. The normal force applied by the flexible shell 6 to the fruit is used to clamp and fix the fruit on the manipulator 8. If the fruit needs to be picked from the plant, the manipulator 8 needs to be used to drive the fruit to move in a tangential direction, and the tangential direction can be determined by the normal force information. Therefore, this embodiment can pick the fruit by controlling the manipulator 8 and its end to move according to the normal force information.

[0038] In some preferred embodiments, the normal force information includes a normal vector, and the controller 9 controls the manipulator 8 and its end to move according to the normal force information to pick the fruit, and the process includes: A1, controlling the end of the manipulator 8 to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches a preset force to grasp the fruit; A2. Obtaining a tangential vector based on all triboelectric information collected by at least one dual-modal tactile sensor when the end of the manipulator 8 moves along the normal vector; A3. Control the manipulator 8 to move along the tangential vector to pick the fruit from the plant.

[0039] Because before picking the fruit, this embodiment only controls the manipulator 8 to move until the dual-modal tactile sensor contacts the fruit. At this time, the clamping force on the fruit is relatively small. If the fruit is picked directly, the fruit may accidentally fall due to the small clamping force on the fruit, and the fruit may be damaged or even destroyed. Therefore, before picking the fruit from the plant, this embodiment needs to increase the clamping force on the fruit by controlling the end of the manipulator 8 to move along the normal vector collected by the dual-modal tactile sensor connected to the end, so as to grasp the fruit firmly and avoid the fruit from accidentally falling. Since the multiple first extension electrodes 22 in the same dual-modal tactile sensor are arranged in a circular array on the periphery of the first planar electrode 21, and the multiple triboelectric sensors 3 are respectively connected to different first extension electrodes 22, if the mechanical force applied to the flexible shell 6 is composed of only normal force, the triboelectric information collected by the multiple triboelectric sensors 3 in the same dual-modal tactile sensor should be the same; if the mechanical force applied to the flexible shell 6 is composed of normal force and tangential force, the triboelectric information collected by the multiple triboelectric sensors 3 in the same dual-modal tactile sensor is different, and the triboelectric information close to the force point is greater than the triboelectric information far from the force point, that is, the difference in triboelectric information can reflect the grasping force vector, and the tangential vector can be obtained by splitting the grasping force vector. Therefore, step A2 can obtain the tangential vector based on all the triboelectric information collected by at least one dual-modal tactile sensor when the end of the manipulator 8 moves along the normal vector.

[0040] In some preferred embodiments, step A1 comprises: A11. Obtaining fruit type information, and obtaining a preset strength according to the fruit type information and a third preset conversion relationship; A12, controlling the end of the manipulator 8 to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches a preset force to firmly grasp the fruit.

[0041] Step A11 can obtain fruit type information by using existing visual recognition technology to identify fruits. Since different types of fruits can withstand different maximum external forces, for example, the maximum external force that apples can withstand is greater than the maximum external force that strawberries can withstand, before executing step A12, this embodiment needs to obtain fruit type information and obtain a preset force based on the fruit type information and the third preset conversion relationship to avoid the situation where some types of fruits are damaged or even destroyed due to excessive force due to using the same preset force to grasp different types of fruits. The third preset conversion relationship of this embodiment is a pre-constructed mapping relationship between fruit type and preset force. This embodiment can obtain the corresponding preset force from the third preset conversion relationship according to the fruit type information by data extraction.

[0042] In some preferred embodiments, each bimodal tactile sensor includes a first triboelectric sensor group and a second triboelectric sensor group, each of the first triboelectric sensor group and the second triboelectric sensor group includes two triboelectric sensors 3 disposed opposite to each other, and step A2 includes: A21, obtaining a first vector corresponding to each dual-modal tactile sensor according to a difference in triboelectric information of two triboelectric sensors 3 in the first triboelectric sensor group and a first preset conversion relationship; A22, obtaining a second vector corresponding to each dual-modal tactile sensor according to a difference in triboelectric information of two triboelectric sensors 3 in the second triboelectric sensor group and a first preset conversion relationship; A23. Obtaining a three-dimensional force vector corresponding to each dual-modal tactile sensor according to the first vector and the corresponding second vector, and obtaining a preliminary tangential vector corresponding to each dual-modal tactile sensor according to the three-dimensional force vector; A24. Generate a tangent vector based on all preliminary tangent vectors.

[0043] Since the number of the triboelectric sensors 3 in this embodiment is four, the first triboelectric sensor group and the second triboelectric sensor group in this embodiment each include two oppositely arranged triboelectric sensors 3, that is, the connecting line of the two triboelectric sensors 3 in the first triboelectric sensor group is perpendicular to the connecting line of the two triboelectric sensors 3 in the second triboelectric sensor group. When the grasping force exerted on the fruit is composed of normal force and tangential force, there is a difference between the triboelectric information of the two triboelectric sensors 3 in the first triboelectric sensor group and the triboelectric information of the two triboelectric sensors 3 in the second triboelectric sensor group, and the difference can reflect the vector of the grasping force in the XOZ plane or the YOZ plane. Therefore, this embodiment can obtain a first vector according to the difference in triboelectric information of the two triboelectric sensors 3 in the first triboelectric sensor group and a first preset conversion relationship, and obtain a second vector according to the difference in triboelectric information of the two triboelectric sensors 3 in the second triboelectric sensor group and the first preset conversion relationship. The first preset conversion relationship can be a pre-constructed mapping relationship between the difference in triboelectric information of two oppositely arranged triboelectric sensors 3 and a vector. Step A23 can obtain the three-dimensional force vector corresponding to each dual-modal tactile sensor (the vector corresponding to the grasping force applied to the fruit at the location where the dual-modal tactile sensor is located) by adding the first vector and the corresponding second vector. Step A24 can obtain the tangential vector by adding all the preliminary tangential vectors. It should be understood that when the mechanical force applied to the dual-modal tactile sensor is composed of a normal force and a tangential force, the three-dimensional force vector is composed of a normal vector and a tangential vector, and the normal force information includes the normal vector, that is, the normal vector is a known value, so this embodiment can obtain the corresponding preliminary tangential vector based on the three-dimensional force vector and its corresponding normal vector.

[0044] In some preferred embodiments, step A2 further includes the following steps performed before step A21: A25, obtaining fruit maturity information according to the triboelectric information collected by the triboelectric sensor 3 when the dual-modal tactile sensor initially contacts the fruit and a second preset conversion relationship; A26, analyzing the fruit maturity information to see whether it reaches the preset maturity. If so, executing step A21; if not, controlling the end of the manipulator 8 to release the fruit.

[0045] Since the magnitude of the triboelectric information generated by the triboelectric sensor 3 when the dual-modal tactile sensor makes initial contact with the fruit is associated with the maturity of the fruit, the second preset conversion relationship of this example can be a pre-constructed mapping relationship between triboelectric information and maturity. This embodiment can obtain the corresponding fruit maturity information from the second preset conversion relationship according to the triboelectric information by means of data extraction. Step A25 can obtain the fruit maturity information according to any one of the triboelectric information and the second preset conversion relationship, and step A26 can analyze whether the fruit maturity information reaches the preset maturity by judging whether the fruit maturity information is greater than or equal to the preset maturity. This embodiment is equivalent to picking only fruits that have reached the preset maturity, so this embodiment can effectively avoid the situation where the nutritional value of the fruit is reduced and the eating experience is reduced due to the insufficient maturity of the picked fruit, thereby effectively avoiding the situation where the fruit is unsalable due to the reduced eating experience of the fruit, causing additional economic losses.

[0046] In some preferred embodiments, step A25 includes: A251. Obtain fruit maturity information based on the average value of all triboelectric information collected by the triboelectric sensor 3 when the dual-modal tactile sensor initially contacts the fruit and a second preset conversion relationship.

[0047] Since this embodiment obtains the fruit maturity information based on the average value of all triboelectric information collected by the triboelectric sensor 3 when the dual-modal tactile sensor initially contacts the fruit and the second preset conversion relationship, this embodiment is equivalent to picking only the fruits whose overall maturity reaches the preset maturity, thereby effectively avoiding the situation where only the maturity of a local area of ​​the picked fruit reaches the preset maturity.

[0048] As can be seen from the above, the fruit picking system provided by the present application can realize the perception of fruit maturity and the grasping force applied to the fruit using the same tactile sensor through the cooperation of the base 1, the lower electrode layer 2, the triboelectric sensor 3, the piezoresistive sensor 4, the upper electrode layer 5 and the flexible shell 6, that is, the fruit maturity perception position of the present application is the same as the grasping force perception position, that is, the present application can perceive the fruit maturity and the grasping force applied to the fruit at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the fruit maturity and the grasping force applied to the fruit at the same grasping position point.

[0049] In a third aspect, the present application provides a picking robot, which includes the fruit picking system provided in the second aspect above.

[0050] A picking robot provided in the present application includes a fruit picking system provided in the second aspect above. The principle of the picking robot provided in this embodiment is the same as the principle of the fruit picking system provided in the second aspect above, and will not be discussed in detail here.

[0051] From the above, it can be seen that the fruit picking system, method and picking robot provided by the present application can realize the perception of fruit maturity and the grasping force applied to the fruit using the same tactile sensor through the cooperation of the base 1, the lower electrode layer 2, the triboelectric sensor 3, the piezoresistive sensor 4, the upper electrode layer 5 and the flexible shell 6, that is, the fruit maturity sensing position of the present application is the same as the grasping force sensing position, that is, the present application can perceive the fruit maturity and the grasping force applied to the fruit at the same grasping position point. Therefore, the present application can effectively solve the problem of insufficient reliability of the dynamic information and grasping force obtained due to the inability to detect the dynamic information characterizing the fruit maturity and the grasping force applied to the fruit at the same grasping position point.

[0052] In the embodiments provided in the present application, it should be understood that, herein, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0053] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A dual-modal tactile sensor, characterized in that: The dual-modal tactile sensor comprises: Base; A lower electrode layer, fixed on the top of the base, comprising a first planar electrode and a plurality of first extension electrodes, wherein a circumferential array of the plurality of first extension electrodes is arranged outside the first planar electrode and connected to the first planar electrode; an upper electrode layer, comprising a second planar electrode and a plurality of second extension electrodes, wherein the plurality of second extension electrodes are arranged in a circumferential array outside the second planar electrode and connected to the second planar electrode, and the second extension electrodes are arranged above the first extension electrodes and the number of the second extension electrodes is the same as the number of the first extension electrodes; A piezoresistive sensor is disposed between the first planar electrode and the second planar electrode and is used to collect normal force information representing a grasping force applied to the fruit; A plurality of triboelectric sensors, arranged in a circular array between the first extension electrode and the second extension electrode, for collecting triboelectric information indicative of the maturity of the fruit; The flexible shell is connected to the top of the upper electrode layer.

2. The dual-modal tactile sensor according to claim 1, characterized in that: The triboelectric sensor comprises an upper sensor and a lower sensor, the cross-sections of the upper sensor and the lower sensor are both stepped, and the contact surfaces of the upper sensor and the lower sensor are perpendicular to the top surface of the flexible shell.

3. The dual-modal tactile sensor according to claim 2, characterized in that: The material of the upper sensor is polytetrafluoroethylene, and the material of the lower sensor is polyamide.

4. The dual-modal tactile sensor according to claim 2, characterized in that: An area between the upper electrode layer and the lower electrode layer except for the piezoresistive sensor and the triboelectric sensor is filled with an elastic adhesive.

5. A fruit picking system, characterized in that: The fruit picking system includes a manipulator, a controller and a plurality of bimodal tactile sensors as described in any one of claims 1 to 4, wherein the plurality of bimodal tactile sensors are respectively arranged at different ends of the manipulator, and the controller is used to control the movement of the manipulator and its ends so that all the bimodal tactile sensors are in contact with the fruit, and then control the movement of the manipulator and its ends according to normal force information to pick the fruit.

6. The fruit picking system according to claim 5, characterized in that: The normal force information includes a normal vector, and the controller controls the movement of the manipulator and its end according to the normal force information to pick the fruit, and the process includes: A1, controlling the end of the manipulator to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches a preset force to grasp the fruit; A2. Obtaining a tangential vector according to all triboelectric information collected by at least one of the dual-modal tactile sensors when the end of the manipulator moves along the normal vector; A3. Control the manipulator to move along the tangential vector to pick the fruit from the plant.

7. The fruit picking system according to claim 6, characterized in that: Step A1 includes: A11, obtaining fruit type information, and obtaining a preset strength according to the fruit type information and a third preset conversion relationship; A12. Control the end of the manipulator to move along the normal vector collected by the dual-modal tactile sensor connected to the end until the modulus of the normal vector reaches the preset force to grasp the fruit.

8. The fruit picking system according to claim 6, characterized in that: Each of the dual-modal tactile sensors includes a first triboelectric sensor group and a second triboelectric sensor group, wherein the first triboelectric sensor group and the second triboelectric sensor group each include two triboelectric sensors disposed opposite to each other, and step A2 includes: A21. Obtaining a first vector corresponding to each of the dual-modal tactile sensors according to a difference in triboelectric information of two triboelectric sensors in the first triboelectric sensor group and a first preset conversion relationship; A22. Obtaining a second vector corresponding to each of the dual-modal tactile sensors according to a difference in triboelectric information of two triboelectric sensors in the second triboelectric sensor group and the first preset conversion relationship; A23. Obtaining a three-dimensional force vector corresponding to each of the dual-modal tactile sensors according to the first vector and the corresponding second vector, and obtaining a preliminary tangential vector corresponding to each of the dual-modal tactile sensors according to the three-dimensional force vector; A24. Generate a tangent vector based on all the preliminary tangent vectors.

9. The fruit picking system according to claim 8, characterized in that: Step A2 also includes the following steps performed before step A21: A25, obtaining fruit maturity information according to the triboelectric information collected by the triboelectric sensor when the dual-modal tactile sensor initially contacts the fruit and a second preset conversion relationship; A26, analyzing whether the fruit maturity information reaches the preset maturity, if so, executing step A21, if not, controlling the end of the manipulator to release the fruit.

10. A picking robot, characterized in that: The picking robot comprises a fruit picking system as described in any one of claims 5-9.

Images