Surface flexible tactile sensor
Through the combination of the double-layer flexible layer structure and camera of the surface-shaped flexible tactile sensor, the problems of low resolution and poor dynamic performance of traditional equipment are solved, and high resolution and six-dimensional torque measurement is achieved, which is suitable for pressure distribution in multiple industries. analyze.
Patent Information
- Application Number
- CN202210387088.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-04-12
AI Technical Summary
When traditional pressure plate equipment measures the pressure distribution state when a person's feet stands or moves, the resolution is low and can only provide two-dimensional pressure. It has poor dynamic performance and cannot effectively capture three-dimensional torque information.
A surface-shaped flexible tactile sensor was designed, using a bionic skin-dermal double-layer flexible layer structure, combined with a camera, using light-transmitting and flexible materials to capture deformation, and improving resolution through silver pigment auxiliary reflective layer, which can Sensing six-dimensional contact force.
It realizes digital measurements of high resolution, six-dimensional force, and high dynamic performance, and can accurately capture three-dimensional pressure and torque information. It is suitable for sports, auxiliary medical care and shoemaking industries.
Smart Images

Figure CN114659678B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure belongs to sensor technology, and in particular to a surface-shaped flexible tactile sensor, which is mainly used in sports, auxiliary medical treatment, shoemaking and other fields. Background Art
[0002] In the sports, auxiliary medical, shoemaking and other industries, a "pressure plate" device is needed to measure the pressure distribution state of a person's feet when standing or moving. The traditional pressure plate mainly uses a resistor or capacitor array, which can provide a two-dimensional pressure field perpendicular to the plane. The problem is that due to the hardware limitations of the capacitor resistor array, the resolution is low, only two-dimensional pressure can be provided, and the sensor signal tailing causes poor dynamic performance of the sensor. Summary of the invention
[0003] In order to solve the above technical problems, the present disclosure discloses a surface-shaped flexible tactile sensor, which is characterized by:
[0004] The surface-shaped flexible tactile sensor is used to bear the tactile sensation and force of stepping on the feet or other types;
[0005] The surface flexible tactile sensor comprises:
[0006] A double-layer flexible structure consisting of a bionic epidermis layer and a dermis layer and a camera, wherein:
[0007] The camera is used to capture the deformation of the double-layer flexible layer structure;
[0008] The double-layer flexible layer structure is based on a material having light transmittance and softness, wherein:
[0009] According to its softness, the double-layer flexible layer structure is deformable;
[0010] According to its light transmittance, the camera is able to capture the deformation of the double-layer flexible layer structure;
[0011] and,
[0012] The skin layer is mixed with or coated with silver pigment as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure.
[0013] Preferably,
[0014] The surface-shaped flexible tactile sensor is in the form of a rectangular cube with a certain height. The camera is arranged on a side surface of the cube for side imaging.
[0015] Preferably,
[0016] The epidermis layer and the dermis layer are both made of flexible materials based on silicone rubber.
[0017] Preferably,
[0018] The thickness of the dermis layer is greater than the thickness of the epidermis layer. Preferably, the thickness of the dermis layer is 20 mm, and the thickness of the epidermis layer is 3 mm.
[0019] Preferably,
[0020] Close to the epidermis, a light PCB soft board is arranged on the four sides of the surface flexible tactile sensor. Preferably, red, blue, green and white LED lights are arranged on the four sides respectively.
[0021] Preferably,
[0022] The dermis layer is bonded to the epidermis layer by its own adhesiveness.
[0023] Preferably,
[0024] The double-layer flexible layer structure is preferably designed as an easily replaceable consumable flexible layer.
[0025] Preferably,
[0026] The marking can be performed by piercing the interior of the dermis with a needle coated with pigment;
[0027] When the flexible tactile sensor is working, the camera captures the deformation of the double-layer flexible layer structure and further senses its deformation.
[0028] Preferably,
[0029] When the flexible tactile sensor is subjected to a contact force due to contact,
[0030] The skin layer is used to fit a higher resolution, so that the deformation of the double-layer flexible layer structure has the resolution of the skin layer.
[0031] Preferably,
[0032] When the flexible tactile sensor is used, the deformation of the dermis is sensed to express the six-dimensional contact force: in addition to the three-dimensional pressure vector field distributed on the surface of the epidermis as the contact layer, it also includes a three-dimensional torque vector field.
[0033] In this way, the present invention realizes a surface flexible tactile sensor with an innovative structure and good performance, which can be used in sports, auxiliary medical care or shoemaking industries to measure the pressure distribution state of human feet when standing or moving, and can achieve high-resolution, six-dimensional force, and high-dynamic performance digital measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of a sensor structure shown in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0035] In order to make those skilled in the art understand the technical solutions disclosed in the present disclosure, the following will be combined with the embodiments and related drawings. Figure 1 , the technical solutions of various embodiments are described, and the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The terms "first", "second", etc. used in the present disclosure are used to distinguish different objects, rather than to describe a specific order. In addition, "including" and "having" and any variations thereof are intended to cover and not exclude inclusion. For example, a process, or method, or system, or product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, systems, products or devices.
[0036] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present disclosure. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0037] In one embodiment, the present disclosure discloses a surface-shaped flexible tactile sensor, wherein:
[0038] The surface-shaped flexible tactile sensor is used to carry the tactile sensation and force of stepping on the feet or other types;
[0039] The surface flexible tactile sensor comprises:
[0040] A double-layer flexible structure consisting of a bionic epidermis layer and a dermis layer and a camera, wherein:
[0041] The camera is used to capture the deformation of the double-layer flexible layer structure;
[0042] The double-layer flexible layer structure is based on a material having light transmittance and softness, wherein:
[0043] According to its softness, the double-layer flexible layer structure is deformable;
[0044] According to its light transmittance, the camera is able to capture the deformation of the double-layer flexible layer structure;
[0045] and,
[0046] The skin layer is mixed with or coated with silver pigment as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure.
[0047] Exemplarily, the hardness of the epidermis layer is 20 on the Shore A hardness scale, and the hardness of the dermis layer is 10.
[0048] For the above-mentioned embodiment, a new structure of a surface tactile sensor is innovatively proposed, and its innovation is firstly reflected in the double-layer flexible layer structure. It can be understood that the epidermis layer is used to contact the contact surface, and serves as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure. When capturing, the camera not only captures the deformation of the double-layer flexible layer structure through the outermost epidermis layer of the sensor, but also further captures the deformation of the double-layer flexible layer structure through the dermis layer. It should be noted that when it is used in sports, auxiliary medical treatment, and shoemaking industries, it can measure tactile data for real people, or it can be measured for dummies to assist sports training or sports medicine or other auxiliary medical treatment, or for shoemaking purposes.
[0049] In detail, its innovation is mainly reflected in the following points:
[0050] First, the innovation is reflected in the double-layer flexible layer structure. It can be understood that the epidermis layer is used to contact the contact surface. The epidermis layer not only provides a tactile feel (Note: It can be understood that by selecting or designing the epidermis layer, different levels of tactile feel can be provided), but also the epidermis layer is provided with the reflective layer so that the light along the dermis layer and the epidermis layer is reflected, and then enters the camera along the direction of the epidermis layer and the dermis layer. For example, on the side close to the dermis layer, a silver pigment is mixed on a surface somewhere inside the epidermis layer as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure; of course, the silver pigment can also be mixed or applied on the outer surface of the epidermis layer, or on the inner surface of the outer surface of the epidermis layer, as a reflective layer, wherein when mixed or applied on the outer surface, the wear resistance and durability of the reflective layer are required to be higher;
[0051] Secondly, since the double-layer flexible layer structure is a bionic epidermis-dermis layer, it can physically imitate human skin, and even if better materials are used, it can exceed the sensitivity of human skin;
[0052] Third, since the double-layer flexible layer structure is light-transmissive, when it is deformed, the light passing through the double-layer flexible layer structure changes in the optical path, and the light is captured by the camera, so that the tactile sensor realizes the perception of touch. Furthermore, if the tactile sensor also includes a processor, the processor can directly process the image obtained by the camera and directly obtain specific tactile information by analyzing the image.
[0053] It should be noted that when the camera captures light, the camera not only captures the deformation of the double-layer flexible layer structure through the outermost epidermis layer of the sensor, but also further through the dermis layer: when the force applied to the outer surface of the epidermis layer causes the epidermis layer to deform significantly, compared with the initial undeformed state, the light and its optical path passing through the double-layer flexible layer structure of the epidermis layer-dermis layer change, and these changes are captured by the camera. In other words, with regard to the tactile sensor disclosed in the present invention, its working principle is that since the double-layer flexible layer structure has light transmittance, when it is deformed, the light passing through the double-layer flexible layer structure changes in the optical path, so that sensing the change in the optical path enables the tactile sensor to realize the perception of touch. It can be understood that after sensing the deformation of the double-layer flexible layer structure, the force applied to the tactile sensor can be classified and identified according to different sensing results, and if the object contacted by the tactile sensor itself has surface defects, such as protrusions or depressions, the contact surface can also be classified and identified after sensing the deformation of the double-layer flexible layer structure.
[0054] In another embodiment,
[0055] The tactile sensor further includes a light source;
[0056] The camera is used to capture the deformation of the double-layer flexible layer structure;
[0057] The double-layer flexible layer structure is based on a material with light transmittance and softness, wherein the epidermis layer includes a reflective layer to assist the camera in capturing the light that is irradiated by the light source toward the double-layer flexible layer structure and reflected back to the camera, thereby sensing the deformation of the double-layer flexible layer structure.
[0058] In another embodiment,
[0059] The surface-shaped flexible tactile sensor is in the form of a rectangular cube with a certain height. The camera is arranged on a side surface of the cube for side imaging.
[0060] It can be understood that the advantage of this design is that the camera is closer to the soft rubber pad and the structure is thinner when the lens imaging angle is determined. The camera refresh rate is 120Hz to ensure the dynamic performance of the captured image.
[0061] As mentioned above, the working principle of the tactile sensor disclosed in the present invention is that since the double-layer flexible layer structure is light-transmissive, when it is deformed, the light passing through the double-layer flexible layer structure changes in the optical path. Thus, sensing the change in the optical path enables the tactile sensor to perceive touch.
[0062] In another embodiment,
[0063] The epidermis layer and the dermis layer are both made of flexible materials based on silicone rubber.
[0064] Typically, the silicone rubber material is a chemical silicone rubber material. It should be noted that the flexible material with light transmittance and softness can also be further expanded to other materials besides silicone rubber materials. Exemplarily, the epidermis and dermis are both made of a flexible material based on PDMS siloxane material.
[0065] Furthermore, the epidermis and dermis can even be made of flexible materials based on different materials to flexibly adjust softness and light transmittance.
[0066] In another embodiment,
[0067] The thickness of the dermis layer is greater than the thickness of the epidermis layer. Preferably, the thickness of the dermis layer is 20 mm, and the thickness of the epidermis layer is 3 mm.
[0068] In another embodiment,
[0069] Close to the epidermis, a light PCB soft board is arranged on four sides of the surface flexible tactile sensor.
[0070] More preferably, red, blue, green and white LED lights are respectively provided on the four sides. It can be understood that the four sides / sides are each of a color, which is not only more conducive to light color mixing, but also has a deeper reason. The figurative understanding is that since each color of light in the four-color light can be regarded as a variable, this involves four variables, and the tactile sensor aims to perceive at least three-dimensional force (the six-dimensional contact force is also described later, see later for details), wherein the three dimensions involve x, y, and z, and any change in the four-color light forms a four-to-three mapping relationship with the change in the three dimensions of force. Therefore, the four-color light source in the present disclosure is not only more adaptable to the four sides, but also compared with the three-color light, on the premise of adding a light and its corresponding information, it is more conducive to quickly solving the data in the three dimensions of xyz. Therefore, the camera in the present disclosure can be a single camera, a monocular camera. In addition, the cost of LED light sources is much lower than that of cameras, which inevitably means that the present disclosure can significantly reduce hardware costs.
[0071] In another embodiment,
[0072] The dermis layer is bonded to the epidermis layer by its own adhesiveness.
[0073] In this way, the two can be bonded by itself instead of other glue. Of course, as for the edge part after the two are bonded, since it hardly affects the operation of the sensor, if the edge part needs to be more tightly bonded, transparent soft glue can be applied on the edge or its periphery.
[0074] In another embodiment,
[0075] The double-layer flexible layer structure is preferably designed as an easily replaceable consumable flexible layer.
[0076] In another embodiment,
[0077] The marking can be performed by piercing the interior of the dermis with a needle coated with pigment;
[0078] When the flexible tactile sensor is working, the camera captures the deformation of the double-layer flexible layer structure and further senses its deformation.
[0079] For example, along the thickness direction of the dermis, a needle brush coated with paint is used to pierce the interior of the dermis in batches at one time, and the density and height of the pins are set according to the density setting of the mark. It can be understood that the color of the pigment is for the convenience of sensing when the image data captured by the camera is processed later, and the present disclosure is not limited by a specific color. In theory, in addition to the thickness direction, other directions (typically, any direction in the XYZ coordinate system) can be used, as long as the image data captured by the camera is processed later, It is convenient to sense its deformation. It can be understood that even different coordinate systems can process related data through coordinate transformation. It should be noted that the thickness direction is the preferred direction, and the thickness direction refers to the direction from the epidermis to the dermis or the opposite direction. The marking on the thickness is conducive to analyzing the deformation when the image data is processed later.
[0080] What information is contained in the image data is the key to understanding the present disclosure. It can be found that, relative to the present embodiment, the other previous embodiments do not involve any marking means. Taking the RGB three-color light mentioned in the previous text as an example, it can correspond to the information of the three dimensions of XYZ, and the three colors must constitute a mapping relationship with the three dimensions of space. Then, this itself means that the technical solution of the present disclosure can obtain the geometric deformation of the double-layer flexible layer structure after deformation at a certain moment: on a cross-section, this often involves surface changes and thus presents a curved surface. Compared with the initial state of the double-layer flexible structure layer before deformation, for the deformation state at the current moment, in a spatial coordinate system, the final displacement components in the X, Y, and Z directions can be obtained. The reason why it can be obtained is precisely because the three colors of RGB light and the three dimensions constitute a three-to-three mapping, which meets the conditions for obtaining.
[0081] It should be noted that without the above-mentioned mark, assuming that the tactile sensor is in the initial non-deformed state at time 0, then, by using the above-mentioned embodiment, by processing the image data obtained by the tactile sensor, it is only possible to obtain: at time t1, relative to the initial non-deformed state, the deformation of the flexible layer structure. In other words, without the above-mentioned mark, the technical solution of the above-mentioned embodiment can only obtain the geometric state of the tactile sensor after deformation at any time, which is a static geometric feature. Of course, it can be understood that obtaining the static geometric features at any time has realized the perception function of the tactile sensor and can provide corresponding data. In other words, if only the surface geometric features need to be sensed, then, without the need for marking, a three-color RGB light source and a monocular camera can be used.
[0082] However, special attention should be paid: in the absence of the marking means described in this embodiment, the parameters obtained by the previous embodiment are only static geometric features at the current moment. This is because when the tactile sensor changes, it is mainly manifested as a change in the curved surface. The epidermis layer in the flexible layer structure and even the dermis layer in some cases undergo obvious outward extension and inward compression. Compared with the initial non-deformed state, after deformation, new contact points are extended on a certain curved surface, or some old contact points are squeezed to a position that cannot be sensed. This results in: even the initial non-deformed state cannot serve as a benchmark for any displacement changes before and after, and the displacement change cannot be defined. The root cause lies in: there is a lack of a meaningful benchmark from moment t1 to moment t2.
[0083] Back to the present embodiment, the reason why the present embodiment is marked is precisely to solve the problem that parameters such as the displacement change caused by the deformation of the flexible layer structure from time t1 to time t2 involved in the above example cannot be obtained.
[0084] The key to the problem is that the image data needs to contain reference information about the space. This embodiment prefers marks in the thickness direction, and such marks obviously become a kind of auxiliary information (note: at least including information in the z direction), and each mark with color can establish a positioning spatial information in the flexible layer structure. Compared with other embodiments mentioned above, these marks can become additional spatial reference information, thereby solving the problem that parameters such as the displacement change caused by the deformation of the flexible layer structure from time t1 to time t2, and the average speed in each dimension corresponding to the displacement change cannot be obtained.
[0085] Furthermore, in the case of implementing marking, between the time t1 and the time tN, whether the time t1 is the time 0 or another time, during this time interval, the present disclosure can further obtain the contact force, to be precise, at least the three-dimensional contact force can be further obtained (the following text also involves the six-dimensional contact force, see the following text for details). The reason is that when the force is applied to the tactile sensor, the deformation of the epidermis itself involves a process, which is from the time t1 to the time tN, and in this process, it involves multiple intermediate moments ti (i is equal to 2, 3, 4, etc.). If the initial position of the contact is a point, it will soon become multiple contact points, and this process is often accompanied by the outer surface of the epidermis changing in the sense of the surface. Then, if the contact force is obtained, it must involve changes in multiple intermediate time periods, such as the change between the time t2 and the time t1, the change between the time t3 and the time t2, etc. For the outer surface of the epidermis, this is obviously a continuous surface change process, and the force action process is dynamic and involves force components in the three directions of XYZ. Since the marking is equivalent to introducing additional spatial information, more precisely, a kind of positioning information, this enables the present disclosure to obtain the three-dimensional contact force during the deformation of the double-layer flexible layer structure. The colored points marked in this embodiment become reference points or anchor points that can always be found during the change of the outer surface curve of the entire epidermis layer. At any moment, the spatial state of all the marks can always be known, and at the next moment, the spatial state of all the marks can still be known. The light enters the epidermis from the dermis, passes through the reflective layer, and then enters the camera along the direction from the epidermis to the dermis. From time t1 to time t2, it can reflect the changes of all the positioning marks, and the amount of change and the speed of the change have a mapping relationship with the force applied to the double-layer flexible layer structure of the tactile sensor. Therefore, the tactile sensor can also be used to obtain the three-dimensional contact force. Naturally, the dynamic geometric deformation process can also be obtained at this time.
[0086] In addition, as long as the deformation ability of the double-layer flexible layer structure is fine enough, the resolution ability of the camera has no upper limit. In theory, the interval between time t1 and time t2 can be small enough, and the accuracy of the contact force obtained by the present invention can be continuously improved.
[0087] In another embodiment,
[0088] Using a laser or a needle, marking the interior of the dermis with a certain outline, pattern or color block for marking;
[0089] When the flexible tactile sensor is working, the camera captures the deformation of the double-layer flexible layer structure and further senses its deformation.
[0090] Compared with the previous embodiment, the marking of this embodiment is not a dot, but a certain contour or pattern or color block. It can be understood that by sensing the corresponding contour or pattern or color block in the later image processing process, such as a circle, or other contours, or the shape corresponding to a certain pattern, the present embodiment can also sense the contact force. The contour may refer to the outer contour in a narrow sense. In the present disclosure, it is further extended that whether it is marking by dots in the previous embodiment or marking by a certain contour or pattern or color block in the present embodiment, the marking method is not limited, and even the color block can be a random color block and the pattern can be a random pattern, because the inventor has realized through various marking embodiments that the camera can always see the mark and the change of the mark between any two moments before and after.
[0091] In another embodiment,
[0092] From the outer surface of the epidermis to a certain depth of the dermis, the epidermis is penetrated in a straight line direction and pierced into the dermis to form a linear mark with a certain depth. For example, a needle with a color on its body is directly pierced from the outer surface of the epidermis to a certain depth of the dermis, and after the needle is pulled out, a linear mark of the color is formed in the double-layer flexible layer structure.
[0093] It can be understood that this also helps to obtain the three-dimensional contact force, and this method can better observe: when the outer surface of the epidermis is exerted with force, the specific changes that occur inside the epidermis and dermis due to deformation.
[0094] Each of the above marks can be arranged in an array on the double-layer flexible layer structure, or can be arranged in a randomly distributed manner on the double-layer flexible layer structure, or a variety of different types of marks can be mixed and used.
[0095] As far as the different markings are concerned, it should be noted that:
[0096] Color blocks are larger than dots, so when the deformation amplitude is large, color blocks have more advantages than dots. This is because when the deformation amplitude is large, the color blocks are more sensible than dots, and can also meet the sensing of large deformations. In extreme cases, dots may not be sensed at a certain moment or at certain moments due to the large deformation amplitude. If both dots and color blocks can be sensed, then if the mark is a dot, the resolution of the tactile sensor is higher than that of the color block. The reason is precisely that the color block is larger than the dot in size, so the tactile sensor of the color block cannot be fine enough. In addition, the large size of the color block also increases the computational complexity of image processing. Furthermore, the mixed use of multiple marks can balance the sensibility of the mark and the resolution of the tactile sensor.
[0097] In addition, color blocks of different colors are more perceptible than color blocks of a single color, because not only do the color blocks constitute a kind of positioning information, but the different colors themselves also constitute a kind of additional auxiliary information.
[0098] In another embodiment,
[0099] The thickness and softness of the skin layer are constrained by the computational accuracy of the static geometric features of the double-layer flexible layer structure at any moment.
[0100] It can be understood that the thinner and softer the epidermis is, the more accurately the static geometric features, or static geometric deformation, at any moment can be calculated.
[0101] In another embodiment,
[0102] When the flexible tactile sensor is subjected to a contact force due to contact,
[0103] The skin layer is used to fit a higher resolution, so that the deformation of the double-layer flexible layer structure has the resolution of the skin layer.
[0104] It can be understood that the deformation ability of the epidermis itself is related to its resolution, and the finer the deformation ability, the higher the resolution.
[0105] In another embodiment,
[0106] When the flexible tactile sensor is used, the deformation of the double-layer flexible layer structure is sensed to express the six-dimensional contact force: in addition to the three-dimensional pressure vector field distributed on the surface of the epidermis as the contact layer, it also includes a three-dimensional torque vector field. For example, the corresponding torque in the X, Y, and Z coordinate systems can be understood figuratively as follows: the six-dimensional force refers to the force in the three-dimensional coordinate axis and the rotational force around the three-dimensional coordinate axis (Note: In this disclosure, the rotational force around the three-dimensional coordinate axis is understood as torque).
[0107] It should be noted that this further strengthens the innovative characteristics of the disclosed technology, which helps to achieve the sensing of the six-dimensional force field. It can be understood that in order to optimize this performance, the toughness, hardness, thickness and material of the epidermis and dermis can be further adjusted.
[0108] In another embodiment, the light transmittance of the silicone rubber material is 95%.
[0109] In another embodiment, Figure 1 As shown,
[0110] The present disclosure also discloses other auxiliary components in addition to the double-layer flexible layer structure and the camera, for example, it further relates to related components of a light similar to a flash light or a fill light, and inorganic transparent materials on the optical path of the transmitted light, as well as some brackets, housings, etc., wherein:
[0111] The surface of the double-layer flexible layer structure is sticky, and it uses the stickiness of its own surface to bond with inorganic transparent materials, such as glass, quartz, etc.; preferably, the double-layer flexible layer structure can be realized as a double-layer soft pad (or double-layer soft rubber pad) with a breaking elongation of 100%. When it is made, it is formed using a soft pad mold, and is formed into a soft pad by heating and baking after vacuuming; and the dermis layer is relatively soft and has a certain degree of stickiness, so we only apply transparent soft glue on the edge of the dermis-epidermis contact surface, and the middle part is naturally bonded after the bubbles are squeezed out. It can be understood that various bonding or bonding mentioned in the present disclosure may require the exclusion of bubbles, including the exclusion of bubbles by the double-layer flexible layer structure and inorganic transparent materials, such as glass, quartz, etc. through the adaptive surface.
[0112] The inorganic transparent material can be transparent (even transparent and tempered) glass, which plays a supporting and light-transmitting role. In addition, the edges of the inorganic transparent material are also polished to be transparent as much as possible.
[0113] As for the cushion and the bracket, they can be connected by sticking the bracket with strong glue, wherein the bracket can be a plastic bracket.
[0114] In addition, for Figure 1 As shown, the components involved are:
[0115] A double-layer flexible layer structure (exemplarily, implemented as a double-layer soft rubber pad or other soft pad) for carrying (for example, for carrying the feet of a real person or a dummy or a robot), a glass plate under the double-layer flexible layer structure, lamps and their lamp PCB soft boards around the double-layer flexible layer structure and the glass plate, four brackets (which can be implemented in a form similar to legs), a shell covering the surroundings, a camera located on one side of the surroundings (including a camera PCB board), etc.; wherein:
[0116] The camera is connected to the camera PCB board via a flexible circuit board FPC; and Figure 1 In the case of a 3D image sensor, the camera is side-imaged, and is located inside the sensor, aiming at the bottom of the glass plate. When the camera refresh rate is 120Hz or even higher, the dynamic performance of the image can be captured.
[0117] Since the surface of the double-layer flexible layer structure can be selected to have a certain degree of stickiness, and the glass can be selected to adapt to the surface, the glass plate and the double-layer flexible layer structure can be fully bonded;
[0118] The camera pcb board can communicate with a data processing device such as a computer that later processes the images captured by the camera via its own USB cable or wirelessly.
[0119] It can be understood that when the sensor is working, the camera can see the deformation when the cushion surface is deformed by force, and realize the subsequent applications such as tactile sensing and detection through visual algorithms and deep learning algorithms. As for the touch, it can be derived from the deformation of the contact surface of the epidermis.
[0120] In another embodiment, an inorganic transparent material, such as glass, quartz, etc., can also be a lens. Furthermore, a concave lens can be selected to reduce the distance between the camera and the flexible layer, thereby reducing the thickness of the sensor device.
[0121] Furthermore, in another embodiment, the present disclosure also discloses a method for processing image data captured by the flexible tactile sensor, comprising the following steps:
[0122] S100: performing a linear transformation on the original image acquired by the camera, correcting the side image into a front image, performing dermis layer marker detection and sensing on the imaging data of the front image, and calculating the three-dimensional force field using a marker displacement method;
[0123] S200: De-noises and enhances the high-resolution epidermis and dermis imaging data, removes the dermis markers, and obtains high-quality high-resolution epidermis images.
[0124] S300: Calculating pixel gradient values based on the high-quality high-resolution image of the epidermis, and fitting the three-dimensional force field calculated in step 100 using the gradient information to obtain a high-resolution three-dimensional force field;
[0125] S400: Calculate three-dimensional moment data based on the three-dimensional force field data to obtain a high-resolution six-dimensional force field including moment data. Exemplarily, the high-resolution six-dimensional force field is obtained by calculating the curl at each resolution point of the three-dimensional force field.
[0126] For this method, it can judge, sense and analyze the contact force by analyzing the data of the force field. Figuratively speaking, the torque has more distance information than the force, and the deformation of the flexible tactile sensor contains the information of the change in distance, which involves distance information and time information. Therefore, when the tactile sensor of the present disclosure adopts the markers mentioned above to obtain the three-dimensional contact force, the present disclosure can further calculate the three-dimensional torque data to finally obtain the six-dimensional force field containing the torque. When the image data includes information such as markers, even if the double-layer flexible layer structure undergoes surface changes, these markers can always serve as a reference. The method of processing the image data is nothing more than parsing and calculating this information. The fundamental material basis is the flexible tactile sensor disclosed in the present disclosure.
[0127] In another embodiment, the method further comprises the steps of:
[0128] S500: Using a convolutional neural network (CNN) to perform target detection and segmentation on the high-quality high-resolution epidermis image calculated in step S200, and extracting contact surface deformation information.
[0129] In another embodiment, the method further comprises the steps of:
[0130] S600: For the contact surface deformation information, the contact surface deformation information of each frame image detected is fused in time series, and a temporal neural network (RNN) is used to perform behavior sensing and extraction to obtain dynamic tactile information.
[0131] Continuing from the previous embodiment, it can be understood that, since time is involved in the deformation process, in addition to the three dimensions of XYZ, the fourth dimension of time t is involved. This is the fundamental reason why the present invention can obtain contact surface deformation information and dynamic feel information.
[0132] In summary, in addition to proposing a new flexible tactile sensor, the present disclosure further discloses that through advanced image recognition and deep learning technology, based on high-resolution imaging data of the epidermis, it is ultimately possible to realize functions such as six-dimensional force field calculation of the contact surface, deformation sensing of the contact surface, and dynamic hand feel sensing. In addition to the above-mentioned specific image data processing methods, any other algorithm / method that can parse XYZ three-dimensional spatial information, distance information inevitably caused by deformation, and time t can be adapted to the flexible tactile sensor described in the present disclosure. The flexible tactile sensor described in the present disclosure, like any generalized sensor, is not limited to the processing method of the signal obtained by the sensor.
[0133] To summarize, the main features of this disclosure are as follows:
[0134] 1. Provide high-resolution and high-precision "contact force sensing". The contact force obtained by the corresponding processing method is "three-dimensional contact force", the result of which is a three-dimensional pressure vector field distributed on the surface of the contact layer, and can eventually be realized in six dimensions;
[0135] 2. Provide high-resolution and high-precision "contact surface deformation". The result is the three-dimensional spatial position distributed on the contact layer surface;
[0136] 3. The bionic epidermis-dermis double-layer flexible structure, in addition to sensing the deformation of the epidermis, also expresses the "six-dimensional contact force" by sensing the deformation of the dermis. The result includes not only the three-dimensional pressure vector field distributed on the surface of the contact layer, but also the three-dimensional torque vector field;
[0137] 4. Different configurations of the surface toughness, hardness, thickness, material, etc. of the epidermis and dermis. The present disclosure can measure the "contact surface deformation" in a larger spatial range while maintaining accuracy;
[0138] 5. Achieve "high dynamic" contact force through a high dynamic vision module of more than 120 frames, which can more accurately measure rapidly changing contact force;
[0139] 6. "Touch" sensing based on deep learning, which classifies the contact force and contact surface deformation into a variety of different touch categories through deep neural networks;
[0140] 7. "Dynamic touch" sensing based on deep learning can sense and classify dynamic touch during active touch.
[0141] Those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions, modules, and units involved are not necessarily required for the present invention.
[0142] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0143] In several embodiments provided in the present disclosure, it should be understood that the disclosed method can be implemented as a corresponding functional unit, a processor or even a system, wherein the various parts of the system can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit can be integrated in a processing unit, or each unit can exist separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a smart phone, a personal digital assistant, a wearable device, a laptop, a tablet computer) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage media include: USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard disks, magnetic disks or optical disks, and other media that can store program codes.
[0144] As described above, the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure.
Claims
1. A method for processing image data captured by a surface flexible tactile sensor, Features: The surface-shaped flexible tactile sensor is used to bear the tactile sensation and force of stepping on by both feet; The surface flexible tactile sensor comprises: A double-layer flexible structure consisting of a bionic epidermis layer and a dermis layer and a camera, wherein: The camera is used to capture the deformation of the double-layer flexible layer structure; The double-layer flexible layer structure is based on a material having light transmittance and softness, wherein: According to the softness, the double-layer flexible layer structure is deformed; According to the light transmittance, the camera is able to capture the deformation of the double-layer flexible layer structure; and, The epidermis layer is mixed with or coated with a silver pigment as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure; in: The surface-shaped flexible tactile sensor is in the form of a rectangular cube with a certain height, and the camera is arranged on one side of the cube for side imaging; in, Close to the epidermis, a light PCB soft board is arranged on the four sides of the surface flexible tactile sensor, and red, blue, green and white LED lights are arranged on the four sides respectively; in, The reflective layer is used to reflect light along the dermis layer and the epidermis layer, so that the light enters the camera along the direction of the epidermis layer and the dermis layer; on the side close to the dermis layer, a silver pigment is mixed on one surface inside the epidermis layer as a reflective layer to assist the camera in capturing the deformation of the double-layer flexible layer structure; the outer surface of the epidermis layer or the inner surface of the outer surface of the epidermis layer may also be mixed or coated with a silver pigment as a reflective layer; Using a needle coated with pigment, piercing the inner part of the dermis to mark; When the surface flexible tactile sensor is working, the camera captures the deformation of the double-layer flexible layer structure and then further senses the deformation of the dermis layer; When the surface flexible tactile sensor is subjected to a contact force due to contact, The epidermis layer is used to fit a higher resolution, so that the deformation of the double-layer flexible layer structure has the resolution of the epidermis layer; When the surface flexible tactile sensor is used, the deformation of the dermis is sensed to express the six-dimensional contact force: in addition to the three-dimensional pressure vector field distributed on the surface of the epidermis as the contact layer, a three-dimensional torque vector field is also included; the six-dimensional contact force refers to the force in the three-dimensional coordinate axis and the rotational force around the three-dimensional coordinate axis, and the rotational force around the three-dimensional coordinate axis is the torque; The method comprises the following steps: S100: performing a linear transformation on the original image acquired by the camera, correcting the side image into a front image, performing dermis layer marker detection and sensing on the imaging data of the front image, and calculating the three-dimensional pressure vector field using a marker displacement method; S200: De-noises and enhances high-resolution epidermis and dermis imaging data, removes dermis markers, and obtains high-quality high-resolution epidermis images. S300: Calculating pixel gradient values based on the high-quality high-resolution epidermis image, and fitting the three-dimensional pressure vector field calculated in step S100 using the gradient information to obtain a high-resolution three-dimensional pressure vector field; S400: Calculate three-dimensional torque data based on high-resolution three-dimensional pressure vector field data to obtain a high-resolution six-dimensional force field including torque; wherein the high-resolution six-dimensional force field is obtained by calculating the curl at each resolution point of the three-dimensional pressure vector field.
2. The method according to claim 1, in: The epidermis layer and the dermis layer are both made of flexible materials based on silicone rubber.
3. The method according to claim 1, in: The thickness of the dermis layer is greater than the thickness of the epidermis layer.
4. The method according to claim 3, in: The thickness of the dermis layer is 20 mm, and the thickness of the epidermis layer is 3 mm.
5. The method according to claim 1, in: The dermis layer is bonded to the epidermis layer by its own adhesiveness.
6. The method according to claim 1, in: The double-layer flexible layer structure is designed as a consumable flexible layer that is easily replaceable.
