Flexible touch sensing structure and handwritten number recognition method thereof

Abstract

The invention discloses a flexible touch sensing structure and a handwritten number recognition method thereof, solves the problem that in an existing flexible touch human-computer interface, most of the number input modes are simple pulse signal detection, and encryption protection of information cannot be achieved, and belongs to the field of biological recognition identity authenticators. The flexible touch sensing structure comprises a feature region and a substrate region, the characteristic region is made of an ion conductor material, and the substrate region is made of a flexible insulating polymer; carrying out gray scale superposition on the plurality of handwritten digital images to obtain a gray scale distribution diagram; the areas with the highest and lowest handwriting probability distribution in the gray distribution map are substrate areas, and the areas with the highest and lowest handwriting probability distribution are feature areas. In order to classify and identify handwritten digits, a convolutional neural network signal classification algorithm is used. A single-dimensional time sequence voltage signal of a flexible touch sensing structure is collected to serve as input data of a convolutional neural network, and after the input data pass through the convolutional neural network, a digital classification result is output.

Description

Technical Field

[0001] The invention relates to a flexible touch sensing structure and a handwritten digit recognition method thereof, belonging to the field of biometric identity authenticators. Background Art

[0002] With the continuous development of electronic devices, traditional rigid human-machine interfaces are increasingly unable to meet the growing needs of modern users for portability and comfort. This has driven the rise of flexible human-machine interfaces, which are often used in wearable devices. This flexibility expands the application range of smart devices, allowing them to play a more important role in health monitoring, virtual reality and other fields. Touch is one of the most basic means of perceiving the interaction between organisms and the environment, which makes tactile interaction widely used in the field of flexible human-machine interaction. As an ionic conductor, organogel has a Young's modulus similar to that of biological tissues and organs and has excellent deformation ability. In addition, compared with traditional hydrogels, organogels are more durable. Therefore, a variety of flexible tactile human-machine interfaces based on ionic organogels have emerged. However, existing research mainly focuses on the basic tactile response of ionic organogels, and the sensitive elements are arranged in the detection area in a fully covered form, and the tactile sensing structure is not optimized for specific application scenarios. Therefore, it is of great significance to further study the response of ionic organogels to tactile signals and optimize the sensing structure.

[0003] In the field of human-computer interaction, the digital input function is an indispensable part. As a universal digital system, Arabic numerals are of great significance in daily life and various professional fields such as identity authentication, financial management and personal communication. Therefore, the security of digital input is very important. Information security may seriously threaten our normal life, business development, company operations, and even national security. In the existing flexible tactile human-computer interface, the digital input method is mostly simple pulse signal detection, which cannot realize the encryption protection of information. Therefore, it is necessary to build a flexible tactile human-computer interface with multiple encryption functions. Summary of the invention

[0004] Aiming at the problem that in existing flexible tactile human-machine interfaces, most digital input modes are simple pulse signal detection and information encryption protection cannot be achieved, the present invention provides a flexible touch sensing structure and a handwritten digital recognition method thereof.

[0005] A flexible touch sensing structure of the present invention comprises a feature area and a base area;

[0006] The feature region is made of an ion conductor material, and the base region is made of a flexible insulating polymer;

[0007] Superimpose the grayscale of multiple handwritten digital images to obtain a grayscale distribution map;

[0008] The areas with the highest and lowest probability distribution of handwriting in the grayscale distribution diagram are the base areas, and the areas with the handwriting probability distribution between the highest and the lowest are the feature areas.

[0009] Preferably, the handwritten digital image is derived from the MNIST handwritten digital database.

[0010] Preferably, the feature region is made of organic gel and the base region is made of silicone gel.

[0011] The present invention also provides a method for preparing the above-mentioned flexible touch sensing structure, comprising:

[0012] S1. preparing a mold according to the flexible touch sensing structure;

[0013] S2, fully mix the silicone oil and silica gel for 5 minutes;

[0014] S3, degassing the mixture in a vacuum desiccator for 30 minutes;

[0015] S4, pouring the degassed mixture into the mold and heating it to 70° C. for 20 minutes to solidify the mixture in the mold, and then forming the prepared silicone base;

[0016] S5, soaking the prepared silica substrate in a 10 wt % benzophenone / ethanol solution at room temperature for 3 minutes, then washing with methanol and drying thoroughly with nitrogen;

[0017] S6. Pour the organogel precursor solution into the dried silica gel substrate and irradiate it with a 365nm, 20W ultraviolet lamp for 10 minutes to fully cure the organogel and bond it to the silica gel substrate, thereby obtaining an assembly of the organogel and the silica gel substrate.

[0018] S7. Soak the obtained component in an ethylene glycol solution for 60 minutes to promote solvent replacement, thereby completing the preparation of the flexible touch sensing structure.

[0019] Preferably, the organogel precursor solution comprises 16 wt % acrylamide, 0.048 wt % N,N'-methylenebisacrylamide, 10 wt % sodium chloride, 0.1 wt % photoinitiator 2-hydroxy-2-methyl-1-phenyl-1-propanone and deionized water.

[0020] Preferably, the organic gel and the silica gel in the flexible touch sensing structure are dyed, and the organic gel and the silica gel have different colors.

[0021] Preferably, in S1, a mold is prepared according to the flexible touch sensing structure of claim 1 by 3D printing.

[0022] The present application also provides a method for recognizing handwritten digits using a flexible touch sensing structure, comprising:

[0023] Providing an alternating voltage to the flexible touch sensing structure, applying handwritten digits to the flexible touch sensing structure, and collecting a single-channel sensing signal response of the organic gel to obtain a corresponding voltage signal;

[0024] Construct a training set using handwritten digits and the corresponding voltage signals obtained;

[0025] The constructed training set is used to train the detection network, the input of which is a voltage signal and the output is a handwritten number;

[0026] The handwritten digit to be recognized acts on the flexible touch sensing structure, and the voltage signal of the flexible touch sensing structure is collected.

[0027] The voltage signal is input into the trained detection network for detection, and the detection network outputs the detection result.

[0028] Preferably, the metal electrodes are connected to the corners of the organic gel in the flexible touch sensing structure to provide an AC signal with a peak voltage of 1 V and a frequency of 1 MHz for the flexible touch sensing structure.

[0029] Preferably, the detection network comprises an input layer, a convolutional layer, a pooling layer, a flattening layer, a fully connected layer and an output layer which are connected in sequence.

[0030] Beneficial effects of the present invention: The present invention obtains a flexible touch sensing structure through a flexible touch sensing structure topology optimization method based on handwriting probability distribution to promote the development of a flexible tactile human-machine interface with multiple encryption functions. The flexible touch sensing structure of the present invention can detect individual bioelectricity, digital writing trajectory and speed, thereby realizing dual encryption through biometrics and handwriting recognition. In addition, the flexible touch sensing structure of the present invention has good flexibility and wearability, thereby enhancing the portability and privacy of the authenticator and improving encryption protection. The flexible touch sensing structure of the present invention is combined with a corresponding control circuit and a digital recognition algorithm based on a convolutional neural network. Compared with the traditional full-coverage sensing structure, the optimized structure improves the digital recognition accuracy by 107.91%. The flexible touch sensing structure proposed in the present invention has flexible wearability and multiple encryption functions, and provides a novel solution for a high-security authentication system based on behavioral biometrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the principle of the sensing structure topology optimization method of the present invention;

[0032] Figure 2 This is a photo of the flexible touch sensing structure after topology optimization of the sensing structure of the present invention;

[0033] Figure 3 It is a schematic diagram of the principle of the detection network in the handwritten digit recognition method of the present invention;

[0034] Figure 4 The sensitivity change before and after the topological structure optimization of the present invention;

[0035] Figure 5 Comparison of handwritten digit recognition accuracy before and after topology optimization;

[0036] Figure 6 Output voltage waveforms of different users when handwriting numbers using the flexible touch sensing structure. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0039] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0040] This implementation is based on the MNIST handwritten digit database and performs grayscale superposition on 500 groups of handwritten digit images. Figure 1 As shown. In this database, each image contains a number, and the number is written in a standardized square area. After superimposing 5000 images, the grayscale distribution map of the square area was obtained. The higher the grayscale value of the square area, the lower the probability of digital handwriting appearing in the area. The grayscale is divided into three parts. The area with the highest grayscale indicates that almost no handwriting appears, while the area with the lowest grayscale indicates that the handwriting of almost every number appears in this area. In the sensing detection of handwritten numbers, the areas with the highest and lowest probabilities of handwriting appearing are not the main focus of sensing detection. On the contrary, the middle area of ​​the probability distribution can be used as a feature area for more targeted digital handwriting detection. Therefore, the distribution of organic gel on the two-dimensional plane can be optimized according to the grayscale distribution of handwritten numbers. Specifically, a two-dimensional detection plane was constructed using a combination of an ion conductor and a flexible insulating polymer substrate, the ion conductor was placed in the feature area, and the flexible insulating polymer was placed in the areas with the highest and lowest probabilities of handwriting appearing.

[0041] The flexible touch sensing structure of this embodiment includes a feature area and a base area; the feature area is made of an ion conductor material, the ion conductor material can be selected from conductive hydrogel, organic gel, etc., preferably organic gel, and the base area is made of a flexible insulating polymer, the flexible insulating polymer can be selected from Ecoflex, silica gel, PDMS, preferably; grayscale superposition is performed on multiple handwritten digital images to obtain a grayscale distribution map;

[0042] The areas with the highest and lowest probability distribution of handwriting in the grayscale distribution map are the base areas, and the areas with the handwriting probability distribution between the highest and lowest are the feature areas.

[0043] The preparation of the flexible touch sensing structure in this embodiment includes the following steps:

[0044] First, a 3D printed flexible touch sensing structure is used to obtain a 3D printed mold;

[0045] Mix an appropriate amount of silicone oil (500 cSt) with the A and B components of silica gel (at a weight ratio of 1.25:1:1) for 5 minutes;

[0046] Then, the mixture was degassed in a vacuum desiccator for 30 minutes.

[0047] Next, the mixture was poured into a 3D printed mold and heated to 70°C for 20 minutes to cure it. At this stage, a silicone base with no organogel adhered was obtained.

[0048] In order to ensure strong adhesion between the silica gel and the organic gel and prevent the organic gel from falling off when the flexible touch sensing structure is deformed, the surface treatment is performed in the next step. The prepared silica gel substrate is immersed in a 10wt% benzophenone / ethanol solution at room temperature for 3 minutes, then washed with methanol and thoroughly dried with nitrogen. Next, the organic gel precursor solution is poured into the silica gel substrate and irradiated with a 365nm, 20W ultraviolet lamp for 10 minutes to fully cure the hydrogel and bond it to the silica gel substrate. The entire assembly is then immersed in an ethylene glycol solution for 60 minutes to promote solvent replacement, and finally a flexible touch sensing structure is obtained. The organic gel precursor solution in this embodiment includes 16wt% acrylamide, 0.048wt% N,N'-methylenebisacrylamide, 10wt% sodium chloride, 0.1wt% photoinitiator 2-hydroxy-2-methyl-1-phenyl-1-propanone and deionized water.

[0049] The final flexible touch sensing structure is as follows Figure 2 As shown in Figure 2, both organogel and silica gel are transparent, so they were stained for better visualization.

[0050] In addition to the flexible touch sensing structure, the handwritten digit recognition method of this embodiment also requires supporting hardware and software to realize the recognition of handwritten digits. The hardware control circuit includes a signal generation module and a signal acquisition module. The signal generation module is connected to the corners of the organic gel in the flexible touch sensing structure through metal electrodes, and provides the organic gel with an AC signal with a peak voltage of 1V and a frequency of 1MHz. The signal acquisition module collects the single-channel sensor signal response of the organic gel when touched, and reflects the touch information in the form of voltage. The handwritten digit recognition method includes:

[0051] Providing an alternating voltage to the flexible touch sensing structure, applying handwritten digits to the flexible touch sensing structure, and collecting a single-channel sensing signal response of the organic gel to obtain a corresponding voltage signal;

[0052] Construct a training set using handwritten digits and the corresponding voltage signals obtained;

[0053] The constructed training set is used to train the detection network, the input of which is a voltage signal and the output is a handwritten number;

[0054] The handwritten digit to be recognized acts on the flexible touch sensing structure, and the voltage signal of the flexible touch sensing structure is collected.

[0055] The voltage signal is input into the trained detection network for detection, and the detection network outputs the detection result.

[0056] In order to classify and recognize handwritten digits, the detection network of this embodiment uses a convolutional neural network signal classification algorithm. Figure 3 As shown in the figure, the one-dimensional time series voltage signal collected by the hardware control circuit is used as the input data of the convolutional neural network. After passing through the input layer, convolution layer, pooling layer, flattening layer, and fully connected layer in sequence, the output layer outputs the classification result of the number.

[0057] verify:

[0058] A traditional fully covered organic gel with the same size was prepared using a similar processing method for comparison. The position close to the electrode was taken as the detection origin, and the output voltage at this time was defined as the initial voltage U0. The changes in the output voltage relative to the initial voltage at different contact points were recorded, such as Figure 4 As shown. Compared with the traditional full-coverage structure, the optimized sensing structure shows higher sensitivity. In addition, due to the presence of the non-sensing area, the output potential difference between some adjacent contact points increases, breaking the symmetry and continuity of the output voltage in the traditional full-coverage sensing structure, which helps to more accurately identify the touch track during handwritten digits.

[0059] The output voltages of different handwritten digits under two sensor structures were collected for convolutional neural network model training. Specifically, for each sensor structure, 40 sets of data were collected for each digit. Among them, 80% of the data was used as a training set and the remaining data was used as a test set. Figure 5 As shown in the figure, the structurally optimized flexible touch sensing structure achieved a recognition accuracy of 97.53%, while the traditional full-coverage structure only achieved an accuracy of 46.91%. For the optimized sensing structure, the time output signals corresponding to different numbers are obviously different, while in the traditional full-coverage structure, the time output signals of different numbers are similar, and it is difficult to effectively distinguish the numbers even using the CNN algorithm.

[0060] In the process of detecting handwritten numbers by the flexible touch sensing structure, human body impedance, writing trajectory and speed, and changes in numbers will affect the output response signal. Therefore, using a flexible touch sensing structure to input passwords provides multiple encryption capabilities. The impedance of the human body is determined by the physiological characteristics of the individual. In addition, writing, as an important method in criminal investigation and analysis, has a high degree of encryption. Figure 6 As shown, thanks to multi-layer encryption, when two people write the same digital password, the waveform of the output voltage shows significant differences, which effectively reduces the risk of digital password leakage and demonstrates the feasibility of flexible touch sensing structure in encrypted information transmission.

[0061] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A flexible touch sensing structure, characterized in that: Includes characteristic regions and basement regions; The feature region is made of an ion conductor material, and the base region is made of a flexible insulating polymer; Superimpose the grayscale of multiple handwritten digital images to obtain a grayscale distribution map; The areas with the highest and lowest probability distribution of handwriting in the grayscale distribution diagram are the base areas, and the areas with the handwriting probability distribution between the highest and the lowest are the feature areas.

2. The flexible touch sensing structure according to claim 1, characterized in that: The handwritten digit image is derived from the MNIST handwritten digit database.

3. The flexible touch sensing structure according to claim 1, characterized in that: The feature region is made of organogel, and the base region is made of silicone.

4. The method for preparing the flexible touch sensor structure according to claim 3, characterized in that: The preparation method comprises: S1. preparing a mold according to the flexible touch sensing structure; S2, fully mix the silicone oil and silica gel for 5 minutes; S3, degassing the mixture in a vacuum desiccator for 30 minutes; S4, pouring the degassed mixture into the mold and heating it to 70° C. for 20 minutes to solidify the mixture in the mold, and then forming the prepared silicone base; S5, soaking the prepared silica substrate in a 10 wt % benzophenone / ethanol solution at room temperature for 3 minutes, then washing with methanol and drying thoroughly with nitrogen; S6. Pour the organogel precursor solution into the dried silica gel substrate and irradiate it with a 365nm, 20W ultraviolet lamp for 10 minutes to fully cure the organogel and bond it to the silica gel substrate, thereby obtaining an assembly of the organogel and the silica gel substrate. S7. Soak the obtained component in an ethylene glycol solution for 60 minutes to promote solvent replacement, thereby completing the preparation of the flexible touch sensing structure.

5. The method for preparing a flexible touch sensing structure according to claim 4, characterized in that: The organogel precursor solution includes 16 wt % acrylamide, 0.048 wt % N,N'-methylenebisacrylamide, 10 wt % sodium chloride, 0.1 wt % photoinitiator 2-hydroxy-2-methyl-1-phenyl-1-propanone and deionized water.

6. The method for preparing a flexible touch sensing structure according to claim 4, characterized in that: The organic gel and the silicone in the flexible touch sensing structure are dyed so that the organic gel and the silicone have different colors.

7. The method for preparing a flexible touch sensing structure according to claim 4, characterized in that: In S1, a mold is prepared according to the flexible touch sensing structure of claim 1 by using 3D printing.

8. The handwritten digit recognition method based on the flexible touch sensing structure of claim 1 is characterized in that: Handwritten digit recognition methods include: Providing an alternating voltage to the flexible touch sensing structure, applying handwritten digits to the flexible touch sensing structure, and collecting a single-channel sensing signal response of the organic gel to obtain a corresponding voltage signal; Construct a training set using handwritten digits and the corresponding voltage signals obtained; The constructed training set is used to train the detection network, the input of which is a voltage signal and the output is a handwritten number; The handwritten digits to be recognized act on the flexible touch sensing structure, and the voltage signal of the flexible touch sensing structure is collected. The voltage signal is input into the trained detection network for detection, and the detection network outputs the detection result.

9. The handwritten digit recognition method according to claim 8, characterized in that: The metal electrodes are connected to the corners of the organic gel in the flexible touch sensing structure to provide an AC signal with a peak voltage of 1V and a frequency of 1MHz for the flexible touch sensing structure.

10. The handwritten digit recognition method according to claim 8, characterized in that: The detection network includes an input layer, a convolutional layer, a pooling layer, a flattening layer, a fully connected layer and an output layer which are connected in sequence.

Images