Object recognition system based on tactile sensing array and neural network algorithm

Abstract

The object recognition system based on tactile sensing array and neural network algorithm belongs to the field of machine learning pattern recognition. It aims to solve the limitation of using vision-based algorithm to recognize objects due to factors such as occlusion and lighting conditions. It uses a tactile sensing array that relies on pressure information rather than visual information, making it immune to factors such as lighting and occlusion. It also combines neural network algorithms such as multi-layer perceptron (MLP) and convolutional neural network (CNN) to process pressure information from the sensing array. After comparison, the MLP algorithm was ultimately chosen, with a verification accuracy of 99.45%, superior to the CNN algorithm's 97.43%. The invention demonstrates the ability to recognize six everyday objects in real-time in an open environment, with the advantages of simple signal processing circuit, easy deployment, and low cost, making it a promising solution for smart home applications.

Description

Technical Field

[0001] This invention belongs to the field of machine learning applications, specifically relating to an object recognition system based on a tactile sensor array and a neural network algorithm. Background Technology

[0002] With the rapid development of artificial intelligence and the Internet of Things (IoT), the demand for building sensor networks to collect sensor information and provide intelligent services is constantly growing. Object recognition refers to the process of classifying and recognizing objects in images, videos and other types of information using computer technology. It has a wide range of applications in today's society. For example, smart homes can control home appliances and security systems through object recognition, and industries can improve production efficiency and quality through object recognition. These fields all require the recognition and interaction of objects in the environment. Therefore, object recognition capability is a key technology for realizing intelligent monitoring and stimulus response. The development of object recognition has gone through several stages, from methods based on manually designed features and template matching, to methods based on machine learning and statistical models, and then to methods based on deep learning and neural networks. With the increase in the scale and quality of datasets, as well as the progress in computing power and algorithms, the performance of object recognition has also been significantly improved. Current object recognition technology mainly relies on vision-based methods, which classify and recognize objects by extracting features such as color, texture, and shape. However, these algorithms have some defects and limitations, such as: (1) visual data may contain sensitive information, such as faces or other identifiable features, which may lead to the risk of personal information leakage. (2) Visual data is affected by factors such as occlusion, lighting conditions, background clutter, viewing angle and image acquisition resolution, which may reduce recognition efficiency and accuracy. (3) Visual data usually requires preprocessing and feature extraction, which increases computational complexity and resource consumption.

[0003] Therefore, it is necessary to study alternative methods for image object recognition to overcome the shortcomings of vision-based algorithms. In recent years, with the development of micro-sensors and intelligent robot technology, flexible tactile sensors have become increasingly important in daily life. Flexible tactile sensors can perceive the external environment through pressure information without relying on visual information, which makes them unaffected by factors such as light and occlusion, and have higher stability and reliability. In addition, flexible tactile sensors can be embedded in various forms of intelligent sensor networks, such as wristbands, armbands, gloves, robotic hands, etc., to provide more detailed information for more intelligent services. Combining multiple pressure-sensitive elements into a tactile sensor array can increase the dimension and amount of information of the perceived environment. On the market, some studies use flexible sensor arrays for object recognition. However, these methods generally have some defects and limitations, such as: (1) The circuit of the prepared flexible sensor array is relatively complex and it is difficult to extend to high spatiotemporal resolution to adapt to more intelligent and demanding scenarios. (2) There is crosstalk phenomenon inside the flexible tactile sensor array, which may damage the purity of sensor information and interfere with subsequent applications. Traditional blurring and sharpening methods can handle noise and crosstalk effects from sensor arrays during information processing, but they introduce complex and additional processing steps, making such systems difficult to popularize.

[0004] Compared to conventional sensor array circuits, row-column scanning arrays effectively reduce circuit complexity and improve spatial resolution and sensitivity, finding wide applications in fields such as liquid crystal displays, digital X-rays, digital microfluidics, and biosensing. A row-column scanning array utilizes the intersections of rows and columns to control the output of each unit. For an N×M array, only N+M lines are needed, while traditional sensor array circuits require 2N×M lines. This allows row-column scanning circuits to form larger-scale arrays while maintaining circuit rationality and timeliness. For example, for a 4×4 array, the number of leads in a row-column scanning circuit is four times that of a traditional circuit.

[0005] Machine learning is a technique that uses data and algorithms to train models and make predictions. It has wide applications in fields such as image recognition, natural language processing, and computer vision. For crosstalk phenomena within sensor arrays, machine learning is an effective solution because it can extract clean features from noisy and perturbed data for learning and processing. Common methods include Multilayer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs). An MLP is a feedforward artificial neural network consisting of an input layer, one or more hidden layers, and an output layer. It uses backpropagation to adjust weights and biases and minimize the loss function. A CNN is a powerful machine learning technique consisting of an input layer, one or more convolutional layers, pooling layers, fully connected layers, and an output layer. It can capture deep spatial features from data without preprocessing. By combining machine learning with sensor arrays, the collected data can be processed effectively to obtain accurate object recognition results. Summary of the Invention

[0006] The purpose of this invention is to address the limitations of existing vision-based object recognition algorithms by proposing an everyday object recognition system based on tactile sensor arrays and neural network algorithms.

[0007] This invention utilizes a row-column scanning piezoresistive sensing array, which effectively reduces circuit complexity and improves spatial resolution and sensitivity. A row-column scanning array is one that uses the intersections of rows and columns to control the output of each unit. A high voltage is alternately applied to the column electrodes, while the other three column electrodes are set to a high-impedance state, preventing additional current loops between sensing units. When a high voltage is applied, the output signals of the four sensor elements in a columnar electrode are obtained from the row electrodes.

[0008] This invention utilizes two neural network algorithms, MLP and CNN, to extract effective features, classify, and recognize data obtained from a sensor array. For the MLP algorithm, the optimal model configuration was determined by modifying the number of hidden books and neurons. For the CNN model, the optimal configuration was determined by adjusting the number of convolutional layers and kernel size; the experimental process was similar and will not be repeated. Unlike MLP, pooling layers were eliminated in the optimal CNN configuration because pooling layers would lead to the loss of feature information due to the concentrated input information. Next, we evaluated the performance of the optimal MLP and CNN models on training and validation datasets. In each training epoch, the MLP model was evaluated on both training and testing datasets, recording the loss and accuracy. The optimal MLP model achieved the highest accuracy over 40 training periods. Although the validation loss increased slightly thereafter, the recognition accuracy remained stable, indicating that the model has the ability to generalize and prevent overfitting; its final accuracy on the validation dataset was 99.45%. The CNN reached its maximum accuracy after 30 training sessions, after which the accuracy decreased as the loss on the validation dataset increased, with the best accuracy being 97.43%. Compared to MLP models, both its accuracy and generalization ability are lower.

[0009] This invention also constructs a recognition system to demonstrate its application in smart homes. It provides users with a user-friendly graphical user interface (GUI) for obtaining feedback from the environment, displaying pressure signal graphs from the sensor array, real-time pressure data, and object recognition results. The sensor array collects environmental data, which is processed by machine learning algorithms for object recognition. The hardware platform integrates an Arduino for communication between the circuit and the PC, using serial data transmission. A trained and optimized MLP model analyzes real-time data from the sensor array and identifies objects placed on it. The system provides feedback to the user through an interactive interface, displaying the real-time pressure status of the sensor array and the recognition results. The real-time pressure graph of the sensor array is displayed on the left side of the interface, while the specific values ​​of each sensor are displayed in the right corner. The recognition results are presented in the text box below. Attached Figure Description

[0010] Figure 1 Figures are attached to the abstract;

[0011] Figure 2 (a) Schematic diagram of the voltage divider circuit used in this paper, where 1-voltage divider resistor, 2-ADC interface, 3-DAC interface for supplying voltage, 4-tactile sensor array, 5-Arduino Nano development board; (b) Physical diagram of the circuit; high voltage is alternately input through ports D1-D4, and analog signals A1-A4 are input to the analog-to-digital converter (ADC) interface and converted into digital signals;

[0012] Figure 3 Box plots are used to measure the accuracy of the MLP model with different configurations. The results show that the MLP model exhibits the most significant accuracy and stability when three hidden layers are included.

[0013] Figure 4 This is the architecture diagram of the optimal MLP model. It has one input layer, one output layer, three hidden layers, and three ReLU activation layers. The model uses Softmax for classification.

[0014] Figure 5 This is a diagram of the architecture of the optimal CNN model;

[0015] Figure 6 The accuracy and loss curves of the MLP model show that the accuracy on both the training and validation sets remained stable over 40 batches.

[0016] Figure 7 The accuracy and loss curves of the CNN model;

[0017] Figure 8 The image shows the result when a mug is placed on a flexible sensor array, which is a visual interface (GUI). Detailed Implementation

[0018] The following detailed description, in conjunction with the accompanying drawings, is intended to better illustrate the features and functional characteristics of this patent, rather than to limit the scope of protection of this patent.

[0019] See attached Figure 2 The hardware of this patent mainly revolves around a circuit composed of Arduino hardware. Each unit consists of a flexible piezoresistive sensor and a reference resistor, connected to the intersection of the row and column, respectively. Four wires are led out from the electrodes of each row and column, connecting to the Arduino development board. The resistance of each flexible sensor can be calculated using the following formula:

[0020]

[0021] In the formula, R sensor R represents the resistance value of each flexible sensor. X The resistance value, V, represents the voltage divider resistor. out This is the output voltage, which is read and converted from the power supply voltage by the Arduino's analog-to-digital converter. The source voltage is applied sequentially to the four column electrodes, and the output is obtained from the four row electrodes.

[0022] In this invention, MLP and CNN networks were designed for flexible sensor arrays within the PyTorch framework, such as... Figure 3 and Figure 4As shown in the figure, the MLP network consists of one input layer, three hidden layers, and one output layer, with 60 neurons in each layer. The CNN network consists of three convolutional layers, which use one-dimensional convolutional kernels to extract feature information from various channels. To retain more feature information, pooling operations are omitted. Both models are optimized using the Adam optimizer with a learning rate of 0.0001 and trained on a training dataset consisting of 7000 object placement samples (1000 per label). The training dataset includes six everyday objects (thermos, mouse, mug, milk, power bank, and battery), each with different mass, size, and shape. Their mass is shown in Table 1.

[0023]

[0024] Table 1 Six common objects and their masses

[0025] This invention constructs a recognition system to demonstrate its application in smart homes, providing users with a user-friendly interactive interface for obtaining feedback from their environment. A sensor array collects environmental data, which is processed by a machine learning algorithm for object recognition. The hardware platform integrates an Arduino for communication between the voltage divider circuit and the processing unit, using serial transmission for data transfer. A trained and optimized MLP model analyzes real-time data from the sensor array and identifies objects placed on it. The system provides feedback to the user through the interactive interface. Figure 8 ).

Claims

1. An object recognition system based on a tactile sensing array and neural network algorithm, characterized in that The object recognition system based on a tactile sensing array and a neural network algorithm comprises a voltage dividing resistor (1), an ADC interface (2), a DAC interface (3), a tactile sensing array (4), and an Arduino development board (5), wherein the voltage dividing resistor (1), the ADC interface (2), the DAC interface (3), and the tactile sensing array (4) are connected around the Arduino development board (5) through a breadboard and a flying wire to form an entire circuit, and the entire circuit comprises the following steps: step one, providing a tactile sensing array (4) and designing a voltage dividing circuit; step two, controlling a row-column scanning voltage dividing circuit through the Arduino development board (5) to read an output signal; step three, transmitting the output signal to a PC end to collect data sets of six daily objects, i.e., a thermos cup, a mouse, a mug, milk, a power bank, and a battery, and no object, and taking the data sets as a training set; step four, processing the output signal through a multilayer perceptron (MLP) network, wherein the network comprises an input layer, three hidden layers, and an output layer, each layer comprises 60 neurons, and the network is trained on the above training data set; and step five, integrating the MLP model into a graphical user interface (GUI) written by a user, wherein the GUI can also display a pressure map and real-time values of an object placed on the sensing array in real time and give a corresponding recognition result.

2. The object recognition system based on a tactile sensing array and neural network algorithm of claim 1, wherein The tactile sensing array has a size of 4*4.

3. The object recognition system based on a tactile sensing array and neural network algorithm of claim 1, wherein The Arduino development board is of Nano type.

4. The object recognition system based on tactile sensing array and neural network algorithm of claim 1, wherein When the row-column scanning voltage dividing circuit is controlled, only one high voltage is alternately applied to a column electrode, and the other three column electrodes are set to a high impedance state to prevent the generation of an extra circuit loop.

5. The object recognition system based on tactile sensing array and neural network algorithm of claim 1, wherein The transmission to the PC end is serial transmission.

6. The object recognition system based on a tactile sensing array and neural network algorithm of claim 1, wherein The training data set has a total of 7000 samples, 1000 samples are collected for each case, and each sample comprises a 1*16 pressure signal and a corresponding label.

7. The object recognition system based on a tactile sensing array and neural network algorithm of claim 1, wherein The GUI interface is designed by using Python language.

Images