Pressure sensor data shape recognition method

Abstract

The invention relates to the technical field of tactile perception and artificial intelligence, in particular to a pressure sensor data shape recognition method, which comprises the following steps of: preprocessing an original pressure matrix acquired by a pressure sensor array, and performing feature extraction on the preprocessed pressure matrix by using a CNN backbone network to obtain a feature map with a channel number and a spatial size; processing the feature map through a convolutional layer, dividing the feature map into a plurality of image blocks, linearly projecting each image block into a high-dimensional token, generating a token sequence, and adding a two-dimensional position code to each token in the token sequence to reserve spatial position information, carrying out global modeling on the token sequence added with the position codes by utilizing a Transform encoder so as to capture remote dependence and overall shape topology and obtain global feature representation; the target shape category probability is obtained through the classification head, the shape recognition result is output, high-robustness shape recognition is achieved, and the method can adapt to different sensor specifications and edge calculation scenes.

Description

Technical Field

[0001] This invention relates to the fields of tactile perception and artificial intelligence, and in particular to a method for shape recognition of pressure sensor data based on a CNN-Transformer hybrid architecture. Background Technology

[0002] Pressure sensor arrays, as a type of two-dimensional distributed sensor, are widely used in scenarios such as robotic tactile sensing, smart seats, body pressure monitoring, and human-computer interaction. Their output is a pressure intensity matrix that varies spatially, but they suffer from problems such as noise, individual differences, and diverse contact shapes.

[0003] Traditional recognition methods (such as threshold segmentation, connected component analysis, and template matching) are sensitive to deformation, rotation, and translation, have weak generalization ability, require manual feature design, and are difficult to adapt to complex shapes. While pure convolutional neural network (CNN) methods can extract local features such as edges and textures, their global modeling ability is insufficient, and their ability to capture long-distance relationships and overall topology is limited. Especially in scenarios such as large-scale contact, sparse multi-point contact, and shape combinations, the receptive field of local convolution grows slowly, and the cost of modeling global relationships is high. Although pure visual Transformer (ViT) methods are good at modeling global relationships, when directly performing sequence modeling on the original 2D array, their robustness to sensor noise and local details is not as good as pure CNN methods, and they usually require a large amount of data for effective training, making convergence difficult on small datasets.

[0004] Therefore, there is a need to provide a method that has a hybrid architecture that takes into account both local feature extraction and global relationship modeling, and can robustly and accurately identify the shape of objects from their array data, so as to make the shape recognition of pressure arrays better. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a shape recognition method for pressure sensor data that combines a hybrid architecture of CNN (Convolutional Neural Network) and Transformer, taking into account both local features and global relationships. By designing a hybrid neural network architecture that balances the extraction of fine local features with the modeling of global long-range dependencies, the method extracts local detailed features through CNN and models global dependencies through Transformer, achieving highly robust shape recognition. This improves the model's robustness under adverse conditions such as sensor noise, missing local data (bad pixels), scale / rotation / translation changes, and sparse contact points. This also makes the method highly scalable and deployable, adaptable to different sensor specifications and edge computing scenarios.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] This invention provides a method for shape recognition of pressure sensor data, comprising the following steps:

[0008] The original pressure matrix obtained by the pressure sensor array is normalized and denoised, and the size of the pressure matrix is ​​unified to a preset size;

[0009] The preprocessed stress matrix is ​​used to extract features using a CNN backbone network to obtain a feature map with channel number and spatial size;

[0010] The feature map is processed by a convolutional layer, which divides the feature map into multiple image patches, and linearly projects each image patch into a high-dimensional token to generate a token sequence.

[0011] A two-dimensional position code is added to each token in the token sequence to preserve spatial position information, and a Transformer encoder is used to perform global modeling on the token sequence with added position codes to capture remote dependencies and overall shape topology, thereby obtaining a global feature representation.

[0012] Based on the global feature representation, the target shape category probability is obtained through the classification head, and the shape recognition result is output.

[0013] As a further aspect of the present invention, the original pressure matrix obtained by the pressure sensor array is normalized and denoised, including:

[0014] The original pressure matrix is ​​pruned and normalized, with the normalization method being either min-max normalization or z-score normalization.

[0015] Denoising processes include using median filtering or bilateral filtering to suppress impulse noise;

[0016] The pressure matrix is ​​sized and aligned to a fixed-size grid while preserving the geometric proportions.

[0017] And / or, generate a mask for bad or defective regions of the stress matrix, the mask being used for attention masking or loss weighting in subsequent processing.

[0018] As a further embodiment of the present invention, the CNN backbone network is a lightweight network, including ResNet or ConvNeXt variants, with the downsampling stride set to an integer between 4 and 16.

[0019] As a further embodiment of the present invention, in the CNN backbone network, the number of channels of the output feature map is C, and the spatial size is H'×W', where H' and W' are the original height H and width W divided by the sampling stride, respectively.

[0020] As a further aspect of the present invention, the feature map is divided into multiple image blocks by a convolutional layer, wherein the kernel size and stride of the convolutional layer are both set to the image block size (ph, pw) to achieve non-overlapping or controllable overlap of the grid division, and the number of output channels is D, where D is the hidden layer size of the Transformer, and the local features of each image block are mapped to D-dimensional tokens.

[0021] As a further aspect of the present invention, a two-dimensional position code is added to each token in the token sequence, using learnable position code or two-dimensional sine and cosine code, the shape of the position code matching the length of the token sequence, and supporting cross-size interpolation to adapt to different sensor specifications.

[0022] As a further aspect of the present invention, the learnable position encoding shape is matched with the length of the token sequence and the hidden layer size D, and supports adaptation to different sequence lengths through interpolation; when constructing the token sequence, a [CLS] token for global classification is added before the token generated from the image patch; wherein, the learnable position encoding shape can be (1, N(+1), D), and when constructing the token sequence, [CLS] is concatenated with the image patch token to form a sequence of length N(+1).

[0023] As a further aspect of the present invention, when generating the token sequence, a [CLS] token is pre-added for global classification. The [CLS] token participates in the global relationship modeling of the Transformer encoder and is used for classification output.

[0024] As a further aspect of the present invention, the classification head is based on the global feature representation output by the Transformer, and the classification head adopts one of the following methods:

[0025] Use the output of the [CLS] token for fully connected classification;

[0026] Alternatively, perform global average pooling or attention pooling on all tokens, then perform fully connected classification, and finally output the target shape category probability through a fully connected layer.

[0027] As a further aspect of the present invention, the training process of the pressure sensor data shape recognition method includes: using a cross-entropy loss function, supplemented by label smoothing or focus loss to handle class imbalance, and using pre-trained weights of similar tasks for the CNN backbone network, using pre-trained ViT weights of the image domain or tactile domain for the Transformer encoder, and applying data augmentation to the training data including random rotation, translation, scaling, intensity perturbation, random drop points, or noise injection.

[0028] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0029] The pressure sensor data shape recognition method of the present invention effectively overcomes the limitations of single technologies by using a hybrid architecture of CNN and Transformer. The CNN backbone network operates preferentially in the original pressure domain, and by utilizing its inherent local connectivity and weight sharing characteristics, it can efficiently extract local detail features such as noise-resistant edges, contours and textures. On this basis, the Transformer performs global self-attention operation on the token sequence generated by the CNN feature map, which can capture the dependencies between distant regions in the pressure distribution and the overall shape topology. This makes the method of the present invention more stable and accurate in shape recognition than traditional methods or pure CNN / ViT schemes when facing complex scenarios such as sensor noise, local bad pixels / missing points, shape scaling, rotation, translation changes and sparse contact combinations.

[0030] The pressure sensor data shape recognition method of this invention also uses convolutional layers to directly embed feature maps into blocks. That is, the size of the convolutional kernel and the stride are set as the block size, and the block division and linear projection are completed in one step. This adapts to two-dimensional geometric structures and can smoothly connect the feature map resolution output by the CNN backbone with the token sequence length required by the Transformer, reducing the overhead of data reorganization and making the overall model architecture more compact and efficient. Furthermore, by introducing two-dimensional positional encoding, it supports learnable patterns or sine and cosine encoding and provides cross-size interpolation strategies, making it easier for the trained model to adapt to pressure sensor arrays of different specifications and resolutions, thereby improving the generalization ability and ease of practical deployment of the method.

[0031] The pressure sensor data shape recognition method of the present invention adopts a shallow CNN backbone, a controllable downsampling stride and a small Transformer configuration, and combines the efficiency of convolutional block embedding. The entire model maintains high performance while effectively controlling the number of parameters and computational load. Through the above ingenious architecture integration and process innovation, it brings significant benefits in terms of recognition accuracy, robustness, computational efficiency, deployment flexibility and scalability.

[0032] These or other aspects of the invention will become more apparent from the following description of embodiments. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. In the drawings:

[0034] Figure 1This is a flowchart of a pressure sensor data shape recognition method according to the present invention.

[0035] Figure 2 This is a schematic diagram of the recognition process of a pressure sensor data shape recognition method according to the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0037] The technical solutions in the exemplary embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] See Figure 1 and Figure 2 As shown, embodiments of this application also provide a method for shape recognition of pressure sensor data, including the following steps:

[0039] Step S10. Normalize and denoise the original pressure matrix obtained by the pressure sensor array, and unify the size of the pressure matrix to a preset size.

[0040] In this step, the raw pressure matrix acquired by the pressure sensor array is normalized and denoised, including:

[0041] The original pressure matrix is ​​pruned and normalized, with the normalization method being either min-max normalization or z-score normalization.

[0042] Denoising processes include using median filtering or bilateral filtering to suppress impulse noise;

[0043] Size alignment of the pressure matrix is ​​performed to unify the pressure matrix to a fixed-size (e.g., H×W) grid, preserving the geometric scale, or the original coordinate scale is explicitly recorded in subsequent location encoding;

[0044] And / or, generate a mask for bad or defective regions of the pressure matrix, and use the mask and defect processing to perform attention masking or loss weighting in subsequent processing.

[0045] Step S20. Use a CNN backbone network to extract features from the preprocessed stress matrix to obtain a feature map with channel number and spatial size.

[0046] In this step, the CNN backbone network is a lightweight network, including ResNet or ConvNeXt variants, with a downsampling stride set to an integer between 4 and 16. Preferably, the downsampling stride is set to 8 or 16 to balance feature resolution and computational cost. The CNN backbone network maintains stable training and feature transferability through lightweight normalization (BN / GN) and residual connections. By combining small convolutional kernels with dilated convolutions or multi-scale branches, edge, contour, and texture responses can be enhanced. In the CNN backbone network, the number of channels of the output feature map is C, and the spatial size is H'×W', where H' and W' are the original height H and width W divided by the downsampling stride, respectively.

[0047] Step S30. The feature map is processed by a convolutional layer, dividing the feature map into multiple image patches, and each image patch is linearly projected into a high-dimensional token to generate a token sequence.

[0048] In this step, the feature map is divided into multiple image patches using convolutional layers. The kernel size and stride of these convolutional layers are set to the image patch size (ph, pw), achieving non-overlapping or controlled-overlapping grid partitioning. The output channel number is D, where D is the hidden layer size of the Transformer. The local features of each image patch are mapped to D-dimensional tokens. This efficient implementation through convolution adapts to a two-dimensional layout. The formula for calculating the number of high-dimensional tokens N is as follows:

[0049] ;

[0050] Each high-dimensional token represents a local region, with a spatial size of [missing information]. Space size In the formula, This represents the total number of high-dimensional tokens generated after dividing the feature map into patches. This represents the spatial dimension of the height of the output feature map after feature extraction by the CNN backbone network. This represents the spatial dimensions of the output feature map width after feature extraction by the CNN backbone network. This represents the height of each image patch when performing patch embedding on the feature map. This represents the width of each image patch when performing patch embedding on the feature map. This represents the height of the original pressure matrix within the spatial resolution of the original pressure sensor array. This represents the width of the original pressure matrix within the spatial resolution of the original pressure sensor array. This represents the stride of the downsampling operation in a convolutional neural network.

[0051] Step S40. Add a two-dimensional position code to each token in the token sequence to preserve spatial position information, and use a Transformer encoder to perform global modeling on the token sequence with added position codes to capture remote dependencies and overall shape topology, and obtain a global feature representation.

[0052] In this step, a two-dimensional position code is added to each token in the token sequence, using either learnable position coding or two-dimensional sine / cosine coding. The position code shape matches the length of the token sequence and supports cross-size interpolation to accommodate different sensor specifications.

[0053] The learnable position encoding shape matches the length of the token sequence and the hidden layer size D, and supports adaptation to different sequence lengths through interpolation. When constructing the token sequence, a [CLS] token for global classification is added before the token generated from the image patch. The learnable position encoding shape can be (1, N(+1), D). When constructing the token sequence, [CLS] is concatenated with the image patch token to form a sequence of length N(+1).

[0054] When generating the token sequence, a [CLS] token is pre-added for global classification. The [CLS] token participates in the global relation modeling of the Transformer encoder and is used for classification output.

[0055] Step S50. Based on the global feature representation, obtain the target shape category probability through the classification head and output the shape recognition result.

[0056] In this step, the classification head is based on the global feature representation output by the Transformer, and the classification head adopts one of the following methods:

[0057] Use the output of the [CLS] token for fully connected classification;

[0058] Alternatively, perform global average pooling or attention pooling on all tokens, then perform fully connected classification, and finally output the target shape category probability through a fully connected layer.

[0059] This invention provides a shape recognition method for pressure sensor array data. It utilizes a CNN to extract high-quality local details and noise-resistant features. Based on this, it performs patch embedding on the CNN feature map through convolutional layers, dividing the two-dimensional feature map into a token sequence. Then, it combines a Transformer to model long-range dependencies and overall shape relationships, achieving robust recognition of complex contact shapes. This method can stably and accurately recognize two-dimensional shapes even under conditions of sensor noise, local missing pixels (bad pixels / dropped pixels), scale / rotation / translation changes, and sparse contact combinations. Furthermore, it possesses good scalability and deployability.

[0060] In some embodiments, the training process of the pressure sensor data shape recognition method includes: using a cross-entropy loss function, supplemented by label smoothing or focus loss to handle class imbalance, and using pre-trained weights of similar tasks on the CNN backbone network, using pre-trained ViT weights of the image domain or tactile domain on the Transformer encoder, and applying data augmentation to the training data including random rotation, translation, scaling, intensity perturbation, random drop points, or noise injection.

[0061] The pressure sensor data shape recognition method of the present invention effectively overcomes the limitations of single technologies by using a hybrid architecture of CNN and Transformer. The CNN backbone network operates preferentially in the original pressure domain, and by utilizing its inherent local connectivity and weight sharing characteristics, it can efficiently extract local detail features such as noise-resistant edges, contours and textures. On this basis, the Transformer performs global self-attention operation on the token sequence generated by the CNN feature map, which can capture the dependencies between distant regions in the pressure distribution and the overall shape topology. This makes the method of the present invention more stable and accurate in shape recognition than traditional methods or pure CNN / ViT schemes when facing complex scenarios such as sensor noise, local bad pixels / missing points, shape scaling, rotation, translation changes and sparse contact combinations.

[0062] The pressure sensor data shape recognition method of this invention also uses convolutional layers to directly embed feature maps into blocks. That is, the size of the convolutional kernel and the stride are set as the block size, and the block division and linear projection are completed in one step. This adapts to two-dimensional geometric structures and can smoothly connect the feature map resolution output by the CNN backbone with the token sequence length required by the Transformer, reducing the overhead of data reorganization and making the overall model architecture more compact and efficient. Furthermore, by introducing two-dimensional positional encoding, it supports learnable patterns or sine and cosine encoding and provides cross-size interpolation strategies, making it easier for the trained model to adapt to pressure sensor arrays of different specifications and resolutions, thereby improving the generalization ability and ease of practical deployment of the method.

[0063] The pressure sensor data shape recognition method of the present invention adopts a shallow CNN backbone, a controllable downsampling stride and a small Transformer configuration, and combines the efficiency of convolutional block embedding. The entire model maintains high performance while effectively controlling the number of parameters and computational load. Through the above ingenious architecture integration and process innovation, it brings significant benefits in terms of recognition accuracy, robustness, computational efficiency, deployment flexibility and scalability.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for shape recognition of pressure sensor data, characterized in that, Includes the following steps: The original pressure matrix obtained by the pressure sensor array is normalized and denoised, and the size of the pressure matrix is ​​unified to a preset size; The preprocessed stress matrix is ​​used to extract features using a CNN backbone network to obtain a feature map with channel number and spatial size; The feature map is processed by a convolutional layer, which divides the feature map into multiple image blocks, and linearly projects each image block into a high-dimensional token to generate a token sequence. A two-dimensional position code is added to each token in the token sequence to preserve spatial position information, and a Transformer encoder is used to perform global modeling on the token sequence with added position codes to capture remote dependencies and overall shape topology, thereby obtaining a global feature representation. Based on the global feature representation, the target shape category probability is obtained through the classification head, and the shape recognition result is output.

2. The pressure sensor data shape recognition method as described in claim 1, characterized in that, The raw pressure matrix acquired from the pressure sensor array is normalized and denoised, including: The original pressure matrix is ​​pruned and normalized, with the normalization method being either min-max normalization or z-score normalization. Denoising processes include using median filtering or bilateral filtering to suppress impulse noise; The pressure matrix is ​​sized and aligned to a fixed-size grid while preserving the geometric proportions. And / or, generate a mask for bad or defective regions of the stress matrix, the mask being used for attention masking or loss weighting in subsequent processing.

3. The pressure sensor data shape recognition method as described in claim 1, characterized in that, The CNN backbone is a lightweight network, including ResNet or ConvNeXt variants, with downsampling strides set to integers between 4 and 16.

4. The pressure sensor data shape recognition method as described in claim 3, characterized in that, In the CNN backbone network, the number of channels of the output feature map is C, and the spatial size is H'×W', where H' and W' are the original height H and width W divided by the sampling stride, respectively.

5. The pressure sensor data shape recognition method as described in claim 4, characterized in that, The feature map is divided into multiple image blocks by a convolutional layer. The kernel size and stride of the convolutional layer are set to the image block size, and the number of output channels is D, where D is the hidden layer size of the Transformer. The local features of each image block are mapped to D-dimensional tokens.

6. The pressure sensor data shape recognition method as described in claim 1, characterized in that, A two-dimensional position code is added to each token in the token sequence, using either learnable position code or two-dimensional sine / cosine code. The shape of the position code matches the length of the token sequence and supports cross-size interpolation.

7. The pressure sensor data shape recognition method as described in claim 6, characterized in that, The learnable positional encoding shape matches the length of the token sequence and the hidden layer size D, and supports adaptation to different sequence lengths through interpolation; when constructing the token sequence, a [CLS] token for global classification is added before the token generated from the image patch; wherein, the learnable positional encoding shape can be (1, N(+1), D), and when constructing the token sequence, [CLS] is concatenated with the image patch token to form a sequence of length N(+1).

8. The pressure sensor data shape recognition method as described in claim 7, characterized in that, When generating the token sequence, a [CLS] token is pre-added for global classification. The [CLS] token participates in the global relation modeling of the Transformer encoder and is used for classification output.

9. The pressure sensor data shape recognition method as described in claim 8, characterized in that, The classification head is based on the global feature representation output by the Transformer, and the classification head adopts one of the following methods: Use the output of the [CLS] token for fully connected classification; Alternatively, perform global average pooling or attention pooling on all tokens, then perform fully connected classification, and finally output the target shape category probability through a fully connected layer.

10. The pressure sensor data shape recognition method according to any one of claims 1-9, characterized in that, The training process of the pressure sensor data shape recognition method includes: using the cross-entropy loss function, supplemented by label smoothing or focus loss to handle class imbalance, using pre-trained weights of similar tasks for the CNN backbone network, using pre-trained ViT weights of the image domain or tactile domain for the Transformer encoder, and applying data augmentation to the training data, including random rotation, translation, scaling, intensity perturbation, random drop points, or noise injection.

Images