Electronic skin-based material soft and hard identification method and system

Abstract

The application provides a material soft and hard recognition method and system based on electronic skin. The method is applied to the technical field of robot tactile perception. The method comprises the following steps: acquiring pressure array data and environment data collected by the electronic skin when the electronic skin contacts a target material; performing multi-modal feature extraction on the pressure array data in the space, time, frequency domain and gradient, respectively, and performing feature fusion to obtain a multi-modal feature vector; predicting the multi-modal feature vector by using a preset regression model to obtain a hardness prediction value; and classifying the hardness prediction value according to the environment data to obtain a hardness category of the target material. The application realizes high-precision soft and hard material recognition.

Description

Technical Field

[0001] This invention relates to the field of robot tactile perception technology, specifically to a method and system for identifying the softness and hardness of materials based on electronic skin. Background Technology

[0002] Precise perception of the hardness of objects is crucial for intelligent interaction between machines and the physical world. In fields such as intelligent manufacturing, medical diagnostics, and service robots, the rapid and accurate identification of the hardness of objects in contact is of paramount importance for achieving precision operations (such as flexible grasping) and intelligent decision-making (such as quality sorting and assisted palpation).

[0003] Existing technologies for material hardness identification have fundamental limitations, primarily manifested in insufficient information dimensions and poor environmental adaptability. On the one hand, traditional methods are mostly based on a single information dimension, capturing only one aspect of a material's hardness characteristics, making it difficult to comprehensively characterize the material's overall physical properties. This can easily lead to confusion and misjudgment when faced with objects of similar hardness but different materials. On the other hand, traditional methods generally ignore the complexity of real-world working environments. For example, changes in ambient temperature and individual differences in sensors can affect the stability of the identification system, resulting in inaccurate identification results and potentially causing serious decision-making risks in high-risk and critical scenarios. Summary of the Invention

[0004] In view of this, the present invention provides a method for identifying the hardness and softness of materials based on electronic skin, comprising:

[0005] Acquire pressure array data and environmental data collected by the electronic skin when it comes into contact with the target material;

[0006] Multimodal features of the pressure array data in spatial, temporal, frequency domain, and gradient domains are extracted and fused to obtain multimodal feature vectors.

[0007] The multimodal feature vector is predicted using a preset regression model to obtain the hardness prediction value;

[0008] The hardness prediction values ​​are classified based on the environmental data to obtain the hardness category of the target material.

[0009] Optionally, the material hardness / softness identification method based on electronic skin provided by the present invention further includes:

[0010] The hardness category is thermally encoded to obtain the hardness category code;

[0011] The confidence level is obtained by using a pre-set confidence verification model to verify the pressure array data, the multimodal feature vector, the hardness prediction value, and the hardness category.

[0012] The confidence level and the confidence threshold are compared to obtain the actual hardness category.

[0013] Optionally, the multimodal features include spatial feature vectors, temporal feature vectors, frequency domain feature vectors, and gradient feature vectors. The spatial feature vectors reflect the pressure distribution when the electronic skin contacts the target material. The temporal feature vectors reflect the dynamic velocity of pressure changes during the contact between the electronic skin and the target material. The frequency domain feature vectors reflect the vibration characteristics of the contact between the electronic skin and the target material. The gradient feature vectors reflect the spatial distribution of hardness of the target material.

[0014] Optionally, spatial feature extraction is performed on the pressure array data, including:

[0015] Pressure feature quantization is performed on the pressure array data of the current frame to obtain the spatial feature vector. The pressure feature quantization includes at least one of average pressure, pressure standard deviation, peak pressure, kurtosis, and skewness.

[0016] Optionally, time feature extraction is performed on the pressure array data, including:

[0017] Determine whether the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold;

[0018] If the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold, then the time difference mean and maximum rate of change of the pressure array data in the current frame sequence are calculated to obtain the time feature vector;

[0019] If the frame sequence length of the current pressure array data is less than the frame sequence length threshold, then a preset value is selected as the time feature vector.

[0020] Optionally, frequency domain feature extraction is performed on the pressure array data, including:

[0021] Perform a Fourier transform on the pressure array data of the current frame, and calculate the low-frequency energy and high-frequency energy according to a preset frequency division standard to obtain the frequency domain feature vector.

[0022] Optionally, gradient feature extraction is performed on the pressure array data, including:

[0023] The average gradient intensity is calculated on the pressure array data of the current frame to obtain the gradient feature vector.

[0024] Optionally, the environmental data includes ambient temperature, and the step of classifying the predicted hardness value based on the environmental data to obtain the hardness category of the target material includes:

[0025] The mapping relationship between the preset hardness value and the hardness category is corrected based on the ambient temperature to obtain the mapping relationship between the target hardness value and the hardness category.

[0026] The hardness prediction value is classified according to the mapping relationship between the target hardness value and the hardness category to obtain the hardness category of the target material.

[0027] Optionally, the material hardness and softness identification method based on electronic skin provided by the present invention further includes preprocessing the pressure array data, wherein the pressure array data is calibrated according to the real-time baseline value, and the calibrated pressure array data is subjected to multi-scale filtering and fusion.

[0028] A second aspect of the present invention provides a material hardness / softness identification system based on electronic skin, the system comprising: a pressure sensor array, an environmental sensor, a data acquisition module, and a processor; wherein the pressure sensor array is used to acquire pressure array data generated when the electronic skin contacts the target material, the environmental sensor is used to acquire environmental data when the electronic skin contacts the target material, the data acquisition module is used to acquire the pressure array data, and the processor is used to execute any one of the material hardness / softness identification methods based on electronic skin.

[0029] This invention achieves "multimodal feature fusion + real-time optimization". In step S1, the electronic skin can rapidly acquire pressure array data and environmental data in different application scenarios, providing a rich and realistic data foundation for subsequent analysis. This rapid data acquisition capability ensures the real-time performance of the system. Step S2 extracts features from multi-channel pressure array data from multiple dimensions including space, time, frequency domain, and gradient, and fuses them into multimodal feature vectors to comprehensively reflect material properties, achieving multimodal feature fusion and providing accurate and comprehensive information for identification. Step S3 uses a preset regression model to predict hardness values ​​from the multimodal feature vectors. This model optimizes parameters through grid search and ensures model reliability through cross-validation and multi-environment and multi-material testing. Step S4 considers the impact of environmental data on the sensor and classifies and corrects the hardness prediction values ​​to make the hardness category more consistent with the actual material conditions in real-world scenarios. This ensures that the system can quickly adapt to environmental changes in different scenarios, guaranteeing the accuracy and real-time performance of the identification results. The synergy between multimodal feature fusion and real-time optimization provides a strong guarantee for high-precision identification of soft and hard materials.

[0030] This invention also quantifies the reliability of identification based on the XGBoost discriminator. Through an innovative multi-dimensional input fusion approach and a series of rigorous verification steps, it comprehensively improves the accuracy and reliability of soft and hard material identification. During the verification process, the combination of one-hot encoding and sensor stability, along with the construction of multi-dimensional input data, allows the model to comprehensively consider multiple factors and fully utilize various types of information to verify the initial identification results. By comparing the confidence level with the threshold, the reliability of the identification results can be accurately determined, yielding the actual hardness category. This provides a more accurate and reliable identification basis for practical applications, demonstrating good practicality and promotional value, and providing decision-making support for high-risk scenarios. Attached Figure Description

[0031] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 This is a structural diagram of the material hardness and softness recognition system based on electronic skin in an embodiment of the present invention;

[0033] Figure 2 This is a flowchart of a material hardness / softness identification method based on electronic skin in an embodiment of the present invention;

[0034] Figure 3 This is a flowchart of another material hardness / softness identification method based on electronic skin in an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0037] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] like Figure 1 As shown, this embodiment of the invention provides a material hardness / softness identification system based on electronic skin, including: a pressure sensor array 11, an environmental sensor 12, a data acquisition module 13, and a processor 14; wherein, the pressure sensor array 11 is used to acquire pressure array data generated when the electronic skin contacts the target material, the environmental sensor 12 is used to acquire environmental data when the electronic skin contacts the target material, the data acquisition module 13 is used to acquire pressure array data, and the processor 14 is used to execute... Figure 2 A material hardness and softness identification method based on electronic skin.

[0039] For example, the pressure sensor array 11 is a 64×64 channel piezoresistive tactile sensor array with an area of ​​10cm×10cm. It is connected to the acquisition module 13 via an SPI (Serial Peripheral Interface) interface. Its sampling frequency is configurable from 50Hz to 100Hz, and the single-channel resolution is 0.1kPa. It supports the simultaneous acquisition of pressure distribution (spatial characteristics) and dynamic changes (temporal characteristics). The sensor surface is covered with a 0.5mm thick flexible base to adapt to curved surface contact scenarios.

[0040] The environmental sensor 12 can be a temperature and humidity sensor, which is connected to the processor 14 via I2C (Inter-Integrated Circuit) and has a sampling frequency of 10Hz for dynamically correcting thresholds.

[0041] The acquisition module 13 includes an 8-channel synchronous amplifier circuit (adjustable gain, range 10-100 times) and a 16-bit AD converter (conversion time <1μs). It is connected to the processor 14 via the SPI interface to ensure multi-channel data time synchronization (error <1ms).

[0042] Processor 14 is an embedded AI processor, such as NVIDIA Jetson Nano (ARM Cortex-A57 architecture, 4 cores 1.4GHz), which supports CUDA (Compute Unified Device Architecture) acceleration (128-core GPU) to meet the needs of real-time feature extraction and model inference (power consumption <10W, suitable for mobile robots).

[0043] The acquisition module 13 used in this invention is the core bridge connecting the physical sensors and the computing unit. It adopts a modular design, using a multi-channel synchronous sample and hold (S / H) circuit and a high common-mode rejection ratio (CMRR) (CMRR>100dB) instrumentation amplifier to ensure the synchronization of all channels in the 64×64 array within a microsecond-level time window (error <1ms), effectively avoiding dynamic characteristic distortion caused by signal transmission delay. Simultaneously, the module incorporates a built-in programmable gain amplifier (PGA) with a software-configurable gain range of 10-100 times. This allows it to adapt to a wide range of signals (0.01N~10N) generated by soft (low-pressure) and hard (high-pressure) material contacts, fully utilizing the dynamic range of the 16-bit ADC (Analog-to-Digital Converter) and ensuring signal integrity. The SPI interface operates in full-duplex mode, responsible not only for transmitting digitized sensor data but also for receiving configuration instructions from processor 14 (such as setting the sampling rate, trigger mode, and gain value), enabling flexible software control of the acquisition process. Furthermore, the acquisition module 13 employs low-temperature drift components and an independent regulated power supply. Its 5V / 2A power input undergoes multi-layer LC (Inductor-Capacitor) filtering, resulting in an output ripple of <10mV. This significantly reduces the impact of environmental noise and power fluctuations on weak tactile signals at the hardware level, providing a stable, reliable, and high-quality data source for subsequent high-precision recognition algorithms.

[0044] like Figure 2 As shown, a method for identifying the hardness and softness of materials based on electronic skin specifically includes:

[0045] S1, acquire pressure array data and environmental data collected by the electronic skin when it comes into contact with the target material.

[0046] In this embodiment, the electronic skin is a robotic component that integrates sensors and other functions, enabling it to collect pressure array data and environmental data from multiple channels when in contact with target materials. The specific carrier and target materials of the electronic skin vary depending on the application scenario. In intelligent robot flexible grasping applications, the electronic skin manifests as a sensor array on the surface of the robot gripper, contacting various objects such as tomatoes and metal parts. In automated agricultural product sorting applications, the electronic skin is a tactile sensor at the end of a robotic arm, contacting fruits on a conveyor belt, such as avocados and mangoes. In medical assisted diagnosis (robotic palpation) applications, the electronic skin is a component integrated into a handheld probe or the end of a minimally invasive surgical robot, contacting target tissue areas of the human body, such as the breast or thyroid gland.

[0047] S2 extracts multimodal features from the pressure array data in the spatial, temporal, frequency, and gradient domains, and then fuses these features to obtain a multimodal feature vector.

[0048] This step involves extracting features from multiple dimensions—space, time, frequency domain, and gradient—from the pressure array data. These features from different dimensions are then fused to form a multimodal feature vector. This fully explores the information in the data from multiple aspects, making subsequent analysis and applications based on this feature vector more accurate and comprehensive.

[0049] S3 uses a preset regression model to predict the multimodal feature vectors and obtain the hardness prediction value.

[0050] This embodiment presupposes an SVM (Support Vector Machines) regression model with an RBF (Radial Basis Function) kernel. The dataset used in the model consists of five material classes, with 300 samples per class, and each sample contains 100 frames of pressure array data (100Hz × 1s). For the RBF kernel, parameters were optimized through grid search, ultimately determining C=10 and γ=0.1. C serves as a penalty parameter, controlling the degree of punishment for misclassified samples; a larger C value increases the likelihood of overfitting, while a smaller C value increases the likelihood of underfitting. γ is the kernel coefficient, determining the distribution of the mapped data; a large γ value focuses on local data, leading to overfitting, while a small γ value considers a wider data range, leading to underfitting. Cross-validation showed that the model has high accuracy and can accurately predict multimodal feature vectors.

[0051] The specific test sample details are as follows:

[0052] Materials: Sponge (soft, Shore A=20), silicone (relatively soft, Shore A=40), PVC (medium, Shore A=60), ABS plastic (relatively hard, Shore A=80), aluminum alloy (hard, Shore A=90), 30 samples of each type (size 2cm×2cm×2cm).

[0053] Environmental conditions: ambient temperature (25℃), high temperature (40℃), slight vibration (0.1g acceleration), pressure sensor drift (4 hours of continuous operation).

[0054] S4. Classify the predicted hardness values ​​based on environmental data to obtain the hardness category of the target material.

[0055] In this embodiment, environmental data can affect the sensor's sensitivity, leading to deviations in the predicted hardness value. To ensure that the hardness category better reflects the actual situation of the target material in real-world scenarios, it is necessary to correct the hardness category using environmental data. The final hardness category can be divided according to the following reference table: "Soft": Hardness value range (-0.1, 0.15), corresponding Shore A < 30; "Relatively Soft": Hardness value range (0.15, 0.35), corresponding Shore A 30 ≤ Shore A < 50; "Medium": Hardness value range (0.35, 0.58), corresponding Shore A 50 ≤ Shore A < 70; "Relatively Hard": Hardness value range (0.58, 0.7), corresponding Shore A 70 ≤ Shore A < 85; "Hard": Hardness value range (0.7, 1.0), corresponding Shore A ≥ 85.

[0056] This embodiment achieves "multimodal feature fusion + real-time optimization". In step S1, the electronic skin can rapidly acquire multi-channel pressure array data and environmental data in different application scenarios, providing a rich and realistic data foundation for subsequent analysis. This rapid data acquisition capability ensures the real-time performance of the system. Step S2 extracts features from the pressure array data from multiple dimensions, including space, time, frequency domain, and gradient, and fuses them into a multimodal feature vector to comprehensively reflect material properties, achieving multimodal feature fusion and providing accurate and comprehensive information for identification. Step S3 uses a preset regression model to predict hardness values ​​from the multimodal feature vector. This model optimizes parameters through grid search and undergoes cross-validation and multi-environment and multi-material testing to ensure model reliability. Step S4 considers the impact of environmental data on the sensor and classifies and corrects the hardness prediction values ​​to make the hardness category more consistent with the actual material conditions in real-world scenarios. This ensures that the system can quickly adapt to environmental changes in different scenarios, guaranteeing the accuracy and real-time performance of the identification results. The synergy between multimodal feature fusion and real-time optimization provides a strong guarantee for high-precision identification of soft and hard materials.

[0057] like Figure 3 As shown, the material hardness and softness identification method based on electronic skin provided in this embodiment of the invention further includes:

[0058] S5. Perform unique thermal coding on the hardness category to obtain the hardness category code.

[0059] This embodiment converts hardness categories into binary vectors to facilitate model recognition. For example, textual information such as "soft," "medium," and "hard" cannot be directly used by the model. Through one-hot encoding, hardness categories are converted into binary vectors, with each dimension corresponding to a hardness category. Only the dimension corresponding to that category is 1, and the rest are 0. This allows the model to effectively calculate and analyze hardness category information, further improving the model's ability to process hardness information and providing more accurate input for subsequent verification and recognition.

[0060] S6 uses a pre-set confidence verification model to verify the data based on pressure array data, multimodal feature vectors, hardness prediction values, and hardness categories to obtain the confidence level.

[0061] The pre-set reliability verification model employs an XGBoost discriminator to verify the reliability of the identification results. This model comprehensively analyzes and verifies various information sources, including the initially collected pressure array data, multimodal feature vectors, hardness predictions obtained through a pre-set regression model, and hardness category codes. The pressure array data reflects the response of the identified object under pressure; the multimodal feature vectors describe the characteristics of the identified object from multiple dimensions; the hardness predictions provide preliminary hardness identification results; and the hardness category codes clarify the information for different hardness categories. By comprehensively utilizing this information, the pre-set reliability verification model can more comprehensively and accurately assess the reliability of the identification results, outputting a confidence value between 0 and 1 to measure the reliability of the prediction results.

[0062] The training scenario for the XGBoost discriminator is the same as the preset regression model in step S3. The data used to train this discriminator is extremely rich, containing over 2000 samples. These samples comprehensively cover five different types of materials and also consider three different environmental noise conditions to simulate more realistic and complex scenarios. During training, the XGBoost discriminator is set with a tree depth of 5 and a learning rate of 0.1 to ensure that the model, while possessing a certain level of complexity, can converge stably and efficiently, thereby improving the discriminator's performance and accuracy.

[0063] S7 compares the confidence level with the confidence threshold to obtain the actual hardness category.

[0064] The confidence level output by the model is compared with a pre-set confidence threshold. If the confidence level is greater than or equal to the threshold, the corresponding hardness category is considered reliable and is adopted as the actual hardness category. Otherwise, it indicates that the model's identification of the hardness category is inaccurate and may require further processing or re-evaluation. This comparison method based on confidence level and threshold can accurately determine the reliability of the identification results.

[0065] This embodiment quantifies the reliability of identification based on the XGBoost discriminator. Through an innovative multi-dimensional input fusion approach and a series of rigorous verification steps, it comprehensively improves the accuracy and reliability of soft and hard material identification. During the verification process, the combination of one-thermal encoding and sensor stability, along with the construction of multi-dimensional input data, allows the model to comprehensively consider multiple factors and fully utilize various types of information to verify the initial identification results. By comparing the confidence level with the threshold, the reliability of the identification results can be accurately determined, yielding the actual hardness category. This provides a more accurate and reliable identification basis for practical applications, demonstrating good practicality and promotional value, and providing decision-making support for high-risk scenarios.

[0066] In some optional embodiments of this example, the multimodal features include spatial feature vectors, temporal feature vectors, frequency domain feature vectors, and gradient feature vectors. The spatial feature vectors are used to reflect the pressure distribution of the electronic skin in contact with the target material, the temporal feature vectors are used to reflect the dynamic velocity of pressure changes during the contact between the electronic skin and the target material, the frequency domain feature vectors are used to reflect the vibration characteristics of the contact between the electronic skin and the target material, and the gradient feature vectors are used to reflect the spatial distribution of the hardness of the target material.

[0067] The spatial feature vector extracted in step S2 helps distinguish local hardness differences in materials by analyzing the spatial distribution differences of pressure when the electronic skin contacts materials of different hardness. The temporal feature vector is extracted by analyzing the dynamic characteristics of pressure changes over time when the electronic skin contacts the target material, reflecting the dynamic deformation of the target material under stress. The frequency domain feature vector is extracted by capturing the vibration frequency information when the electronic skin contacts the target material, providing information for material hardness identification from a frequency domain perspective. The gradient feature vector is extracted by analyzing the spatial distribution of material hardness, helping to identify hardness differences at different locations within the material. By integrating static, dynamic, and vibration features, the problem of insufficient information from a single dimension is fundamentally solved, providing accurate data for identifying the hardness category of the target material, thereby greatly improving the identification accuracy.

[0068] In some optional embodiments of this example, step S2, which involves extracting spatial features from the pressure array data, includes:

[0069] Pressure feature quantization is performed on the pressure array data of the current frame to obtain a spatial feature vector. The pressure feature quantization calculation includes at least one of the following: average pressure, pressure standard deviation, peak pressure, kurtosis, and skewness.

[0070] This embodiment performs spatial feature extraction on pressure array data. Specifically, it performs pressure feature quantification calculations on the pressure array data of the current frame, including calculating the average pressure to measure the overall contact strength, calculating the pressure standard deviation to assess the uniformity of the pressure distribution, calculating the peak pressure to reflect the local contact strength, calculating the skewness to determine the symmetry of the pressure distribution (the pressure distribution of soft materials is often more symmetrical), and calculating the kurtosis to determine the steepness of the pressure distribution (the pressure distribution of hard materials is usually steeper). Through at least one of these calculation methods, a spatial feature vector is finally obtained.

[0071] This embodiment extracts spatial features from pressure array data as described above, quantifying pressure characteristics from multiple key dimensions. Average pressure directly reflects the overall contact intensity; pressure standard deviation clearly shows whether the pressure distribution is uniform; peak pressure helps focus on localized high-intensity contact areas; skewness helps distinguish between soft and hard materials based on distribution symmetry; and kurtosis further determines material hardness based on the steepness of the distribution. These multi-dimensional pressure characteristic quantification indicators, combined, comprehensively and meticulously depict the pressure distribution, helping to more accurately capture spatial feature information in the data. These rich spatial feature vectors provide a more reliable and discriminative basis for subsequent analysis and identification.

[0072] In some optional embodiments of this example, step S2, which involves extracting time features from the pressure array data, includes:

[0073] Determine whether the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold; if the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold, calculate the mean of time difference and the maximum rate of change of the pressure array data in the current frame sequence to obtain the time feature vector; if the frame sequence length of the current pressure array data is less than the frame sequence length threshold, select a preset value as the time feature vector.

[0074] In step S2, when extracting time features from the pressure array data, it is first determined whether the frame sequence length of the total pressure array data collected so far reaches the frame sequence length threshold. If the frame sequence length threshold is reached, the mean time difference (reflecting deformation rate) and the maximum rate of change (reflecting response sensitivity) of all pressure array data in the current frame sequence are calculated to obtain the time feature vector. If the frame sequence length threshold is not reached, a preset value (such as [0, 0]) is selected as the time feature vector. The first 0 in [0, 0] indicates that if the frame sequence length of the current pressure array data is less than the threshold (preset to 5), the deformation rate is assumed to be 0. The second 0 can be regarded as the default value of the maximum rate of change, that is, the response sensitivity is assumed to be 0.

[0075] This embodiment determines the calculation method by judging the frame sequence length. When the frame sequence length is sufficient, calculating the mean of time differences and the maximum rate of change can effectively reflect the dynamic change speed of the pressure data, providing key information about deformation rate and response sensitivity for subsequent analysis. For cases where the frame sequence length is insufficient, a preset value is used to avoid inaccurate calculation results or inability to calculate due to insufficient data, ensuring the continuity and stability of time feature extraction and helping to improve the system's analysis and processing capabilities under different data conditions.

[0076] In some optional embodiments of this example, step S2, which involves extracting frequency domain features from the pressure array data, includes:

[0077] Perform a Fourier transform on the pressure array data of the current frame, and calculate the low-frequency energy and high-frequency energy according to the preset frequency division standard to obtain the frequency domain feature vector.

[0078] When extracting frequency domain features from the pressure array data in step S2, a Fourier transform is performed on the pressure array data of the current frame. Then, based on the preset frequency division standard, the low-frequency energy (e.g., 0-10Hz) (where soft materials have a high energy ratio) and high-frequency energy (e.g., 10-50Hz) (where hard materials have a high energy ratio) are calculated to obtain the frequency domain feature vector.

[0079] This embodiment converts pressure data to the frequency domain using Fourier transform to calculate low-frequency and high-frequency energy, effectively reflecting contact vibration characteristics. Different materials exhibit varying energy distributions across different frequency bands; this characteristic can be used to help distinguish between hard and soft materials, providing a strong basis for material property identification. The frequency domain feature vectors provide richer and more discriminative information for subsequent data analysis and processing, contributing to improved accuracy and reliability in material property identification.

[0080] In some optional embodiments of this example, step S2 involves gradient feature extraction of the pressure array data, including:

[0081] The average gradient intensity is calculated on the pressure array data of the current frame to obtain the gradient feature vector.

[0082] In step S2, when extracting gradient features from the pressure array data, the average gradient intensity is calculated for the pressure array data of the current frame. This calculation yields a gradient feature vector that reflects the spatial distribution of hardness. Since hard materials have larger gradients, the hardness of the target material can be determined based on the gradient.

[0083] This embodiment provides a quantitative indicator for judging the spatial distribution of material hardness by calculating the average gradient intensity. Utilizing the characteristic that hard materials have larger gradients, it can help identify the hardness of materials in different regions. The gradient feature vector provides more targeted information for subsequent analysis and decision-making, helping to improve the accuracy of judging the material hardness distribution.

[0084] In some optional embodiments of this example, the environmental data obtained in step S2 includes the ambient temperature. Therefore, in step S4, the hardness prediction values ​​are classified according to the environmental data to obtain the hardness category of the target material, including:

[0085] S41, Based on the ambient temperature, the mapping relationship between the preset hardness value and the hardness category is corrected to obtain the mapping relationship between the target hardness value and the hardness category.

[0086] In this embodiment, the mapping relationship between the preset hardness value and the hardness category is constructed based on a reference temperature of 25°C. Since changes in ambient temperature affect the sensor's measurement results, and consequently the threshold used to determine material hardness, the mapping relationship between the preset hardness value and the hardness category needs to be adjusted accordingly when the ambient temperature changes. For example, for every 5°C change in temperature, the hardness value for each hardness category is adjusted by ±2%; that is, if the temperature increases by 5°C, the hardness value for each hardness category increases by 2%; if the temperature decreases by 5°C, the hardness value for each hardness category decreases by 2%. This allows the threshold to be dynamically adjusted according to the ambient temperature to adapt to measurements under different temperature conditions, improving the accuracy of the judgment.

[0087] S42, classify the predicted hardness value according to the mapping relationship between the target hardness value and the hardness category to obtain the hardness category of the target material.

[0088] The hardness category of the target material can be obtained by comparing the predicted hardness value obtained from the preset regression model with the mapping relationship.

[0089] This embodiment corrects the preset mapping relationship between hardness values ​​and hardness categories based on ambient temperature. It fully considers that changes in ambient temperature can affect sensor measurement results and hardness judgment thresholds, adapting to measurements under different temperature conditions and avoiding judgment errors caused by temperature fluctuations. Then, based on the corrected mapping relationship between the target hardness value and hardness category, the predicted hardness value is classified to obtain the hardness category of the target material. This classification method based on accurate mapping relationships, combined with the advantages of temperature correction, makes the determination of material hardness more precise and reliable.

[0090] In some optional embodiments of this example, the pressure array data is further preprocessed, wherein the pressure array data is calibrated according to the real-time baseline value, and the calibrated pressure array data is subjected to multi-scale filtering and fusion.

[0091] Because pressure sensor arrays may drift during use, causing deviations in the acquired pressure array data, errors caused by drift can be effectively eliminated by subtracting the baseline value from the original sensor pressure array data. This baseline value is updated in real time (the initial baseline value is calculated using 100 frames of idle data, and is updated every 10 frames of pressure array data, with a smoothing factor of 0.1). Multi-scale filtering fusion includes channel consistency compensation, multi-scale noise suppression, basic median filtering, time-domain exponential smoothing, and spatial-domain Gaussian filtering.

[0092] Specifically, channel consistency compensation: Because individual channels of the pressure sensor array may have individual differences, this can affect the accuracy of the measurement data. Therefore, a channel calibration matrix (used to compensate for individual differences) is used to process the pressure array data. The channel calibration matrix is ​​obtained during factory testing and can adjust the data of each channel to make the data between channels more consistent, keeping the error of each channel within, for example, 2%.

[0093] Multi-scale noise suppression: The stress array data is processed using a 3×3 kernel size. It sorts all pixel values ​​within a window and takes the median value as the new value of the center pixel of that window, which can effectively eliminate impulse noise, because impulse noise usually manifests as abnormally high or low values, which can be filtered out by taking the median value.

[0094] Basic median filtering: Exponential smoothing is a weighted averaging method that uses a smoothing coefficient of 0.3 to exponentially smooth the pressure array data, balancing real-time performance and smoothness.

[0095] Time-domain exponential smoothing: Butterworth filter with a cutoff frequency of 50Hz is used to filter out high-frequency noise above 50Hz.

[0096] Spatial Domain Gaussian Filtering: Gaussian filtering is performed with a standard deviation of 1.0. Gaussian filtering is a linear smoothing filter that uses a weighted average of pixel values ​​in the neighborhood based on the weights of the Gaussian function. This smooths out the distribution of pixels and reduces spatial noise and abrupt changes.

[0097] This embodiment eliminates errors caused by sensor drift through dynamic baseline calibration, and channel consistency compensation ensures the consistency of data across all channels. Multi-scale filtering fusion processes the data from different perspectives; basic median filtering eliminates impulse noise; time-domain exponential smoothing balances real-time performance and smoothness; frequency-domain low-pass filtering removes high-frequency noise; and spatial-domain Gaussian filtering smooths the pressure distribution, effectively suppressing various types of noise. These preprocessing steps improve the quality and reliability of sensor data, providing a more accurate and stable foundation for subsequent material hardness identification.

[0098] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for identifying the hardness and softness of materials based on electronic skin, characterized in that, include: Acquire pressure array data and environmental data collected by the electronic skin when it comes into contact with the target material, wherein the environmental data includes ambient temperature; Multimodal features of the pressure array data in spatial, temporal, frequency domain, and gradient domains are extracted and fused to obtain multimodal feature vectors. The multimodal feature vector is predicted using a preset regression model to obtain the hardness prediction value; The mapping relationship between the preset hardness value and the hardness category is corrected based on the ambient temperature to obtain the mapping relationship between the target hardness value and the hardness category. The hardness prediction value is classified according to the mapping relationship between the target hardness value and the hardness category to obtain the hardness category of the target material.

2. The method according to claim 1, characterized in that, Also includes: The hardness category is thermally encoded to obtain the hardness category code; The confidence level is obtained by using a pre-set confidence verification model to verify the pressure array data, the multimodal feature vector, the hardness prediction value, and the hardness category. The confidence level and the confidence threshold are compared to obtain the actual hardness category.

3. The method according to claim 1, characterized in that, The multimodal features include spatial feature vectors, temporal feature vectors, frequency domain feature vectors, and gradient feature vectors. The spatial feature vectors reflect the pressure distribution when the electronic skin contacts the target material. The temporal feature vectors reflect the dynamic velocity of pressure changes during the contact between the electronic skin and the target material. The frequency domain feature vectors reflect the vibration characteristics of the contact between the electronic skin and the target material. The gradient feature vectors reflect the spatial distribution of hardness of the target material.

4. The method according to claim 3, characterized in that, Spatial feature extraction of the pressure array data includes: Pressure feature quantization is performed on the pressure array data of the current frame to obtain the spatial feature vector. The pressure feature quantization includes at least one of average pressure, pressure standard deviation, peak pressure, kurtosis, and skewness.

5. The method according to claim 3, characterized in that, Extracting time features from the pressure array data includes: Determine whether the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold; If the frame sequence length of the current pressure array data is greater than or equal to the frame sequence length threshold, then the time difference mean and maximum rate of change of the pressure array data in the current frame sequence are calculated to obtain the time feature vector; If the frame sequence length of the current pressure array data is less than the frame sequence length threshold, then a preset value is selected as the time feature vector.

6. The method according to claim 3, characterized in that, Frequency domain feature extraction of the pressure array data includes: Perform a Fourier transform on the pressure array data of the current frame, and calculate the low-frequency energy and high-frequency energy according to a preset frequency division standard to obtain the frequency domain feature vector.

7. The method according to claim 3, characterized in that, Gradient feature extraction is performed on the pressure array data, including: The average gradient intensity is calculated on the pressure array data of the current frame to obtain the gradient feature vector.

8. The method according to claim 1, characterized in that, It also includes preprocessing the pressure array data, wherein the pressure array data is calibrated according to the real-time baseline value, and the calibrated pressure array data is subjected to multi-scale filtering and fusion.

9. A material hardness / softness recognition system based on electronic skin, characterized in that, include: The system comprises a pressure sensor array, an environmental sensor, a data acquisition module, and a processor; wherein the pressure sensor array is used to acquire pressure array data generated when the electronic skin contacts the target material, the environmental sensor is used to acquire environmental data when the electronic skin contacts the target material, the data acquisition module is used to acquire the pressure array data, and the processor is used to execute the material hardness / softness identification method based on electronic skin as described in any one of claims 1-8.

Images