A tactile sensing device and a method for analyzing the hardness and flatness of an object

Abstract

The present invention discloses a tactile sensor device, comprising a sliding bracket driven by a robotic arm, a sensing link, a spring, an FBG tactile sensor, and an optical path system. The FBG tactile sensor comprises at least three bionic fingers, each containing an FBG sensor encapsulated within a flexible structure; a sliding bracket having an internal groove for accommodating the sensing link and the spring; the sensor link comprising a thin rod and a thick rod, with a slider positioned laterally above the thick rod, the lower portion of which extends out of the internal groove of the sliding bracket; a slideway mateable with the link slider positioned laterally within the internal groove of the sliding bracket, and a hole formed above the groove for allowing the thin rod to pass through. The present invention also provides a method for analyzing the hardness and flatness of an object using the aforementioned tactile sensor device.

Description

Technical Field

[0001] The present invention relates to a bionic sliding tactile sensing system, specifically a sliding tactile sensing device based on a fiber Bragg grating (FBG) sensor, which can detect and classify the hardness and flatness of an object and is suitable for the fields of robot perception, prosthetic tactile feedback, and complex human-computer interaction. Background Art

[0002] With the widespread application of bioinspired design in engineering, bionics has played a key role in sensor design. Traditional sensors typically detect only single signals, such as pressure or temperature, and are unable to simultaneously detect multiple mechanical stimuli in complex and changing environments. However, tactile receptors in human skin can simultaneously sense multiple mechanical stimuli, enabling detailed perception of the external environment through the integrated perception of multiple signals such as pressure and vibration. This provides an important theoretical basis for the design of new multimodal sensors. Conventional multimodal sensors face challenges such as electromagnetic interference and parasitic circuits. Fiber Bragg Grating (FBG) sensors, on the other hand, not only enable highly sensitive strain detection but also exhibit resistance to electromagnetic interference, miniaturization, and lightweight properties, making them ideally suited for integration into biomimetic sensing systems. Furthermore, because FBG sensors can integrate multiple sensing points within a single optical fiber, they offer greater integration and resolution, and hold broad application prospects in multimodal signal fusion and processing.

[0003] Multimodal sensor technology still faces many challenges, including multimodal data redundancy, processing complexity, and sensor stability and durability. Multimodal sensing systems based on FBG sensors, due to their high sensitivity and anti-interference performance, are an important means of addressing these issues. Given the shortcomings of traditional multimodal sensors, designing a tactile sensing system capable of simultaneously detecting the softness, hardness, and flatness of an object has become a pressing technical challenge.

[0004] Patent application 2024111204716 provides a visual-tactile multimodal object recognition method based on deep learning, and elaborates in detail the method of processing sliding tactile data and constructing a tactile sliding dataset. However, the data collected by the sliding tactile sensing device of this patent is not accurate enough, and it needs to be combined with visual information to identify objects. Summary of the Invention

[0005] This invention provides a device capable of accurately collecting multimodal tactile data and a method for analyzing the hardness and flatness of objects based on this device. The invention constructs a tactile sensing system, designs and integrates FBG sensors encapsulated in a three-finger structure, and collects data from silicone blocks of varying hardness and surface protrusions through sliding tests. Demodulation techniques are used to obtain three-channel stress and vibration signals, and a multimodal fusion algorithm is used to identify and analyze the collected multimodal data. The technical solutions provided by this invention are as follows:

[0006] A sliding tactile sensing device includes a sliding bracket driven by a mechanical arm, a sensing link, a spring, an FBG sliding tactile sensor and an optical path system, wherein:

[0007] The FBG tactile sensor consists of at least three bionic fingers, each of which contains an FBG sensor. The FBG sensor senses external stress and vibration signals by detecting the wavelength and phase changes of the Bragg grating in the optical fiber. The FBG sensor is encapsulated in a flexible structure:

[0008] The flexible structure includes three gaskets, the FBG sensor is encapsulated between the upper gasket and the middle gasket, and the lower part of the middle gasket is provided with a recess;

[0009] Lower gasket: directly contacts the test object, and its surface is textured to increase friction during sliding, amplify vibration caused by friction, and improve the sensitivity of vibration signals. The lower gasket is designed with protrusions that match the depressions of the middle gasket.

[0010] The sliding bracket is used to drive the bionic finger to move along the test surface under the drive of the robotic arm. The sliding bracket has a groove inside for accommodating a sensor connecting rod and a spring. The sensor connecting rod includes a thin rod and a thick rod. A slider is provided at the upper side of the thick rod. The lower part of the thick rod extends out of the groove inside the sliding bracket. The upper end of each bionic finger is fixedly connected to the lower part of the thick rod. A slideway is provided on the side of the groove inside the sliding bracket to cooperate with the connecting rod slider. A hole is provided above the groove to allow the thin rod to pass through. The spring is sleeved on the outer circumference of the thin rod, and its two ends respectively press against the inner surface of the groove inside the sliding bracket and the upper surface of the thick rod.

[0011] A limiting piece for preventing the connecting rod slider from moving out of the sliding bracket is provided at the lateral lower part of the inner groove of the sliding bracket.

[0012] Furthermore, the optical system includes a light source part, a sensor part and a data acquisition part;

[0013] The light source uses a wavelength-tunable laser with a base filter, which achieves self-excited amplification through an erbium-doped fiber amplifier (EDFA), providing stable energy output for the optical system. The polarization state of the light is controlled by a first polarization controller, and the light beam then enters the first coupler, where the optical power is distributed. A portion enters the sensing part, assuming n bionic fingers are used, and is divided into n paths through a second coupler. The other portion returns to the light source, is filtered again by an FFP-TF filter, and returns to the EDFA through the second polarization controller to form a closed loop. An arbitrary waveform generator is used to generate a sawtooth wave signal to tune the filter.

[0014] In the sensing part, the optical path system adopts a Michelson interferometer structure, and each path obtains coherent output light.

[0015] Furthermore, the filter is an FFP-TF filter.

[0016] Furthermore, in the sensing part, the reference arm is connected to a Faraday rotator mirror to reflect the incident light transmitted from the light source part back to the system; the sensing arm uses the FBG sensor of each bionic finger to reflect the light that matches its Bragg wavelength. For the FBG sensor, only the incident light corresponding to the Bragg grating wavelength can be reflected, and light of other wavelengths is transmitted; the reflected light from the reference arm and the sensing arm is coupled in a third coupler to form coherent output light.

[0017] Furthermore, in the data acquisition part, the coherent output light is converted into an electrical signal by a photodetector, which is then collected by a data acquisition card and transmitted to the host computer for collection and processing.

[0018] The present invention also provides a method for analyzing the hardness and flatness of an object using the sliding tactile sensing device, comprising the following steps:

[0019] Acquisition and processing of sliding tactile signals: A robotic arm controls the sliding tactile sensing device to slide across the surface of the object to be measured, collecting sample data. During the sliding tactile signal processing, the stress signal collected by the FBG sensor is quantified through the wavelength shift of the fiber Bragg grating, and the vibration signal is extracted through the phase change of the coherent signal. The data is then collected by a data acquisition card and transmitted to the host computer for processing. Each FBG sensor 8 can simultaneously output multimodal data including stress mode and vibration mode.

[0020] Multimodal tactile data fusion: Multimodal data contains n*2 input signals from various FBG sensors, namely the stress signals and vibration signals of each channel; the data is processed by the low-rank multimodal fusion network LMF to generate the final classification and recognition results.

[0021] Furthermore, the data is processed by a low-rank multimodal fusion network LMF to generate the final classification and recognition results as follows: the LMF network first extracts a single-modal representation of the data of each modality, and uses an LSTM network to independently process each input modality to extract a single-modal representation from it; the single-modal representation is expanded through the hidden state to convert the multimodal input into a high-dimensional tensor, and then mapped back to the low-dimensional output vector space; the LMF network adopts low-rank tensor decomposition technology to decompose the weight tensor into multiple modality-specific vector groups, and generates a low-rank approximate multimodal fusion representation through the outer product of the vector groups; the fused representation is processed by a linear layer to integrate and transform the high-dimensional features into multi-dimensional output; the Softmax activation function is used to convert the output of the linear layer into a probability distribution of the category, thereby generating the final classification and recognition result; each dimension of output is a classification output.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] (1) Compared to traditional sensors, the present invention can simultaneously measure static stress and dynamic vibration signals within a single sensor, making it suitable for complex multimodal signal detection. The three-finger design improves the system's detection stability and data reliability. The FBG sensor integrates multiple sensing points within a single optical fiber, providing higher integration and resolution.

[0024] (2) Compared with traditional multimodal signal fusion networks, the present invention utilizes modality-specific low-order factors to perform multimodal fusion, avoiding the calculation of high-dimensional tensors, reducing memory overhead, reducing exponential time complexity to linear, and achieving high-precision classification.

[0025] (3) The present invention has a modular design, simple structure, easy manufacturing and strong adaptability. It can be widely used in fields such as robotic arms, prostheses and human-computer interaction, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the overall flow chart of the system of the present invention.

[0027] Figure 2 It is a three-finger structural design diagram of the sliding tactile sensing device of the present invention.

[0028] Figure 3 This is a diagram of the optical path structure of the FBG slip tactile sensing system of the present invention.

[0029] Figure 4 It is a schematic diagram of dual-parameter demodulation of a single FBG sensor of the present invention.

[0030] Figure 5 This is a diagram of the multimodal fusion network structure used in the present invention to perform hardness and flatness analysis.

[0031] Figure 6 This is a five-fold cross-validation result diagram of the multimodal fusion network used in the present invention to perform hardness and flatness analysis.

[0032] Figure 7 This is an analysis diagram of the object flatness recognition accuracy studied in the present invention.

[0033] The accompanying drawings are as follows

[0034] 1. Sliding bracket 2. Sensing link 3. Spring 4. Link slider 5. Limiter 6. Slider 7. Upper gasket 8. FBG sensor 9. Middle gasket 10. Lower gasket 11. Erbium-doped fiber amplifier 12. First polarization controller 13. First coupler 14. FFP-TF filter 15. Second polarization controller 16. Second coupler 17. Third coupler 18. Faraday rotator 19. Photodetector 20. Data acquisition card 21. Arbitrary waveform generator 22. Host computer 23. Internal groove of the sliding bracket DETAILED DESCRIPTION

[0035] The following further illustrates how to use the object hardness and flatness analysis device based on the FBG tactile sensing system proposed in the present invention to distinguish the hardness and flatness of different objects with reference to the accompanying drawings and formulas.

[0036] The present invention is achieved through the following technical solutions: Figure 1 As shown, the specific steps of the present invention are:

[0037] The first step is to build a tactile sensing system

[0038] The sliding tactile sensing system constructed by the present invention is as follows Figure 2 As shown in Figure 2, the tactile sensor consists of three bionic fingers arranged in an isosceles triangle. This arrangement has demonstrated several advantages in experiments:

[0039] The isosceles triangle structure provides a stable contact surface, allowing the finger to better adapt to surface contours, ensuring reliable contact with the surface and maintaining high sensitivity when collecting stress and vibration data.

[0040] The redundant design of the three bionic fingers reduces measurement inaccuracies caused by errors in a single sensor, ensuring data reliability.

[0041] Each FBG sensor collects surface information from different angles. Through multimodal data fusion technology, stress and vibration signals are effectively combined, significantly improving the system's ability to distinguish different surfaces.

[0042] Each finger is integrated with an FBG sensor 8. The FBG sensor 8 senses external stress and vibration signals by detecting the wavelength and phase changes of the Bragg grating within the optical fiber. To improve detection accuracy, the FBG sensor 8 is encapsulated in a flexible structure consisting of three layers of gaskets:

[0043] Upper gasket 7: used to protect the sensor and ensure the stability of signal transmission.

[0044] Middle gasket 9: The FBG sensor 8 is encapsulated between the upper and middle gaskets and fixed with silicone adhesive to ensure a tight fit between the sensor and the gasket. The lower half of the middle gasket is recessed.

[0045] Lower gasket 10: This directly contacts the test object. Its surface is textured to increase friction during sliding, amplify friction-induced vibration, and enhance sensitivity to vibration signals. The lower gasket also features a raised center. Its shape matches the recessed shape of middle gasket 9, ensuring a tight fit between the three gasket layers. The raised center serves to enhance pressure sensitivity for the FBG sensor 8 mounted above it.

[0046] The device uses a robotic arm to drive a sliding bracket 1, which then moves the three bionic fingers along the test surface. A groove 23 is located within the sliding bracket 1 to accommodate a sensing link 2 and a spring 3, ensuring that the bionic fingers can move up and down with changes in height when encountering a bump 6.

[0047] The sensor link 2 is the core structure of the tactile sensing system, responsible for transmitting mechanical signals. When the bionic finger encounters the protrusion 6, the sensor link 2 moves upward, and the spring 3 provides elastic support for the sensor link 2, allowing it to move flexibly up and down while maintaining stability. The position of the spring 3 is constrained in the groove of the sliding bracket 1, so that the sensor link 2 is properly controlled by the elastic force during its up and down movement, avoiding damage or offset due to excessive displacement. The spring 3 not only enables the sensor link 2 to move up and down precisely with the changes in the surface protrusion 6, but also ensures that the sensor link 2 automatically returns to its initial position when no external force is applied, allowing for continuous detection.

[0048] To prevent the sensing link 2 from rotating during its vertical movement, a connecting rod slider 4 is affixed to it. This slider 4 slides in conjunction with a guideway (not shown) in a groove within the sliding bracket 1, effectively limiting the sensing link 2's rotational freedom to vertical movement. This structural design ensures accurate transmission of sensing signals and avoids errors caused by rotation. Furthermore, a stopper 5 is mounted at the bottom of the sliding bracket 1. This stopper 5 only allows the sensing link 2 to pass through its opening while preventing the connecting rod slider 4 from exiting the sliding bracket 1, further ensuring the sensing link 2's range and direction of motion.

[0049] The lower gasket 10 at the bottom of the device is in close contact with the test surface. Through the action of the sensing link 2, it can directly contact and sense subtle changes in the test surface's features, such as surface protrusions 6. During the sliding process, when the lower gasket 10 contacts the protrusions 6, the device can accurately capture the mechanical response caused by changes in hardness and unevenness. The overall structure, through the interaction of the sliding bracket 1, spring 3, connecting rod slider 4, and limiter 5, ensures that the sensing link 2 can move up and down stably during the detection process, sensing the mechanical characteristics of different surfaces, thereby achieving accurate detection of the hardness and flatness of the object.

[0050] The sensing device consists of a sliding bracket 1, a sensing link 2, a spring 3, a link slider 4, and a limiter 5. The following is a detailed description of each component, including its function and its constraint relationship with other components:

[0051] Sliding bracket 1:

[0052] The sliding bracket, connected to the robotic arm, is the core moving structure of the entire device. It drives the sensing link 2 and the bionic finger to slide across the test surface, simulating tactile detection. The sliding bracket 1 has an internal groove for accommodating the sensing link 2 and spring 3, ensuring sufficient space for the sensing link 2 to move up and down. Lateral guideways within the internal groove of the sliding bracket mate with the connecting rod slider 4 to prevent the sensing link 2 from rotating. Furthermore, a limiter 5 restricts the vertical range of motion of the sensing link 2, ensuring that its vertical movement remains within a controlled range.

[0053] Sensing link 2:

[0054] The sensing link 2 is the core component of the tactile sensing system, transmitting the force of external surface contact to the sensor and capturing subtle changes in surface bumps 6 or hardness. During testing, the sensing link 2 moves up and down, providing vertical feedback to the sensor. A connecting rod slider 4 is mounted on the sensing link 2, limiting its rotational freedom and maintaining its position during vertical movement. The vertical movement of the connecting rod is limited by a stopper 5 and a spring 3, ensuring that it moves only vertically within the sliding bracket 1 and prevents excessive deflection.

[0055] Spring 3:

[0056] Spring 3, mounted above the sensing link 2, provides elastic support and an appropriate reaction force during its upward and downward movement. During sliding, spring 3 ensures that the sensing link 2 rises and falls freely with the surface protrusions, while returning to its initial position when no external force is applied. Spring 3, confined within a groove in the sliding bracket 1, ensures its secure position and effectively controls the vertical displacement of the sensing link 2, preventing it from exceeding its normal range and potentially causing sensor damage or measurement errors.

[0057] Connecting rod slider 4:

[0058] Connecting rod slider 4 is fixed to sensing link 2 to prevent it from rotating during vertical motion. This slider slides along a lateral path within the internal groove of the sliding bracket, constraining it against the sliding bracket 1 and limiting the rotational freedom of sensing link 2. This design ensures the stability of sensing link 2 during vertical motion, limiting its vertical displacement when subjected to force, and ensuring accurate sensor signal transmission.

[0059] Limiter 5

[0060] The stopper 5, installed at the bottom of the sliding bracket 1, serves as a retaining structure. The stopper 5 has a small opening. When not mounted and secured to the sliding bracket 1, it only allows passage of the connecting rod slider 4 of the sensing link 2. Once the connecting rod slider 4 is inserted into the internal groove of the sliding bracket 1, the stopper 5 restricts the sliding movement of the connecting rod slider 4 within the slideway and prevents it from moving downward and out of the sliding bracket. This design ensures that the sensing link 2 can only move up and down within a certain range, preventing the connecting rod slider 4 from sliding out of the sliding bracket and maintaining the structural stability of the entire device.

[0061] Bump 6:

[0062] The bumps 6 provide a measurement target for the tactile sensor, simulating surfaces of varying hardness and unevenness. During the device's sliding test, the bumps 6 on the test surface trigger the bionic finger and sensing link 2 to move up and down, transmitting information such as surface hardness and height changes, providing mechanical response data to the sensor.

[0063] Through the coordinated operation of these components, the device can detect the hardness and flatness characteristics of an object's surface during sliding. The mutual constraints among the sliding bracket 1, sensing link 2, spring 3, link slider 4, and limiter 5 ensure that the sensing link 2 can only move up and down within a controlled range, thus enabling the tactile sensor to stably detect surface features.

[0064] Optical system such as Figure 3 As shown in the figure, it is mainly divided into light source part, sensing part and data acquisition part.

[0065] In the light source, the system utilizes a wavelength-tunable laser based on an FFP-TF filter 14. Self-excited amplification is achieved through an Erbium-doped fiber amplifier 11, providing stable energy output. Because light polarization significantly impacts the stability of the interferometer system, a first polarization controller 12 is incorporated into the system to control the polarization state and mitigate the adverse effects of polarization effects. The light beam then enters the first coupler 13, where the optical power is evenly distributed. A portion enters the sensing section, where it is split into three paths by the second coupler 16. The remaining portion returns to the light source, where it is filtered again by the FFP-TF filter 14 and returned to the Erbium-doped fiber amplifier 11 through the second polarization controller 15, completing the loop. The system uses an arbitrary waveform generator 21 to generate a sawtooth signal to tune the FFP-TF filter 14. In the sensing section, the system employs a Michelson interferometer structure. A Faraday rotator mirror 18 is connected to the reference arm, which reflects incident light from the light source back into the system. The sensing arm utilizes an FBG sensor 8 to reflect light that matches its Bragg wavelength. For FBG sensor 8, only incident light corresponding to the Bragg grating wavelength is reflected, while light of other wavelengths is transmitted. The reflected light from the two arms is coupled in a third coupler 17 to form coherent output light. In the data acquisition section, the coherent output light is converted into an electrical signal by a photodetector 19, which is then collected and processed by a data acquisition card 20.

[0066] The relevant parameters of the system's optical path are as follows: The power of the erbium-doped fiber amplifier (11) is adjusted to 13.6 dBm, which is sufficient for interferometric measurement and has relatively low noise. The three FBG sensors (8) used in the experiment have a central wavelength of 1551 nm and a corresponding size of 1 mm. Using this wide-spectrum, small-sized FBG sensor (8) allows for modulation of more vibration information and high spatial resolution. The optical path difference of the three Michelson interferometer structures is controlled to 5 cm ± 5 mm, resulting in an interference signal frequency of approximately 5 kHz. The wavelength scanning range of the light source must cover the main peak of the FBG sensor and its variation range. The voltage scanning range of the arbitrary waveform generator (21) is set to 12.6 V to 13.0 V, corresponding to a wavelength scanning range of 1550.285 nm to 1554.215 nm. The maximum tuning frequency of the FFP-TF filter (14) is 800 Hz, but signal interference increases with increasing tuning frequency. In order to ensure clean scanning light, the tuning frequency of the FFP-TF filter 14 is 50 Hz and the sampling rate of the data acquisition card 20 is 100 kSa / s.

[0067] The second step is the collection and processing of sliding tactile signals

[0068] The experimental objects were silicone blocks with hardness of 30, 50, 70, and 90, and protrusion heights of 1, 2, 3, and 4 mm. By combining two characteristics in pairs, a total of 16 types of experimental objects were obtained. The system used a robotic arm to control the sensor device to slide over the surface of these objects, collecting 14,400 sample data.

[0069] During the tactile signal processing, the stress signal collected by the FBG sensor 8 is quantified by the wavelength shift of the fiber Bragg grating, while the vibration signal is extracted by the phase change of the coherent signal. The data is then collected by the data acquisition card 20 and transmitted to the host computer 22 for processing.

[0070] Each FBG sensor 8 can output stress mode and vibration mode data simultaneously. The data processing process is as follows: Figure 4 .

[0071] Patent application 2024111204716 provides a method for processing sliding tactile data for stress and vibration and constructing a tactile sliding dataset. This method can meet the requirements of the present invention for a sliding tactile dataset, so we use the same processing method to construct a sliding tactile dataset.

[0072] The three raw interference signals were processed to obtain the stress information detected by the three FBG sensors 8 at their respective locations during the sliding process, which served as the stress mode data set. Phase changes were primarily observed in the frequency domain. Because the amplitude of dynamic vibration signals in the kHz range is much smaller than that of static stress, the static stress and dynamic vibration measurements were effectively separated in the time and frequency domains, verifying that the present invention can achieve simultaneous dual-parameter measurement of static stress and dynamic vibration.

[0073] The third step is to integrate multimodal tactile data.

[0074] After completing data collection, the system processes the data through a low-rank multimodal fusion network (LMF). The LMF structure is as follows: Figure 5 As shown. The multimodal data includes six input signals from three FBG sensors 8, namely, three stress signals and vibration signals. In order to effectively fuse these multimodal information, the LMF network first extracts the single-modal representation of each modal data. Since the data set of the present invention contains six modes, it is necessary to design six single-modal sub-networks. Since the six input signals {x ch1_s ,x ch1_vib ,x ch2_s ,x ch2_vib ,x ch3_s ,x ch3_vib} are all time series, so the LSTM network is used to process each input modality independently and extract the single modality representation {z ch1_s ,z ch1_vib ,zch2_s ,z ch2_vib ,z ch3_s ,z ch3_vib}.

[0075] Next, the single-modal representation is expanded through the hidden state to generate a high-dimensional tensor that captures the complex interaction information between the modalities. This process first converts the multimodal input into a high-dimensional tensor and then maps it back to the low-dimensional output vector space. The input tensor Z is an M-order tensor, where M represents the total number of modal information inputs. The weight tensor W is an M+1-order tensor with dimensions d1×d2×...×d M ×d h , where the additional M+1th layer represents the output dimension d h ; b is the bias term. The entire conversion process is shown in the figure, and the calculation formula is as follows:

[0076]

[0077] In order to reduce the computational complexity and the number of parameters, the LMF network uses low-rank tensor decomposition technology to represent the weight tensor in the fusion process as a modality-specific low-rank factor. Specifically, the LMF network decomposes the weight tensor into multiple modality-specific vector groups, and generates a low-rank approximate multimodal fusion representation through the outer product of these vector groups. This method not only greatly reduces the computational complexity and the number of model parameters, but also effectively retains the interaction information between modalities. Low-rank tensor decomposition approximates high-dimensional fusion by the product of low-rank matrices. Consider W as a matrix consisting of d h M-order tensors, each of which can be expressed as:

[0078]

[0079] Where R is the minimum rank that makes tensor decomposition effective. By fixing R = r, we can use To reconstruct low rank M specific low-rank modal factors can be reorganized, so Then the low-rank factor corresponding to mode m is Therefore the low-rank weight tensor can be reconstructed as The final output h is calculated by parallel decomposition with the input tensor Z, and the calculation formula is:

[0080]

[0081] in Represents the element-wise product of a sequence of tensors:

[0082] The fused representation is processed through a linear layer, integrating the high-dimensional features and converting them into the required 16-dimensional output. A softmax activation function is then used to convert the linear layer output into a probability distribution for each class, generating the final classification and recognition result. This design ensures that the model can accurately perform 16-dimensional classification and facilitates the interpretation and evaluation of the classification and recognition results.

[0083] Step 4: Results and Analysis

[0084] A classification analysis was conducted on 14,400 samples. To ensure experimental rigor, the dataset underwent rigorous random screening and partitioning, resulting in 10,080 training samples, 2,880 test samples, and 1,440 validation samples. Experiments demonstrated that the system was able to classify and identify 16 categories of experimental objects with varying hardness and surface characteristics with 93.47% accuracy.

[0085] In order to further verify the performance of the LMF network and eliminate problems such as overfitting, data partitioning bias, insufficient generalization ability and result variability that may occur during the training process, the present invention adopts a five-fold cross validation. First, the complete data set is randomly divided into five equal-sized subsets, each subset contains 2880 sets of data. In each training, four subsets are selected as training sets, and the remaining one subset is used as a validation set for model training and evaluation. This process is repeated five times, and a different subset is selected as the validation set each time to ensure that all data can participate in training and verification. The results of the five-fold cross validation are shown in Figure 2. Figure 6 As shown in Figure 2, the results show that the classification accuracy of the LMF model on the validation set reaches 92.27%, proving the effectiveness of the model and dataset.

[0086] In order to further explore the measurement accuracy of surface protrusions of this system, based on the above error analysis of different surface conditions, the present invention divides the height difference Δh into 3mm, 2mm, and 1mm for analysis. In the experiment, the surface protrusion height combinations with Δh of 1mm are (1mm, 2mm), (2mm, 3mm), and (3mm, 4mm), the combinations with Δh of 2mm are (1mm, 3mm), and (2mm, 4mm), and the combination with Δh of 3mm is (1mm, 4mm). The classification accuracy of different Δh is as follows Figure 7 As shown in the figure, the experimental results show that when Δh is 3mm and 2mm, the measurement accuracy is close to 100%, and when Δh is 1mm, the accuracy can reach 97%.

[0087] Through multiple experiments, it was verified that the FBG slip tactile sensing system of the present invention has excellent anti-interference ability and detection accuracy, can effectively distinguish the softness, hardness and surface characteristics of different materials, and is suitable for multimodal information perception in complex environments.

Claims

1. A sliding tactile sensing device, comprising a sliding bracket driven by a robotic arm, a sensing link, a spring, an FBG sliding tactile sensor, and an optical path system, wherein: The FBG tactile sensor consists of at least three bionic fingers, each of which contains an FBG sensor. The FBG sensor senses external stress and vibration signals by detecting the wavelength and phase changes of the Bragg grating in the optical fiber. The FBG sensor is encapsulated in a flexible structure: The flexible structure includes three gaskets, the FBG sensor is encapsulated between the upper gasket and the middle gasket, and the lower part of the middle gasket is provided with a recess; Lower gasket: directly contacts the test object, and its surface is textured to increase friction during sliding, amplify vibration caused by friction, and improve the sensitivity of vibration signals. The lower gasket is designed with protrusions that match the depressions of the middle gasket. The sliding bracket is used to drive the bionic finger to move along the test surface under the drive of the robotic arm. A groove is provided inside the sliding bracket to accommodate a sensor connecting rod and a spring. The sensor connecting rod includes two parts, a thin rod and a thick rod. A slider is provided at the upper side of the thick rod. The lower part of the thick rod extends out of the internal groove of the sliding bracket, and the upper end of each bionic finger is fixedly connected to the lower part of the thick rod. A slideway is provided on the side of the internal groove of the sliding bracket to match the connecting rod slider, and a hole is provided above the groove to allow the thin rod to pass through. The spring is sleeved on the outer circumference of the thin rod, and its two ends are respectively pressed against the inner surface of the internal groove of the sliding bracket and the upper surface of the thick rod.

2. The sliding tactile sensor device according to claim 1, characterized in that: The optical system includes a light source part, a sensor part and a data acquisition part; The light source uses a wavelength-tunable laser with a base filter, which achieves self-excited amplification through an erbium-doped fiber amplifier (EDFA), providing stable energy output for the optical system. The polarization state of the light is controlled by a first polarization controller, and the light beam then enters the first coupler, where the optical power is distributed. A portion enters the sensing part, assuming n bionic fingers are used, and is divided into n paths through a second coupler. The other portion returns to the light source, is filtered again by an FFP-TF filter, and returns to the EDFA through the second polarization controller to form a closed loop. An arbitrary waveform generator is used to generate a sawtooth wave signal to tune the filter. In the sensing part, the optical path system adopts a Michelson interferometer structure, and each path obtains coherent output light.

3. The sliding tactile sensor device according to claim 2, characterized in that: The filter is an FFP-TF filter.

4. The sliding tactile sensor device according to claim 1, characterized in that: In the sensing part, the reference arm is connected to a Faraday rotator mirror to reflect the incident light transmitted from the light source part back to the system; the sensing arm uses the FBG sensor of each bionic finger to reflect the light that matches its Bragg wavelength. For the FBG sensor, only the incident light corresponding to the Bragg grating wavelength can be reflected, and light of other wavelengths is transmitted; the reflected light from the reference arm and the sensing arm is coupled in a third coupler to form coherent output light.

5. The sliding tactile sensor device according to claim 1, characterized in that: In the data acquisition part, the coherent output light is converted into an electrical signal by a photodetector, which is then collected by a data acquisition card and transmitted to the host computer for collection and processing.

6. The sliding tactile sensor device according to claim 1, characterized in that: A limiting piece for preventing the connecting rod slider from moving out of the sliding bracket is provided at the lateral lower part of the inner groove of the sliding bracket.

7. A method for analyzing the hardness and flatness of an object using the tactile sensing device according to any one of claims 1 to 6, characterized in that: The following steps are involved: Acquisition and processing of sliding tactile signals: A robotic arm controls the sliding tactile sensing device to slide across the surface of the object to be measured, collecting sample data. During the sliding tactile signal processing, the stress signal collected by the FBG sensor is quantified through the wavelength shift of the fiber Bragg grating, and the vibration signal is extracted through the phase change of the coherent signal. The data is then collected by a data acquisition card and transmitted to the host computer for processing. Each FBG sensor 8 can simultaneously output multimodal data including stress mode and vibration mode. Multimodal tactile data fusion: Multimodal data contains n*2 input signals from various FBG sensors, namely the stress signals and vibration signals of each channel; the data is processed by the low-rank multimodal fusion network LMF to generate the final classification and recognition results.

8. The object hardness and flatness analysis method according to claim 7, characterized in that: The method of processing data through a low-rank multimodal fusion network LMF to generate the final classification and recognition results is as follows: the LMF network first extracts a single-modal representation of the data of each modality, and uses an LSTM network to independently process each input modality to extract a single-modal representation from it; the single-modal representation is expanded through the hidden state to convert the multimodal input into a high-dimensional tensor, and then mapped back to the low-dimensional output vector space; the LMF network uses low-rank tensor decomposition technology to decompose the weight tensor into multiple modality-specific vector groups, and generates a low-rank approximate multimodal fusion representation through the outer product of the vector groups; the fused representation is processed through a linear layer to integrate and transform the high-dimensional features into multi-dimensional output; the Softmax activation function is used to convert the output of the linear layer into a probability distribution of the category, thereby generating the final classification and recognition result; each dimension of output is a classification output.

Images