A multi-point haptic intelligent encoding and decoding device, system and method

Abstract

The application belongs to the technical field of tactile communication and intelligent human-computer interaction, and discloses a multi-point tactile intelligent encoding and decoding device, system and method. An acquisition end obtains original tactile signals, and extracts features through a neural network self-encoder. A weighted fusion module based on an attention mechanism dynamically groups and quantizes residual errors of the encoded features in combination with channel state information (CSI), so as to realize low-code-rate compression transmission. A perception end inversely quantizes received data and reconstructs signals, and optimizes output tactile signals in combination with a multi-scale perception calibration mechanism, so that the output tactile signals are closer to original inputs in the human perception dimension. A multi-point tactile feedback device obtains vibration feedback corresponding to real-time multi-channel vibration tactile signals based on the reconstructed signals. The application realizes efficient compression, low-delay transmission and high-quality reconstruction of tactile signals by fusing a deep learning architecture and a psychological tactile perception model, and is suitable for virtual reality, remote operation, medical robots and other high-precision tactile interaction scenes.

Description

Technical Field

[0001] This invention belongs to the field of tactile communication and intelligent human-computer interaction technology, and more specifically, relates to a multi-point tactile intelligent encoding and decoding device, system and method. Background Technology

[0002] Tactile communication technology, as a key method for expanding the dimensions of human-computer interaction, has wide applications in many fields such as remote control, medical robots, and metaverse interaction. However, existing tactile encoding and decoding methods still have many technical bottlenecks.

[0003] Current tactile encoding and decoding methods mainly rely on traditional signal processing techniques, such as wavelet transform, discrete cosine transform (DCT) coding, or processing architectures based on shallow neural networks. These methods typically perform well in analyzing low-frequency features, but they are difficult to comprehensively characterize the overall characteristics of tactile signals, leading to distortion of the recovered tactile signals. Furthermore, the encoding delay is relatively long, making it difficult to meet the low-latency requirements for tactile signal transmission.

[0004] Furthermore, existing technologies do not fully consider the nonlinear perception characteristics of human tactile receptors during the encoding and decoding process, such as frequency masking effects and absolute perception thresholds, and generally use mean squared error (MSE) as the main optimization metric. While this design approach can approximate the original signal in the time domain, it is difficult to guarantee the accurate reproduction of the decoded signal at the tactile perception level.

[0005] Existing tactile encoding and decoding systems mostly focus on end-to-end transmission of single-point tactile signals. Although some research has attempted to explore the feasibility of multi-point tactile encoding and decoding, due to its reliance on traditional serial encoding mechanisms, it is difficult to achieve synchronous transmission between multiple tactile channels. Related solutions remain in the theoretical research stage and have not yet formed a practical system implementation. Summary of the Invention

[0006] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides a multi-point tactile intelligent encoding and decoding device, system and method, the purpose of which is to improve the reconstruction quality of tactile signals and reduce the transmission delay of tactile signals.

[0007] To achieve the above objectives, the present invention provides a multi-point tactile intelligent codec device, comprising:

[0008] A neural network autoencoder is used to extract and compress features from multi-channel vibration tactile signals acquired in real time from a touch surface to generate coded features.

[0009] The channel awareness module is used to sense the current channel status information (CSI) in real time.

[0010] The weighted fusion module is used to perform channel attention prediction based on the current channel state information (CSI) and the coding features, obtain the importance weights of each dimension of the coding features, and fuse the importance weights with the coding features of the corresponding dimensions to obtain a feature vector; wherein, the importance weights are used to reflect the transmission priority of coding features of different dimensions in the current channel state;

[0011] The dynamic grouped residual quantizer is used to group the feature vectors after threshold suppression, perform multi-level residual quantization on each group of features, and concatenate the quantization results of each group to obtain discrete quantized data; where the number of groups M = L / N, N is the preset single-group feature dimension, and L is the dimension of the feature vectors after threshold suppression.

[0012] A dequantizer is used to dequantize received discrete quantized data to obtain the recovered encoded features;

[0013] A neural network self-decoder is used to decode the recovered encoded features to obtain a reconstructed signal corresponding to the multi-channel vibration tactile signal.

[0014] Furthermore, it also includes a multi-scale perception calibration module; the multi-scale perception calibration module is used to optimize the reconstructed signal so that the optimized reconstructed signal approximates the multi-channel vibration tactile signal in the human perception dimension, and the specific optimization methods include:

[0015] Power spectral density analysis is performed on the multi-channel vibration tactile signal to obtain the logarithmic domain power spectrum; and the logarithmic domain power spectrum is input into the psychological tactile model to obtain the frequency masking threshold of the multi-channel vibration tactile signal.

[0016] Generate a perception weight vector, the value of which is the logarithmic ratio of the logarithmic domain power spectrum to the frequency masking threshold;

[0017] Frequency domain transformation is performed on the multi-channel vibration tactile signal and the reconstructed signal in different frequency bands to extract the spectral components of the multi-channel vibration tactile signal and the reconstructed signal;

[0018] Calculate the error vector between the multi-channel vibration tactile signal and the reconstructed signal spectrum, and multiply the error vector by the perception weight vector to obtain the multi-scale weighted spectral loss function; optimize the reconstructed signal with the goal of minimizing the multi-scale weighted spectral loss function so that the optimized reconstructed signal approximates the multi-channel vibration tactile signal in the human perception dimension.

[0019] Furthermore, the multi-scale weighted spectral loss function L multi-scale for:

[0020]

[0021] Among them, w i Here, N represents the element value corresponding to the i-th frequency band in the perception weight vector, where N is the number of frequency bands. and , respectively, represent the energy distribution of the multi-channel vibration tactile signal and the reconstructed signal in the i-th frequency band; ||||2 is the L2 operator.

[0022] Furthermore, it also includes a communication module; the communication module is used to encapsulate the discrete quantized data using a low-latency protocol and transmit it to the dequantizer; wherein the data frame format used by the low-latency protocol includes: a frame header, an encoding index for representing the discrete quantized data, and a checksum;

[0023] The frame header includes grouping information for indicating the dynamic grouped residual quantizer, parameter information for executing multi-level residuals, and a timestamp, which are used to ensure the synchronization and dequantization of the discrete quantized data across multiple tactile channels.

[0024] The present invention also provides a training method for a multi-point haptic intelligent codec device as described in any one of the preceding claims, comprising:

[0025] Constructing L including temporal reconstruction loss recon Quantization error loss L quant and the hybrid loss function L, including the multi-scale weighted spectral loss. total L total =αL recon +βL quant +γL multi-scale Wherein, the temporal reconstruction loss L recon The quantization error loss L is the loss between the reconstructed signal and the multi-channel vibration tactile signal. quant The mean square error loss between the feature vector after threshold suppression and the discrete quantized data is α, β, and γ, which are preset weighting coefficients.

[0026] The network parameters of the neural network autoencoder, the dynamic grouped residual quantizer, and the neural network autodecoder are adjusted using the backpropagation algorithm to make the hybrid loss function L... total Minimum, to obtain a well-trained multi-point tactile intelligent codec device.

[0027] This invention also provides a multi-point haptic intelligent encoding and decoding method, comprising:

[0028] The multi-channel vibration tactile signal and current channel state information (CSI) of the touch surface acquired in real time are input into the trained multi-point tactile intelligent codec device to obtain the reconstructed signal corresponding to the multi-channel vibration tactile signal.

[0029] The multi-point tactile intelligent codec device is any one of the multi-point tactile intelligent codec devices described above.

[0030] The present invention also provides a multi-point tactile intelligent interaction system, comprising: a trained multi-point tactile intelligent codec device and a multi-point tactile feedback device; wherein, the multi-point tactile intelligent codec device is any of the multi-point tactile intelligent codec devices described above;

[0031] The multi-point tactile intelligent codec device is used to obtain a reconstructed signal corresponding to the multi-channel tactile vibration signal based on the real-time acquired multi-channel vibration tactile signal of the touch surface and the current channel state information (CSI).

[0032] The multi-point tactile feedback device is used to obtain vibration feedback equivalent to the multi-channel vibration tactile signal based on the reconstructed signal.

[0033] Furthermore, the multi-point haptic feedback device includes a control board and a multi-channel haptic actuator array;

[0034] The control board is used to generate a drive signal based on the reconstructed signal and drive the multi-channel tactile actuator array to perform tactile rendering, thereby obtaining vibration feedback equivalent to the multi-channel vibration tactile signal.

[0035] Furthermore, it also includes a multi-point tactile sensing device; the multi-point tactile sensing device is used to acquire multi-channel vibration tactile signals of the touch surface in real time.

[0036] Furthermore, the multi-point tactile acquisition device includes a multi-axis accelerometer array, which is distributed on the contact surface to acquire multi-channel vibration tactile signals of the contact surface in real time.

[0037] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0038] (1) The multi-point tactile intelligent encoding and decoding device of the present invention extracts nonlinear features of different dimensions that can comprehensively characterize the overall characteristics of multi-channel vibration tactile signals based on neural networks. Based on these nonlinear features and the current channel state information (CSI), a weighted fusion of attention mechanisms is performed to obtain importance weights that reflect the transmission priority of different coding feature dimensions under the current channel conditions. The importance weights are weighted and fused with the corresponding coding features, and the obtained coding features are input to the dynamic group residual quantizer. This allows the dynamic residual quantization to adjust the grouping strategy according to real-time channel changes and thus determine the transmission features (i.e., dynamically adjust the transmission strategy of each dimension in the coding features in combination with channel quality). It is applicable to different channels, especially wireless channels, ensuring that the features can be correctly received at the receiving end, guaranteeing the reliability of signal transmission, reducing signal transmission delay, and improving communication efficiency. The dynamic group residual quantization also completes efficient data compression and quantization, further reducing signal transmission delay. This invention integrates a deep learning architecture and dynamic grouped residuals to achieve end-to-end transmission of multi-channel vibration tactile signals, improving the reconstruction quality of tactile signals. The channel sensing module based on the attention mechanism, combined with the channel state information (CSI) of the acquisition end, performs dynamic grouped residual quantization on the coding features, achieving low bit rate compressed transmission and reducing the latency of tactile signal transmission.

[0039] (2) Further, the multi-scale perception calibration module performs spectral analysis and multi-scale perception weighting on the reconstructed signal based on the psychological tactile model, and constructs a multi-scale weighted spectral loss based on the perception weight. The reconstructed signal is optimized by the multi-scale weighted spectral loss, which improves the approximation effect of the reconstructed signal and the original tactile signal on multiple perception scales, so that the final generated tactile signal is similar to the original tactile signal in the human perception dimension, thus ensuring the true restoration of the reconstructed signal at the tactile perception level.

[0040] (3) Further, the training method of the multi-point tactile intelligent codec device of the present invention constructs a total loss that is a weighted sum of temporal reconstruction loss, quantization error loss and multi-scale weighted spectrum loss. The parameters of each module are optimized synchronously through the backpropagation algorithm to realize the joint training optimization of the neural network self-codec, dynamic group residual quantizer and multi-scale perception calibration module, and ensure the end-to-end neural network training and deployment.

[0041] (4) Furthermore, the multi-point tactile intelligent encoding and decoding method of the present invention inputs the real-time multi-channel vibration tactile signal and the current channel state information into the trained multi-point tactile intelligent encoding and decoding device, and can directly obtain the reconstructed signal, realizing the end-to-end transmission of multi-point tactile signals.

[0042] (5) Furthermore, the multi-point tactile intelligent interaction system of the present invention simultaneously realizes the acquisition, compression encoding, low-latency synchronous transmission between multiple tactile channels, high-quality reconstruction and vibration feedback of multi-point vibration signals, and is particularly suitable for high-precision tactile interaction scenarios such as virtual reality (VR / AR), remote operation, and medical robots.

[0043] Overall, this invention solves the problems of high compression distortion rate, insufficient sensory adaptability and difficulty in multi-point synchronization in existing tactile coding technologies, and achieves efficient compression, low-latency transmission and high-quality reconstruction of tactile signals. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of a multi-point tactile intelligent encoding and decoding device in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a multi-point tactile intelligent encoding and decoding system in an embodiment of the present invention;

[0046] Figure 3 This is a structural diagram of a multi-point tactile intelligent encoding and decoding system in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment of the invention provides a multi-point tactile intelligent encoding and decoding device, including: a neural network autoencoder, a channel sensing module, a weighted fusion module, a dynamic group residual quantizer, a communication module, an inverse quantizer, and a neural network autodecoder.

[0050] The acquired multi-channel vibration tactile signals from the touch surface are input into a neural network autoencoder for feature extraction and compression to generate coded features. The neural network autoencoder is composed of a deep neural network, including a feature extraction layer and an encoding layer, used to perform high-dimensional feature extraction and dimensionality reduction processing on the multi-channel vibration tactile signals, outputting low-dimensional coded features. In this embodiment of the invention, the feature extraction layer uses a convolutional neural network to extract the time-frequency features of the signal, capturing key information of the tactile signal in the time and frequency domains. The encoding layer compresses the high-dimensional features into low-dimensional coded features, reducing the amount of data while retaining key information; its dimension is preferably 128 to 512.

[0051] In this embodiment of the invention, the channel sensing module is deployed at the sensing end to sense the current channel state information (CSI) in real time and transmit the current channel state information back to the acquisition end through the feedback channel to support the dynamic quantization strategy adjustment of the data at the next moment.

[0052] A weighted fusion module based on an attention mechanism is used to predict channel attention based on the current channel state information (CSI) and the coding features output by the neural network autoencoder, obtaining importance weights for the coding feature dimensions. These importance weights reflect the transmission priority of different coding feature dimensions under the current channel conditions. The importance weights are then weighted and fused (multiplied) with the corresponding coding features to obtain a weighted feature vector. Redundant feature vectors are removed using a masking mechanism before being fed into a dynamic group residual quantizer for quantization, generating discrete quantized data. The masking mechanism suppresses feature vectors below a set threshold while retaining salient features. The dynamic group residual quantizer achieves efficient feature representation and compression by dividing the features into multiple independent subspaces and constructing a cascaded residual subcodebook.

[0053] In this embodiment of the invention, the dynamic grouped residual quantizer determines the number of groups based on the dimensionality information of the feature vector after threshold suppression, and performs multi-level residual quantization on each group of features to obtain the quantized features corresponding to each group of features. Finally, the quantization results of all subspaces (all groups of features) are concatenated into complete quantized features to obtain the aforementioned discrete quantized data. This quantization process combines channel state information to adjust the grouping strategy, thereby achieving dynamic resource allocation. The number of groups M = L / N, where N is the preset dimension of a single feature group, and L is the dimension of the feature vector after threshold suppression.

[0054] Discrete quantized data is sent to the sensing end (reconstruction terminal) via a communication module. The communication module encapsulates the discrete quantized data using a low-latency protocol and completes the transmission, which is then transmitted to the sensing end via a network medium. In this embodiment of the invention, the low-latency protocol adopts a custom data frame structure. Each data frame includes a frame header, an encoding index (used to characterize the discrete quantized data), and a checksum to ensure the integrity and real-time performance of data transmission. The frame header information is used to indicate quantizer parameters, including information indicating quantizer parameters (grouping information, the number and size of codebooks required for multi-level residual quantization), as well as key information such as timestamps, used for data synchronization and decapsulation (dequantization) between multiple tactile channels. The checksum is used to ensure the integrity of data transmission, ensuring that the receiving end can accurately reconstruct the original data. The low-latency communication protocol supports multiple channel transmission methods, including Wi-Fi, Bluetooth, 5G, or wired Ethernet connections, to meet the real-time requirements of different scenarios.

[0055] In this embodiment of the invention, at the sensing end, the received discrete quantized data is dequantized by a dequantizer to recover the encoded features; the dequantizer restores the original feature vector according to a preset mapping rule.

[0056] The dequantized encoded features are input into the neural network self-decoder to obtain the decoded signal.

[0057] As a further design of the present invention, the multi-point tactile intelligent encoding and decoding system also includes a multi-scale perception calibration module, which is used to optimize the decoded signal and generate a restored tactile signal that is similar to the original tactile signal in the human perception dimension; the multi-scale perception calibration module performs spectral analysis and perception weighting on the reconstructed signal (decoded signal) based on the psychological tactile model to improve the approximation effect of the output signal on multiple perception scales.

[0058] The multi-scale sensing calibration module specifically includes:

[0059] Power spectral density analysis was performed on the original tactile signal to obtain the logarithmic domain power spectrum.

[0060] Based on the logarithmic power spectrum of the original tactile signal, the frequency masking threshold of the original tactile signal is calculated; specifically, the logarithmic power spectrum of the original tactile signal is input into the psychological tactile model to obtain the frequency masking threshold of the original tactile signal.

[0061] Generate a perception weight vector, the value of which is the logarithmic ratio of the logarithmic power spectrum of the original tactile signal to the frequency masking threshold.

[0062] The reconstructed signal is subjected to frequency domain transformation in multiple frequency ranges to extract the spectral components of the reconstructed signal; among which, frequency domain transformation includes DCT transform, wavelet transform, Fourier transform, etc.

[0063] Constructing a multi-scale weighted spectral loss function includes: calculating the error vector between the spectrum of the original tactile signal and the spectrum of the reconstructed signal, which is composed of the error between the spectrum of the original tactile signal and the spectrum of the reconstructed signal for each frequency band; multiplying the error vector by the perception weight vector to obtain the multi-scale weighted spectral loss function; and optimizing the decoded signal by minimizing the loss function to make it approximate the original signal at multiple perception scales.

[0064] In this embodiment of the invention, the constructed multi-scale weighted spectral loss function L multi-scale for:

[0065]

[0066] Among them, w i Let N be the perceptual weight coefficient for the i-th frequency band, where N is the element value corresponding to the i-th frequency band in the perceptual weight vector, and N is the number of frequency bands. and , respectively, represent the energy distribution of the original and reconstructed signals in this frequency band; || ||2 is the L2 operator.

[0067] The sensing end monitors the current channel state information (CSI) in real time and transmits the CSI information back to the acquisition end through the feedback channel for dynamic quantization strategy adjustment of the data in the next moment.

[0068] In this embodiment of the invention, the neural network self-encoder, dynamic grouped residual quantizer, and multi-scale sensing calibration module are optimized through joint training; the constructed hybrid loss function L total The loss function is a weighted sum of temporal reconstruction loss, quantization error loss, and multi-scale weighted spectral loss. The parameters of each module are simultaneously optimized using a backpropagation algorithm to minimize the hybrid loss function, ensuring that the decoded signal closely approximates the original signal in the human perception dimension, thus improving the perceptual fidelity of the tactile signal. Specifically, the temporal reconstruction loss is the loss between the original tactile signal and the reconstructed signal output by the neural network self-decoder. The quantization error loss of the dynamic group residual quantizer is the mean square error between the threshold-suppressed feature vector (input to the dynamic group residual quantizer) and the discrete quantized data (quantization result output by the dynamic group residual quantizer). During this process, a pass-through estimator is used to truncate the gradients before and after quantization, thereby achieving joint optimization of the multi-layer concatenated codebook.

[0069] In this embodiment of the invention, the constructed hybrid loss function L total for:

[0070] L total =αL recon +βL quant +γL multi-scale

[0071] Among them, L recon For temporal reconstruction loss, L quant L is the quantization error loss of the dynamic grouped residual quantizer. multi-scale This is a multi-scale weighted spectral loss function, where α, β, and γ are preset weighting coefficients.

[0072] Example 2

[0073] This invention provides a multi-point haptic intelligent encoding and decoding method, including:

[0074] The real-time multi-channel vibration tactile signals and real-time channel state information are input into the trained multi-point tactile intelligent encoder and decoder to obtain the reconstructed signal.

[0075] Among them, the multi-point tactile intelligent encoding and decoding device is the multi-point tactile intelligent encoding and decoding device in Example 1.

[0076] The relevant technical solutions are the same as above, and will not be repeated here.

[0077] Example 3

[0078] like Figure 2 and Figure 3 As shown, this embodiment of the invention provides a multi-point tactile intelligent interaction system, including: a multi-point tactile acquisition device, a multi-point tactile intelligent encoding and decoding device, and a multi-point tactile feedback device.

[0079] At its acquisition end, a multi-channel vibration tactile signal (raw tactile signal) of the contact surface is acquired through a multi-point tactile acquisition device. The multi-point tactile acquisition device, deployed at the acquisition end, includes a multi-axis accelerometer array. The sensors are uniformly distributed on the contact surface (such as gloves, handles, etc.) to acquire multi-point vibration tactile signals of the contact surface and convert the raw analog signals into digital signals. The multi-axis accelerometer array uses a dimensionality reduction algorithm to fuse the multi-axis acceleration signals into a single-axis vibration signal, reducing the signal dimensionality while retaining key tactile features. In this embodiment, each multi-axis accelerometer in the multi-axis accelerometer array is responsible for acquiring vibration data at a specific point, ensuring the comprehensiveness and accuracy of the signal. The multi-axis accelerometer is preferably a three-axis or six-axis accelerometer with a sampling rate of not less than 1kHz, which can effectively capture high-frequency vibration information and ensure the dynamic response characteristics of the captured tactile signal. In one embodiment, the multi-point tactile acquisition device is deployed on gloves, handles, or other interactive devices to capture multi-point vibration tactile signals generated when the operator contacts the object in real time, realizing tactile perception of hand movements. A dimensionality reduction algorithm is used to fuse multi-axis acceleration signals and transform them into single-axis vibration signals. For example, the Discrete Fourier Transform (DFT) is used to extract signal components in the dominant frequency direction, reducing the signal dimensionality while preserving key tactile features.

[0080] The multi-point tactile intelligent codec device is used to acquire and reconstruct signals based on real-time multi-channel vibration tactile signals and real-time channel state information. The multi-point tactile intelligent codec device is the multi-point tactile intelligent codec system in the above embodiment 1.

[0081] A multi-point haptic feedback device is deployed at the sensing end, including a control board and a multi-channel haptic actuator array. The control board is used to obtain the drive signal of the actuator array based on the reconstructed signal, and drive the actuator array to perform haptic rendering to obtain vibration feedback that matches the real-time multi-channel vibration haptic signal in terms of frequency response, amplitude intensity, and time sequence. This vibration feedback is used to represent the haptic sensation at the acquisition end, thus achieving high-fidelity haptic reproduction and improving the user's haptic experience and operational accuracy. In this embodiment of the invention, the multi-channel haptic actuator array includes at least five independently controlled haptic feedback execution units, capable of generating multi-point synchronous vibration feedback based on the decoded signal. Each haptic feedback execution unit is responsible for generating vibration feedback at a specific point.

[0082] Specifically, at the acquisition end of the multi-point tactile intelligent codec system, the multi-point tactile acquisition device acquires multi-channel vibration tactile signals from the contact surface in real time through a multi-axis accelerometer array. The features of the vibration tactile signals are extracted and compressed by a neural network autoencoder combined with a channel sensing module, a weighted fusion module, and a dynamic grouped residual quantizer. After weighted fusion based on an attention mechanism and dynamic grouped residual quantization, discrete quantized data is obtained. During communication, the discrete quantized data is transmitted from the acquisition end to the sensing end via the communication module. After receiving the compressed data, the sensing end calls the decoder to perform inverse quantization and signal reconstruction, generating a reconstructed signal, thus forming an end-to-end multi-point tactile signal transmission link. The codec can be configured on a local terminal or edge computing node according to actual deployment requirements.

[0083] The vibration spectrum of the reconstructed signal output by the decoder is dynamically adjusted through a multi-scale perceptual calibration mechanism to improve the tactile reproduction effect. This mechanism uses frequency domain transforms of different lengths to perform multi-scale analysis on the spectrum of the reconstructed signal and introduces a psychological tactile model to adjust the perceptual weights of different frequency bands, so that the reconstructed signal approximates the original input (vibrational tactile signal of the contact surface) at multiple perceptual scales.

[0084] At the sensing end of the system, a multi-point tactile feedback device is also deployed. The multi-point tactile feedback device drives a multi-channel feedback actuator array based on the decoded signal to obtain vibration feedback corresponding to the real-time multi-channel vibration tactile signal. This vibration feedback is used to represent the tactile sensation at the acquisition end.

[0085] In practical deployments, this system can be applied to fields such as virtual reality gloves, remote medical robot operating platforms, and industrial remote control systems. For example, in VR interaction scenarios, users wear gloves with integrated multi-channel haptic feedback devices. When the gloves come into contact with virtual objects, the sensor array at the acquisition end captures tactile signals, compresses and encodes them, and then transmits them wirelessly to the sensing end. The feedback device drives the actuator based on the decoded signals, allowing the user to experience realistic tactile feedback.

[0086] Experimental results show that this system outperforms existing solutions in terms of tactile signal compression efficiency, transmission delay control, and sensory reproduction quality, especially in terms of multi-point tactile synchronization and sensory adaptability.

[0087] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-point haptic intelligent codec apparatus, characterized by, include: A neural network autoencoder is used to extract and compress features from multi-channel vibration tactile signals acquired in real time from a touch surface to generate coded features. The channel awareness module is used to sense the current channel status information (CSI) in real time. The weighted fusion module is used to perform channel attention prediction based on the current channel state information (CSI) and the coding features, obtain the importance weights of each dimension of the coding features, and fuse the importance weights with the coding features of the corresponding dimensions to obtain a feature vector; wherein, the importance weights are used to reflect the transmission priority of coding features of different dimensions in the current channel state; The dynamic grouped residual quantizer is used to group the feature vectors after threshold suppression, perform multi-level residual quantization on each group of features, and concatenate the quantization results of each group to obtain discrete quantized data; where the number of groups M=L / N, N is the preset single-group feature dimension, and L is the dimension of the feature vectors after threshold suppression. A dequantizer is used to dequantize received discrete quantized data to obtain the recovered encoded features; A neural network self-decoder is used to decode the recovered coded features to obtain a reconstructed signal corresponding to the multi-channel vibration tactile signal; It also includes a multi-scale perception calibration module; the multi-scale perception calibration module is used to optimize the reconstructed signal so that the optimized reconstructed signal approximates the multi-channel vibration tactile signal in the human perception dimension, and the specific optimization methods include: Power spectral density analysis is performed on the multi-channel vibration tactile signal to obtain the logarithmic domain power spectrum; and the logarithmic domain power spectrum is input into the psychological tactile model to obtain the frequency masking threshold of the multi-channel vibration tactile signal. Generate a perception weight vector, the value of which is the logarithmic ratio of the logarithmic domain power spectrum to the frequency masking threshold; Frequency domain transformation is performed on the multi-channel vibration tactile signal and the reconstructed signal in different frequency bands to extract the spectral components of the multi-channel vibration tactile signal and the reconstructed signal; Calculate the error vector between the multi-channel vibration tactile signal and the reconstructed signal spectrum, and multiply the error vector by the perception weight vector to obtain the multi-scale weighted spectral loss function; optimize the reconstructed signal with the goal of minimizing the multi-scale weighted spectral loss function so that the optimized reconstructed signal approximates the multi-channel vibration tactile signal in the human perception dimension.

2. The multi-point tactile intelligent encoding and decoding device according to claim 1, characterized in that, The multi-scale weighted spectral loss function is: in, The first in the perception weight vector i The element values ​​corresponding to each frequency band N For the number of frequency bands, and The multi-channel vibration tactile signal and the reconstructed signal are respectively in the first... i Energy distribution across frequency bands; It is a 2-norm operator.

3. The multi-point tactile intelligent encoding and decoding device according to claim 1 or 2, characterized in that, It also includes a communication module; The communication module is used to encapsulate the discrete quantized data using a low-latency protocol and transmit it to the dequantizer; wherein, the data frame format used by the low-latency protocol includes: a frame header, an encoding index for representing the discrete quantized data, and a checksum; The frame header includes grouping information for indicating the dynamic grouped residual quantizer, parameter information for executing multi-level residuals, and a timestamp, which are used to ensure the synchronization and dequantization of the discrete quantized data across multiple tactile channels.

4. A training method for a multi-point haptic intelligent codec device as described in any one of claims 1-3, characterized in that, include: Constructing including temporal reconstruction loss Quantization error loss and the hybrid loss function including the multi-scale weighted spectral loss. , Wherein, the temporal reconstruction loss The quantization error loss is the loss between the reconstructed signal and the multi-channel vibration tactile signal. The mean square error loss is the difference between the threshold-suppressed feature vector and the discrete quantized data. Preset weighting coefficients; The network parameters of the neural network autoencoder, the dynamic grouped residual quantizer, and the neural network autodecoder are adjusted using the backpropagation algorithm to optimize the hybrid loss function. Minimum, to obtain a well-trained multi-point tactile intelligent codec device.

5. A multi-point tactile intelligent encoding and decoding method, characterized in that, include: The multi-channel vibration tactile signal and current channel state information (CSI) of the touch surface acquired in real time are input into the trained multi-point tactile intelligent codec device to obtain the reconstructed signal corresponding to the multi-channel vibration tactile signal. The multi-point tactile intelligent codec device is the multi-point tactile intelligent codec device according to any one of claims 1-3.

6. A multi-point tactile intelligent interaction system, characterized in that, include: A trained multi-point tactile intelligent codec and a multi-point tactile feedback device; wherein the multi-point tactile intelligent codec is the multi-point tactile intelligent codec as described in any one of claims 1-3; The multi-point tactile intelligent codec device is used to obtain a reconstructed signal corresponding to the multi-channel tactile vibration signal based on the real-time acquired multi-channel vibration tactile signal of the touch surface and the current channel state information (CSI). The multi-point tactile feedback device is used to obtain vibration feedback equivalent to the multi-channel vibration tactile signal based on the reconstructed signal.

7. The multi-point tactile intelligent interaction system according to claim 6, characterized in that, The multi-point haptic feedback device includes a control board and a multi-channel haptic actuator array; The control board is used to generate a drive signal based on the reconstructed signal and drive the multi-channel tactile actuator array to perform tactile rendering, thereby obtaining vibration feedback equivalent to the multi-channel vibration tactile signal.

8. The multi-point tactile intelligent interaction system according to claim 6 or 7, characterized in that, It also includes a multi-point tactile acquisition device; the multi-point tactile acquisition device is used to acquire multi-channel vibration tactile signals of the touch surface in real time.

9. The multi-point tactile intelligent interaction system according to claim 8, characterized in that, The multi-point tactile acquisition device includes a multi-axis accelerometer array, which is distributed on the contact surface to acquire multi-channel vibration tactile signals of the contact surface in real time.

Images