An AI chip-based feedback parameter acquisition and processing method and a liquid silicone rubber device

Abstract

The present application relates to the field of robot interaction, and more particularly to a feedback parameter acquisition and processing method based on an AI chip and a liquid silicone device. The method comprises the following steps: collecting multiple sensing parameters based on the liquid silicone device to generate a multi-modal data stream; performing adaptive coding processing on the multi-modal data stream to output a multi-modal perception coding sequence; calling a lightweight behavior recognition model built-in the AI chip to analyze the multi-modal perception coding sequence in terms of semantic features of each mode to generate an interaction intent state; performing interaction demand analysis based on the interaction intent state to output a behavior strategy item and an interaction voice package. The present application improves the interaction capability and personification experience of the liquid silicone device by performing real-time behavior understanding and intent judgment on the collected user parameters, and optimizes the user experience.

Description

Technical Field

[0001] This invention relates to the field of robot interaction, and in particular to a method for acquiring and processing feedback parameters based on an AI chip and a liquid silicone device. Background Technology

[0002] With the rapid development of artificial intelligence, flexible electronic materials, and multimodal sensing technologies, human-computer interaction systems are gradually evolving from traditional single-input / output modes towards multimodal intelligent interaction that integrates visual, voice, tactile, and motion information. Existing interactive robots mostly employ rigid structures and single or low-dimensional sensing solutions, such as relying solely on cameras for visual recognition or microphone arrays for voice recognition. This makes it difficult to comprehensively acquire integrated behavioral information from users during actual interactions, leading to partial and semantically incomplete understanding of the interaction. While flexible materials such as liquid silicone possess excellent tactile and deformation capabilities in the field of biomimetic robots, in practical applications, the complex distribution of internal sensing nodes and the susceptibility of signals to deformation interference present technical challenges for multi-source data acquisition and stable transmission. This makes it difficult to achieve real-time analysis of user interaction data and recognition of behavioral intentions. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a feedback parameter acquisition and processing method based on an AI chip and a liquid silicone device, thereby resolving at least one of the aforementioned technical problems.

[0004] To achieve the above objectives, the present invention provides a feedback parameter acquisition and processing method based on an AI chip, comprising the following steps: Step S1: Acquire multi-sensor parameters based on the liquid silicone device to generate a multimodal data stream; Step S2: Perform adaptive encoding processing on the multimodal data stream and output a multimodal sensing encoded sequence; Step S3: Call the lightweight behavior recognition model built into the AI ​​chip to perform modal semantic feature parsing on the multimodal perception encoding sequence to generate interactive intent state; Step S4: Analyze the interaction requirements based on the interaction intent state, and output the behavior strategy items and interaction voice packs; Step S5: Adjust the logical order of the behavior strategy items and interactive voice packages and compile the instructions to generate control instructions; Step S6: Based on the control commands, drive the instruction execution unit of the liquid silicone device to complete the synchronous driving operation of interactive behavior and interactive voice.

[0005] In this invention, step S1 specifically involves the following steps: Multi-sensor parameter acquisition based on liquid silicone device; extraction of user interaction dataset. The user interaction dataset includes touch feedback parameters, user voice signals, and visual images; Calculate the sampling delay difference and time drift between different sensing nodes; The user interaction dataset is subjected to multi-source synchronous calibration based on the sampling delay difference and time drift to generate a multimodal data stream.

[0006] In this invention, step S2 specifically involves the following steps: The multimodal data stream includes touch feedback parameters, user voice signals, and visual images; Deformation parameters of flexible tactile signals are identified based on touch feedback parameters; Calculate the motor vibration interference value of the touch feedback parameters; Dynamic filtering is performed based on the deformation parameters and the motor vibration interference value to generate filtered tactile parameters; The pressure distribution of the filtered tactile parameters is analyzed to extract the contact intensity, area of ​​action and rate of change, and to construct tactile distribution features. Semantic parsing and emotion analysis of user voice signals are performed to generate interactive voice features; Target detection and key point extraction are performed on visual images to obtain key points of user interaction; Visual behavior analysis is performed on key user interaction points to generate interaction behavior features; Adaptive encoding is performed on tactile distribution features, interactive speech features, and interactive behavior features to output a multimodal perception encoding sequence.

[0007] In this invention, step S3 specifically involves the following steps: The lightweight behavior recognition model built into the AI ​​chip is invoked to perform modal semantic feature parsing on the multimodal perception coding sequence, generating multimodal semantic features; Cross-modal correlation analysis is performed on multimodal sensing encoded sequences to identify interactive correlation features between different modalities; Based on the aforementioned interaction association features, user interaction intent is parsed from the multimodal semantic features to generate user interaction intent; Perform time-series analysis on user interaction intent to obtain the trend of intent change; Calculate the confidence score for each intent category based on the user's interaction intent to obtain the confidence score for multiple intents; The trend of intent change is comprehensively evaluated based on the confidence level to generate the interaction intent state.

[0008] In this invention, step S4 specifically involves the following steps: Based on the interaction intent state, perform interaction requirement analysis to generate interaction action requirements; The interaction action requirements are input into a preset behavior strategy library for retrieval and matching, and behavior strategy entries are output. Deep understanding of dialogue semantics based on interactive voice features, generating structured interactive text and emotion tags; Based on emotion tags, the interaction parameters are adaptively matched, and the interaction voice style and speech rate rhythm parameters are output. Structured interactive text is encapsulated based on interactive speech style and speech rate rhythm parameters to generate interactive speech packages.

[0009] In this invention, step S5 specifically involves the following steps: The behavioral strategy items and interactive voice packages are broken down step by step to generate behavioral sequences and multiple voice sequences. Synchronous mapping is performed on behavioral sequences and multiple speech sequences, and the logical order is analyzed to obtain the logical sequence. Based on logical order, rhythm conflict and semantic inversion are identified, and logical rationality is adjusted to generate a set of behavioral actions with speech segments; The set of actions is compiled into instructions to generate control instructions.

[0010] In this invention, step S6 specifically involves the following steps: The instruction execution unit, which is driven by control commands to drive the liquid silicone device, completes the synchronous driving of interactive behavior and interactive voice, and collects real-time behavioral feedback data from the user during the execution process. The effectiveness of response strategies is evaluated based on real-time behavioral feedback data to obtain an effectiveness score. The lightweight behavior recognition model is fine-tuned locally based on the validity score to perform iterative optimization of the lightweight behavior recognition model.

[0011] This specification provides a liquid silicone device for performing the AI ​​chip-based feedback parameter acquisition and processing method described above, including: The acquisition module is used to acquire multiple sensor parameters based on a liquid silicone device and generate a multimodal data stream; The encoding module is used to perform adaptive encoding processing on multimodal data streams and output multimodal sensing encoded sequences; The semantic parsing module is used to call the lightweight behavior recognition model built into the AI ​​chip to parse the semantic features of the multimodal perception encoding sequence one modality at a time and generate the interactive intent state. The requirements analysis module is used to perform interaction requirements analysis based on the interaction intent state and output behavior strategy items and interaction voice packs. The instruction compilation module is used to logically adjust the order of behavior policy entries and interactive voice packs and compile instructions to generate control instructions. The drive control module is used to drive the instruction execution unit of the liquid silicone device based on control commands, and to complete the synchronous driving operation of interactive behavior and interactive voice.

[0012] The liquid silicone device encapsulates multiple sensing nodes on its surface and in key internal interaction areas; the key internal interaction areas include a head, arms, and torso; the arms and torso encapsulate flexible tactile pressure sensors, temperature sensors, and inertial measurement units; the head encapsulates an AI chip, a visual acquisition unit, and a voice acquisition unit.

[0013] The beneficial effects of this invention are as follows: By distributing multiple sensing nodes, including tactile, visual, speech, and IMU sensors, inside and outside the liquid silicone device, synchronous acquisition of multi-source heterogeneous interactive information is achieved, forming a multimodal data stream with a unified time stamp. This method effectively improves the integrity and spatial coverage of interactive information, avoids information deviation or loss caused by a single sensing modality, and enhances the reliability of subsequent semantic understanding and the comprehensiveness of environmental perception from the source. By adaptively encoding the original multimodal data stream, signals with different sampling frequencies and data structures are uniformly mapped into standardized perceptual encoding sequences, effectively reducing the fusion difficulty caused by data redundancy and modal differences, while improving data compression efficiency and transmission stability, providing a structurally consistent data input foundation for the real-time processing of AI chips. The lightweight behavior recognition model built into the AI ​​chip is used to perform modal semantic parsing of the encoded sequence, realizing independent semantic extraction and correlation modeling of tactile, visual, and speech information, thereby improving the accuracy and response speed of interactive intent recognition, enabling the system to complete real-time behavior understanding and preliminary intent judgment under low computing power conditions.

[0014] By jointly generating behavioral strategies and voice content based on interactive intent states, the system can transform abstract intents into executable interactive strategy items and structured voice packages. This enhances the consistency and contextual adaptability of human-computer interaction, improving the naturalness and emotional matching of interactive output. Through logical reordering and instruction compilation of behavioral strategy items and voice packages, unified scheduling and conflict resolution of multi-task control logic are achieved, effectively avoiding problems such as chaotic execution order or asynchrony between voice and action, thus improving the determinism and consistency of control commands. Based on unified control commands driving the liquid silicone device execution unit, synchronous output of action control and voice feedback is achieved, enabling the robot to maintain a highly consistent multimodal response over time. This significantly enhances the immersive experience and human-like performance of the interaction, improving the overall user experience. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the steps of a feedback parameter acquisition and processing method based on an AI chip according to the present invention. Figure 2This is a detailed flowchart illustrating the implementation steps of step S1. Figure 3 A schematic diagram of a user interaction dataset; Figure 4 This is a schematic diagram of the multimodal data stream after multi-source synchronous calibration. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] This application provides a feedback parameter acquisition and processing method based on an AI chip and a liquid silicone device. The execution entities of the AI ​​chip-based feedback parameter acquisition and processing method and liquid silicone device include, but are not limited to, mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc., which can be considered as general computing nodes in this application. The data processing platform includes, but is not limited to, at least one of an audio-visual management system, an information management system, and a cloud data management system.

[0018] Please see Figures 1 to 4 This invention provides a feedback parameter acquisition and processing method based on an AI chip, comprising the following steps: Step S1: Acquire multi-sensor parameters based on the liquid silicone device to generate a multimodal data stream; Step S2: Perform adaptive encoding processing on the multimodal data stream and output a multimodal sensing encoded sequence; Step S3: Call the lightweight behavior recognition model built into the AI ​​chip to perform modal semantic feature parsing on the multimodal perception encoding sequence to generate interactive intent state; Step S4: Analyze the interaction requirements based on the interaction intent state, and output the behavior strategy items and interaction voice packs; Step S5: Adjust the logical order of the behavior strategy items and interactive voice packages and compile the instructions to generate control instructions; Step S6: Based on the control commands, drive the instruction execution unit of the liquid silicone device to complete the synchronous driving operation of interactive behavior and interactive voice.

[0019] In one specific embodiment, multi-sensor parameters are acquired from the liquid silicone device, and a multimodal data stream is generated. Let the tactile pressure be... Touch duration The audio sampling rate is 16kHz, the visual frame rate is 30fps, and the IMU sampling rate is 200Hz. The initial time delay of each sensor node is as follows: tactile... ,voice Visual The system uses reference time Perform synchronous calibration and introduce drift rate. Sampling time Then the drift compensation amount is:

[0020] Therefore, the calibrated delays are 13ms, 21ms, and 29ms, respectively, and the time error is controlled within ±2ms, thus forming a unified multimodal data stream.

[0021] In the multimodal coding stage, deformation calculation is performed on the tactile signal using a linear material mapping model.

[0022] in Then the deformation is

[0023] In the deformation sensing process of a liquid silicone tactile sensing device, the silicone medium itself exhibits significant viscoelastic characteristics, displaying a quasi-linear elastic response (δ=αP) under external pressure, where α characterizes the equivalent flexibility coefficient of liquid silicone under specific curing ratios and thicknesses. Because this type of flexible material is subject to the coupling effect of structural high-frequency micro-vibrations when driving a motor or actuator, this vibration is not a single-frequency noise but rather a broadband random disturbance superimposed on the low-frequency quasi-static tactile deformation signal. This results in the sensor output macroscopically manifesting as a superposition effect of energy attenuation and measurement drift of the effective deformation amplitude. Therefore, in this technical implementation, the motor vibration interference factor Ev is normalized to a vibration influence coefficient between 0 and 1 to characterize the degree to which vibration weakens the signal-to-noise ratio of the effective deformation signal, essentially corresponding to the loss of fidelity in the effective transmission of the tactile signal in the flexible medium. Based on this, an amplitude fidelity correction model F=δ(1-Ev) is introduced to perform real-time engineering compensation for the liquid silicone tactile signal without introducing complex frequency domain operations. Its function is equivalent to performing a first-order approximate correction on the attenuation of effective deformation observation caused by high-frequency vibration noise, thereby realizing dynamic filtering and fast estimation of tactile deformation under low computational cost conditions.

[0024] Therefore, the motor vibration interference value is introduced. Perform dynamic filtering:

[0025] Further calculation of the tactile distribution characteristics shows that the pressure change rate is...

[0026] The speech signal, after semantic parsing, yielded a request for help with an emotion probability of 0.72. The visual keypoint velocity was 18 px / s, and the proximity velocity was 0.4 m / s. Multimodal fusion inference was performed based on MobileNetV3, and its cross-modal correlation was scored for cross-modal consistency.

[0027] The final intent probability output is a request for assistance interaction. And match the behavior strategy library to get the matching degree. It also generates a speech rate parameter of 4.8 syll / s and a gentle emotional speech pack.

[0028] During the control execution phase, the action sequence and the speech sequence are time-aligned. Assuming a speech lead time of 120ms, the synchronization error satisfies...

[0029] The consistency constraint is satisfied. The control instructions are defined as follows:

[0030] And drive the execution units to complete synchronous interaction. During the feedback phase, computational behavior consistency is ensured. User response Voice matching A weighted model is adopted:

[0031]

[0032] because This triggers the model update mechanism, setting the learning rate to... The parameter update ratio is 1% to 5%, and local fine-tuning is performed on MobileNetV3 to achieve optimization and continuous behavior recognition enhancement.

[0033] In this embodiment, see Figure 2 The diagram below illustrates the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Multi-sensor parameter acquisition based on liquid silicone device; extraction of user interaction dataset. The user interaction dataset includes touch feedback parameters, user voice signals, and visual images; Calculate the sampling delay difference and time drift between different sensing nodes; The user interaction dataset is subjected to multi-source synchronous calibration based on the sampling delay difference and time drift to generate a multimodal data stream.

[0034] In this embodiment, various sensing nodes encapsulated in the surface and key interactive areas of the liquid silicone device are uniformly initialized and their parameters are set. Flexible tactile pressure sensors deployed in the arm and torso areas employ capacitive or piezoresistive structures, with sampling frequencies typically set between 100Hz and 500Hz to ensure high sensitivity to different tactile behaviors such as light touch, pressing, and sliding. Temperature sensors utilize high-precision digital devices with sampling frequencies set between 10Hz and 50Hz, achieving measurement accuracy within ±0.1℃. The inertial measurement unit includes a three-axis accelerometer and a three-axis gyroscope, with sampling frequencies set between 200Hz and 1000Hz to capture dynamic posture changes during user interaction. In the head area, the visual acquisition unit is configured with a 720p or 1080p resolution camera with a frame rate set between 25fps and 60fps. The voice acquisition unit uses a microphone array with a sampling rate set between 16kHz and 48kHz to ensure the integrity of the voice signal. During initialization, a unified time reference is allocated to each sensor node via a unified clock or a high-precision RTC module, and node registration is completed via I2C, SPI, or CAN bus, ensuring that the initial time synchronization error is controlled within ±1ms. Simultaneously, various sensors are calibrated; for example, zero-point and sensitivity calibration is performed on pressure sensors with an error of less than ±2%, and static offset correction is performed on the IMU with a drift of less than 0.05° per second, thereby ensuring the accuracy and consistency of the acquired data.

[0035] After initialization, the system enters the real-time data acquisition phase, continuously acquiring and structuring multimodal data generated during user interaction with the liquid silicone device. Tactile data is output from a flexible pressure sensor, including pressure values ​​in kPa, contact change rate ΔP divided by Δt, and contact distribution information, used to describe user touch behavior characteristics. Temperature data reflects thermal changes in the contact area in degrees Celsius. The inertial measurement unit outputs acceleration in meters per second squared and angular velocity in degrees per second, used to identify the type of interaction action. The voice acquisition unit samples the sound signal and performs noise reduction processing (e.g., spectral subtraction), endpoint detection (VAD), and feature extraction (e.g., 13-dimensional MFCC features) through a preprocessing module. The visual acquisition unit acquires continuous image frames and performs face detection, region cropping, and feature extraction (e.g., 128-dimensional or 512-dimensional vector representation). All data is accompanied by high-precision timestamps, with accuracy down to milliseconds or microseconds, and is locally cached by an AI chip, with a cache length typically set to 1 to 3 seconds. To improve data validity, a threshold filtering mechanism is implemented, for example, only recording data with pressure changes exceeding 5% or temperature changes exceeding 0.2 degrees Celsius. All data types are encapsulated into standard data frames according to a unified structure, including timestamps, data types, and data content, and written to a buffer in chronological order to form a complete user interaction dataset.

[0036] Due to differences in hardware response, communication links, and sampling mechanisms among different sensing nodes, sampling delays and time drifts occur between data sources. Therefore, it is necessary to model and calculate these time errors. In specific implementation, a high-precision clock inside the AI ​​chip is used as the global reference time source, with an accuracy better than ±10ppm, and a synchronization signal is periodically sent, with a period that can be set to 1 second. The timestamps of the synchronization signals received by each node are recorded, and the deviation Δt between them and the reference time is calculated. i This is expressed as the sensor's local time minus the reference time. Through continuous sampling, such as more than 10 times, the average delay and standard deviation are calculated to control the error within ±2ms. To estimate the time drift, analysis is performed over a longer time window, such as 10 to 30 seconds, tracking the trend of time deviation changes. A linear regression method is used to fit the deviation change curve to obtain the drift rate parameter, measured in milliseconds per second. For example, if the deviation of a node increases from 2 milliseconds to 8 milliseconds within 10 seconds, the drift rate is approximately 0.6 milliseconds per second. For higher accuracy requirements, Kalman filtering can be introduced to dynamically estimate the time deviation, thereby suppressing the influence of noise. Furthermore, in speech and visual data, synchronous events, such as sudden sounds and corresponding image changes, can be used for auxiliary alignment verification, further improving the accuracy of time error estimation.

[0037] Fixed delay compensation is applied to the data timestamps of each sensor node to correct the original timestamps and eliminate latency errors. Dynamic compensation is then performed based on the drift rate, using a linear correction model to correct the time, thereby controlling the overall time error within ±1ms. After time alignment, due to inconsistent sampling frequencies among different sensors, unified resampling is required. A unified time reference, such as 100Hz, is typically chosen. Low-frequency data, such as temperature, is padded using linear or spline interpolation methods, while high-frequency data, such as inertial measurement unit data, is downsampled to achieve time dimension consistency. A sliding window mechanism is used for data fusion, with a window length of 200 milliseconds and a step size of 50 milliseconds. The aligned tactile data, speech features, and visual features are uniformly encapsulated to form a multimodal data stream structure, including a unified timestamp, tactile feature vectors, speech features, and visual features. The tactile features are approximately 10 to 20 dimensions, the speech features are approximately 13 dimensions, and the visual features are approximately 128 dimensions. This multimodal data stream can be directly input into the model in the AI ​​chip for analysis and decision-making, enabling high-precision feedback control and intelligent response of the liquid silicone interactive robot.

[0038] In this embodiment, step S2 specifically involves the following steps: The multimodal data stream includes touch feedback parameters, user voice signals, and visual images; Deformation parameters of flexible tactile signals are identified based on touch feedback parameters; Calculate the motor vibration interference value of the touch feedback parameters; Dynamic filtering is performed based on the deformation parameters and the motor vibration interference value to generate filtered tactile parameters; The pressure distribution of the filtered tactile parameters is analyzed to extract the contact intensity, area of ​​action and rate of change, and to construct tactile distribution features. Semantic parsing and emotion analysis of user voice signals are performed to generate interactive voice features; Target detection and key point extraction are performed on visual images to obtain key points of user interaction; Visual behavior analysis is performed on key user interaction points to generate interaction behavior features; Adaptive encoding is performed on tactile distribution features, interactive speech features, and interactive behavior features to output a multimodal perception encoding sequence.

[0039] In this embodiment, data from tactile, voice, and visual channels are uniformly accessed and organized to form a multimodal data stream. Touch feedback parameters primarily originate from flexible tactile pressure and temperature sensors. The data includes pressure values ​​in kPa, estimated contact area, and the pressure change rate ΔP divided by Δt, along with temperature change information to aid in determining the contact state. Voice signals are acquired by a microphone array with a sampling rate of 16kHz or 48kHz, and undergo noise reduction and signal enhancement through a preprocessing module to ensure a signal-to-noise ratio higher than 20dB. Visual images are captured in real-time by a camera with a resolution of 720p or 1080p and a frame rate controlled between 25fps and 60fps to ensure continuous dynamic behavior capture. All data is accompanied by a unified timestamp with millisecond-level time accuracy and is written to a buffer queue in chronological order. The multimodal data is uniformly managed through an internal caching mechanism within the AI ​​chip, with a cache length of 2 to 5 seconds for subsequent synchronous processing and feature extraction. Simultaneously, a data integrity detection mechanism is adopted to mark or remove abnormal data. For example, when the amplitude of the voice signal is lower than a set threshold or the image frame loss rate exceeds 5%, a compensation strategy is triggered to ensure the stability and continuity of the multimodal data stream.

[0040] The pressure values ​​of each sensing node are mapped to a two-dimensional or three-dimensional distributed grid, with a grid resolution of 5 to 10 millimeters. A continuous pressure field is then generated using interpolation methods. Next, based on a nonlinear relationship model between pressure and deformation, such as by obtaining a mapping function through experimental calibration, the pressure values ​​are converted into deformation values, expressed in millimeter-level displacement. Furthermore, by calculating the gradient and curvature changes of the deformation values, local deformation features, such as maximum deformation, average deformation, and deformation distribution uniformity indices, are extracted. For dynamic touch processes, the deformation change rate, measured in millimeters per second, can be calculated through time series analysis to identify different touch behavior types, such as slow pressing or rapid tapping. This results in a set of deformation parameters containing the magnitude, distribution characteristics, and trends of the deformation values, providing a foundation for subsequent filtering and feature extraction.

[0041] The system acquires motor operating parameters, such as rotational speed (RPM), drive frequency, and load variations, and analyzes these parameters in conjunction with the acceleration signal output from the inertial measurement unit. Frequency domain analysis methods, such as Fast Fourier Transform (FFT), are used to convert the tactile signal to the frequency domain, identifying characteristic frequency bands corresponding to the motor's operating frequency, such as periodic vibration components within the 50Hz to 200Hz range. Then, bandpass filtering or spectral separation methods are used to extract the energy of this frequency band, and its amplitude is calculated as an indicator of vibration interference intensity. Furthermore, correlation analysis of multi-point sensor data can distinguish between real touch signals and vibration noise; for example, vibration signals typically exhibit synchronous changes at multiple nodes, while real touch has localized characteristics.

[0042] The filtering parameters are dynamically adjusted based on the intensity of vibration interference. When the detected vibration interference amplitude is high, the filter's cutoff frequency suppression capability is increased, and band-stop filtering or adaptive Kalman filtering methods are used to suppress specific frequency bands. When the interference is low, high signal fidelity is maintained to avoid over-filtering. During the filtering process, deformation parameters are used as a reference signal input to the filter to enhance the weight of the real touch signal, thereby achieving signal separation. The filtering window length can be set from 50 milliseconds to 100 milliseconds to balance real-time performance and stability. After filtering, the output filtered tactile parameters have a signal-to-noise ratio that can be increased to more than twice that of the original signal, while maintaining a deformation characteristic error of less than 5%. This step ensures the reliability of the tactile data.

[0043] The filtered pressure data is remapped into a two-dimensional distribution matrix, and contact areas are identified using a region segmentation algorithm. For example, a pressure threshold of 10% of the maximum pressure is set as the effective contact criterion. The area of ​​the contact area is then calculated in square centimeters, and contact intensity indices, such as average and maximum pressure, are calculated based on the pressure value distribution. Furthermore, the pressure change rate, in kPa per second, is calculated through time series analysis to describe the dynamic characteristics of the touch. Simultaneously, the center position of the contact area can be determined using a centroid calculation method to describe the area of ​​effect. For complex interactions, multi-point contact features can be extracted, such as simultaneous contact in multiple areas or sliding trajectories. The parameters, including contact intensity, area of ​​effect location, and change rate, are combined to form a tactile distribution feature vector, with dimensions ranging from 10 to 20, to characterize the spatial and dynamic characteristics of the user's touch behavior.

[0044] The collected user voice signals are first processed through speech recognition and semantic parsing. The speech recognition model converts the voice signal into text information, and natural language processing methods are used for semantic understanding, such as keyword extraction and intent recognition. Subsequently, sentiment analysis is performed on the voice signal by extracting voice feature parameters, such as fundamental frequency variation, energy distribution, and speech rate, to construct an sentiment recognition model. The fundamental frequency range is typically between 80Hz and 300Hz, and energy variation reflects the intensity of emotion. A classification model categorizes emotions into multiple classes, such as calm, excitement, or dissatisfaction, and assigns a probability value to each class. Semantic information and sentiment features are then fused to form an interactive voice feature vector, with dimensions ranging from 20 to 50, used to describe the user's intent and emotional state.

[0045] In the visual processing stage, target detection and keypoint extraction are performed on the acquired image data. First, a target detection model is used to identify user body regions, such as the head, hands, and torso, and output bounding box information. Then, a keypoint detection algorithm is used to extract the positions of human body keypoints, such as joint positions or facial keypoints; the number of keypoints can be set to 17 or more. Each keypoint is represented in two-dimensional coordinates and accompanied by a confidence score. To improve stability, the keypoint sequence can be temporally smoothed. By analyzing the spatial relationship and temporal changes of keypoints, the user's action type is identified, such as reaching, touching, or waving. Specifically, the distance and angle changes between keypoints are calculated, such as the rate of change of distance between the hand and torso, in pixels per second, while combining this with a time series model to identify action patterns. Further, motion path features, such as direction and speed of movement, are extracted through trajectory analysis. Different modal features are normalized to unify their numerical range, for example, mapped to the interval between 0 and 1. Then, weights are assigned according to the importance of each modality; for example, tactile features are weighted at 0.4, speech features at 0.3, and behavioral features at 0.3. Multimodal features are fused into a unified vector representation through feature concatenation and dimensionality reduction methods. Furthermore, a sequence encoding method is employed to organize the feature vectors within a continuous time window into a time series, with a sequence length that can be set to 10 to 20 frames. The output multimodal perception encoded sequence can be directly input into the model in the AI ​​chip for learning and decision-making, enabling intelligent perception and feedback control of the liquid silicone interactive robot.

[0046] In this embodiment, step S3 specifically involves the following steps: The lightweight behavior recognition model built into the AI ​​chip is invoked to perform modal semantic feature parsing on the multimodal perception coding sequence, generating multimodal semantic features; Cross-modal correlation analysis is performed on multimodal sensing encoded sequences to identify interactive correlation features between different modalities; Based on the aforementioned interaction association features, user interaction intent is parsed from the multimodal semantic features to generate user interaction intent; Perform time-series analysis on user interaction intent to obtain the trend of intent change; Calculate the confidence score for each intent category based on the user's interaction intent to obtain the confidence score for multiple intents; The trend of intent change is comprehensively evaluated based on the confidence level to generate the interaction intent state.

[0047] In this embodiment, a lightweight behavior recognition model deployed within the AI ​​chip is invoked to perform modal-by-modal semantic feature parsing on the input multimodal perception encoded sequence. This model typically employs a compressed deep neural network structure, such as a lightweight architecture based on the fusion of convolutional and temporal networks, with model parameters controlled between 1MB and 10MB to meet the requirements of embedded real-time processing. For the tactile modality, the model focuses on parsing the contact intensity, area of ​​action, and rate of change in the tactile distribution features, extracting tactile semantic labels through feature mapping, such as light touch, sustained pressure, or swiping, and outputting corresponding semantic vectors with dimensions ranging from 16 to 32. For the speech modality, the model performs semantic understanding and emotion recognition based on speech feature vectors, outputting semantic codes including user intent keywords and emotional states, typically with dimensions ranging from 32 to 64. For the visual behavior modality, the model identifies the user's action type based on key point sequences and behavioral features, such as approaching, touching, or waving, and generates corresponding semantic representations. The entire parsing process employs a sliding time window mechanism, with a window length that can be set from 200 milliseconds to 500 milliseconds and a step size of 50 milliseconds, to ensure the continuity and real-time performance of semantic parsing.

[0048] Temporal alignment of different modal features ensures that association analysis is performed within the same time window. Subsequently, correlation analysis methods, such as cosine similarity or mutual information calculation, are used to assess the degree of association between tactile, speech, and visual features. When tactile intensity increases while speech emotion intensifies and visual actions exhibit similar behaviors, a strong association between the multimodal features can be determined. Furthermore, an attention mechanism model can be introduced to assign dynamic weights to different modal features, thereby highlighting key modal information. Weight values ​​are typically normalized to between 0 and 1 and adjusted based on real-time data changes. Additionally, a graph structure modeling approach can be used, treating each modal feature as a node and representing its association strength through edge weights, thus constructing a multimodal association graph.

[0049] The model integrates tactile semantics, speech semantics, and visual behavioral semantics with their associated features for comprehensive reasoning. The model employs an attention-based fusion structure, assigning different weights to different modal features to achieve information complementarity and reinforcement. When the tactile signal indicates continuous pressing, the speech semantics expresses a command intent, and the visual behavior presents a proximity action, the model can comprehensively determine it as a request for interaction or an execution of a command. During the fusion process, the features of each modality are weighted and summed or concatenated, and then non-linearly mapped through a fully connected layer to output the user interaction intent category. The intent category can be predefined with multiple labels, such as interaction requests, emotional expressions, or control commands, with the number of categories ranging from 5 to 15. Simultaneously, a corresponding intent feature vector is output to describe fine-grained information about the intent. The intent sequence is organized chronologically, and a time-series model is constructed for analysis. First, the intent data is segmented using a sliding window method, for example, with a window length of 1 to 3 seconds and a step size of 0.5 seconds, to capture short-term trends. Then, by calculating the frequency of change and transition probability of intent categories within adjacent time windows, an intent transition matrix is ​​constructed to analyze the patterns of intent change. For example, consider the change in probability from observation to contact. Furthermore, exponential smoothing can be used to smooth the intent sequence to reduce the impact of short-term fluctuations. For complex scenarios, time-series models can be introduced to model changes in intent, thereby predicting future trends.

[0050] Based on the probability distribution output by the classification model, a confidence value is calculated for each type of intent. The model typically maps the output to a probability form using a normalization function, with the sum of the confidence values ​​for each category being 1. For each time window, the maximum probability value is extracted as the intent graph confidence, while retaining the probability information of other candidate intents. For example, an intent graph confidence value above 0.7 indicates high reliability, while a value below 0.4 indicates high uncertainty. Furthermore, the confidence value can be adjusted by incorporating multimodal weight information; for instance, when tactile and visual modalities have high consistency, the confidence value of the relevant intent can be appropriately increased. To improve stability, the confidence values ​​of consecutive time windows can be averaged or weighted fused to reduce instantaneous misjudgments.

[0051] The analysis integrates intent change trend parameters with confidence information, for example, using a weighted evaluation method where confidence is the primary weighting factor and the change trend serves as a moderating factor. Intents with high confidence and stable trends are classified as explicit interaction states, such as continuous interaction or stable control states; those with large confidence fluctuations or frequent trend changes are classified as unstable or transitional states. During the evaluation process, a threshold is set: when the intent graph confidence continuously exceeds 0.6 for more than 2 seconds, it is classified as a stable intent state; when the confidence fluctuates between 0.3 and 0.6, it is classified as an ambiguous state. Furthermore, historical data can be used to refine the state and improve robustness.

[0052] In this example, the lightweight behavior recognition model built into the AI ​​chip adopts a multimodal fusion network structure based on an improvement of MobileNetV3 to achieve unified behavior recognition of tactile, speech, and visual data. The model consists of three parts: a lightweight feature extraction backbone network, a multimodal fusion attention module, and a behavior classification output module. The backbone network uses depthwise separable convolutions as its core structure to extract features from different modal inputs at low computational cost. The vision branch processes image or keypoint features, the speech branch processes acoustic features, and the tactile branch processes pressure and deformation features. The three feature paths are uniformly mapped to the same dimensional space in the intermediate layer, typically to 128- to 256-dimensional feature vectors, to ensure cross-modal comparability. The fusion module uses a lightweight attention mechanism to assign weights to features from different modalities, enabling the model to dynamically strengthen the most critical modal information in the current interaction. For example, it enhances tactile weights when touch is dominant and speech weights when speech is dominant, thus achieving adaptive fusion for multimodal behavior recognition.

[0053] The overall parameter size is controlled between approximately 1.2M and 3.5M to adapt to the edge computing capabilities of the AI ​​chip for the liquid silicone interactive robot. The vision branch is based on a trimmed MobileNetV3 structure, containing approximately 12 to 16 depthwise separable convolutional layers with 16 to 96 channels to reduce computational complexity. The speech branch uses a lightweight temporal convolutional structure to encode 13- to 64-dimensional acoustic features, with parameters accounting for approximately 15% to 25% of the overall model. The haptic branch uses a one-dimensional convolutional network structure to model pressure and deformation features, with parameters accounting for approximately 10% to 15%. The fusion module uses a 2- to 3-layer lightweight attention structure, with each attention head set to 32 to 64 dimensions to achieve cross-modal feature interaction computation. The classification output layer uses a single-layer fully connected structure, typically setting the number of output behavior categories to 8 to 16, such as touch, press, drag, and voice commands. The overall model computation is controlled between approximately 1.0 GFLOPs and 2.0 GFLOPs, and the inference latency can be stably controlled within the range of 10 milliseconds to 40 milliseconds to meet the requirements of real-time interaction.

[0054] A modal-wise parallel reasoning and fusion decision-making mechanism is adopted. First, tactile, speech, and visual data are fed into their respective encoding branches for feature extraction. The visual branch extracts spatial and motion features based on MobileNetV3, the speech branch extracts semantic and emotion-related features, and the tactile branch extracts pressure distribution and deformation features. Subsequently, the three types of features are mapped to a unified feature space and input into a multimodal attention fusion module for interactive computation. A dynamic weight adjustment mechanism is used to strengthen the contribution of key modalities. For example, when the tactile signal intensity exceeds a set threshold, such as 3 kPa, the tactile weight ratio is increased to above 0.5, thereby highlighting the physical interaction behavior features. The fused feature vector is input into the behavior classification output layer, and after Softmax normalization, the behavior category and corresponding probability distribution are output. The user behavior recognition result is output, while retaining the contribution information of each modality for subsequent AI chip decision-making module feedback control, realizing the real-time intelligent response and behavior understanding capabilities of the liquid silicone interactive robot.

[0055] In this embodiment, the specific steps of step S4 are as follows: Based on the interaction intent state, perform interaction requirement analysis to generate interaction action requirements; The interaction action requirements are input into a preset behavior strategy library for retrieval and matching, and behavior strategy entries are output. Deep understanding of dialogue semantics based on interactive voice features, generating structured interactive text and emotion tags; Based on emotion tags, the interaction parameters are adaptively matched, and the interaction voice style and speech rate rhythm parameters are output. Structured interactive text is encapsulated based on interactive speech style and speech rate rhythm parameters to generate interactive speech packages.

[0056] In this embodiment, a hybrid analysis module combining rule-based reasoning and lightweight neural networks is built within the AI ​​chip to semantically map intent states, transforming abstract intents into specific interactive action requirements. When the intent state is a high-confidence proactive interaction and the tactile intensity is greater than 3 kPa, it is mapped to a responsive contact enhancement requirement; when the voice emotion tag indicates excitement and the visual behavior indicates a proximity action, it is mapped to a proactive guided interaction requirement. This mapping process employs a multi-condition decision-making mechanism, combining threshold judgment and probability weighting methods. For example, when the intent confidence is greater than 0.7, it preferentially enters the strong response path, and when it is less than 0.4, it enters the conservative response path. The output interactive action requirements are represented in a structured form, including action type, priority, and response intensity parameters. For example, the response intensity can be set to a continuous value between 0.1 and 1.0, and the action priority is divided into three or five levels.

[0057] The requirement is input into a pre-built behavior strategy database for retrieval and matching. This database is a pre-constructed multi-layered strategy database containing a set of behavior strategies based on a fusion of rules and learning models. Each strategy entry includes triggering conditions, an execution action sequence, and feedback rules. The retrieval process employs a combination of vector similarity matching and rule matching. The interactive action requirement is encoded as a feature vector, such as a requirement description vector with dimensions ranging from 32 to 64, and then cosine similarity is calculated with the strategy vectors stored in the database. A match is considered successful when the similarity exceeds a set threshold, such as 0.75. For multiple candidate strategies, a weighted scoring mechanism is used to comprehensively consider intent confidence, historical execution success rate, and current environmental state for ranking. The output behavior strategy entry contains a specific action execution sequence, such as adjusting haptic feedback intensity, setting a head turning angle range, or selecting a voice response mode, along with execution priority and constraints.

[0058] A lightweight semantic understanding model is invoked within the AI ​​chip to perform semantic parsing of speech feature vectors, jointly modeling acoustic features and text semantics. First, the speech signal is converted into initial text information by the speech recognition module. Then, a semantic understanding network is used for intent completion and semantic enhancement, such as supplementing implicit instructions or contextual information. Subsequently, a sentiment analysis model is employed to identify emotions in the speech features, outputting emotion labels such as calm, joy, or tension, with each label corresponding to a probability value ranging from 0 to 1. The structured interactive text adopts a hierarchical representation, including core semantic content, interactive objects, and behavioral instructions, such as user requests for interaction responses or user expressions of emotional changes. Simultaneously, the semantic results are bound to the emotion labels to form a unified structured semantic output, ensuring consistency between content and emotional expression in subsequent speech generation.

[0059] An emotion parameter mapping table is constructed to map different emotions to corresponding speech expression strategies. For example, when the emotion label is pleasant and the confidence level is greater than 0.8, a higher tone and faster speech rate strategy is selected, with the speech rate range set to 5 to 7 syllables per second; when the emotion is calm, a neutral tone and standard speech rate are used, approximately 4 to 5 syllables per second; when negative emotions such as tension or anxiety are detected, the speech rate is reduced to 3 to 4 syllables per second and the amplitude of tone variation is reduced. Vocal style parameters include tone softness, pitch fluctuation amplitude, and pause frequency, with pause time typically set between 200 and 600 milliseconds. This matching process employs a combination of fuzzy control and weighted rules, dynamically adjusting speech parameters based on the emotion probability distribution to achieve a high degree of consistency between the speech output and the user's emotional state.

[0060] Structured text is input into the speech synthesis module, and prosodic modeling is performed using grammatical style parameters, such as adjusting stress positions, pauses, and intonation curves. Speech synthesis employs a lightweight TTS model based on neural networks, generating natural speech by controlling parameters such as the fundamental frequency range of 80Hz to 300Hz, energy intensity, and speech rate rhythm. The text is then segmented according to the speech rate rhythm parameters, for example, long sentences are broken down into 5 to 15 speech units to ensure natural and fluent speech expression. Simultaneously, emotion tags are embedded into the speech synthesis control vector, enabling the speech output to convey emotions. The generated interactive speech package contains speech waveform data, prosodic control parameters, and timestamp information, which can be directly driven by the AI ​​chip to output through a speaker, achieving emotional speech feedback and human-computer interaction for the liquid silicone interactive robot.

[0061] In this embodiment, the specific steps of step S5 are as follows: The behavioral strategy items and interactive voice packages are broken down step by step to generate behavioral sequences and multiple voice sequences. Synchronous mapping is performed on behavioral sequences and multiple speech sequences, and the logical order is analyzed to obtain the logical sequence. Based on logical order, rhythm conflict and semantic inversion are identified, and logical rationality is adjusted to generate a set of behavioral actions with speech segments; The set of actions is compiled into instructions to generate control instructions.

[0062] In this embodiment, the action instructions in the behavior strategy entries are decomposed according to the execution granularity. For example, approaching the user and providing haptic feedback is broken down into four sub-action units: posture adjustment, motion path planning, contact detection, and haptic output. Each unit corresponds to a specific time segment, with a time resolution typically set to 50 milliseconds to 200 milliseconds. Simultaneously, the interactive voice package is decomposed into voice units. Continuous speech is divided into multiple speech segments according to semantic pauses and prosodic boundaries. For example, the length of each speech segment is controlled between 1 and 3 seconds, while retaining emotion tags and speech rate parameters. A timestamp-based alignment method is used during the decomposition process to ensure that the behavior segments and speech segments have a unified time indexing system. A behavior sequence and multiple speech sequences are generated. The behavior sequence includes action type, duration, and intensity parameters, while the speech sequence includes speech segment content, speech rate rhythm, and emotion markers.

[0063] The action sequence and speech sequence are aligned according to timestamps, and interpolation methods are used to correct sampling differences. For example, the 100-millisecond time granularity of the action sequence and the 200-millisecond granularity of the speech sequence are uniformly mapped to a standard 150-millisecond time window. Based on this, a sequence alignment algorithm is used to analyze the sequential and dependent relationships between the two types of sequences. For example, when a speech segment contains a start-contact prompt, the corresponding action sequence must trigger the contact action within 50 to 200 milliseconds after the speech ends. During logical sequence analysis, a method combining rules and probability is used to sort the execution order of actions and speech, generating logical sequence chains, such as structures like synchronous speech supplementing actions before speech prompts, and feedback speech summarizing actions after they are completed.

[0064] After obtaining the initial logical sequence, rhythmic conflict and semantic inversion detection are performed on the behavior sequence and speech sequence to ensure interaction consistency. Rhythmic conflict mainly refers to a mismatch between the speech rhythm and the behavior execution rhythm. For example, if the speech rate is 6 syllables per second, but the behavior execution cycle exceeds 2 seconds, it may lead to asynchronous interaction. Semantic inversion refers to the reverse order of speech content and behavior actions. For example, the voice prompt indicates that contact has been completed, but the actual tactile action has not yet been performed. The detection method uses time difference analysis and a semantic consistency matching model. By calculating the difference between the end time of the speech segment and the start time of the behavior action, a conflict is determined when the deviation exceeds a set threshold, such as 300 milliseconds. At the same time, a semantic consistency scoring mechanism is used to quantify the matching degree between speech and behavior, with a score range between 0 and 1. For detected conflicts or inversions, logical reordering and time axis correction methods are used for adjustment, such as delaying speech output or advancing behavior triggering, thereby generating a logically consistent set of behavior actions and ensuring that the speech segment and behavior action are completely consistent in time and semantics.

[0065] The high-level behavioral action set is broken down into control parameters that the device can recognize, such as motor control commands, haptic feedback intensity commands, and voice playback control commands. Each sub-action within the behavioral action is mapped to a specific control field. For example, motion control fields include speed parameters ranging from 0.1 m / s to 1.0 m / s, angle control ranges from 0 degrees to 180 degrees, and haptic feedback intensity control ranges from 0 to 100%. Voice control commands include playback timestamps, voice segment indexes, and speech rate parameters. A hybrid scheduling mechanism of sequential and parallel execution is used during compilation. Actions without dependencies are executed in parallel to improve real-time performance, while actions with dependencies are executed strictly in logical order. A unified control command set is generated, which can be directly input into the execution control module of the liquid silicone interactive robot to achieve synchronous and coordinated control of behavioral actions and voice output, ensuring consistency and stability at both the physical and semantic levels.

[0066] In this embodiment, step S6 is as follows: The instruction execution unit, which is driven by control commands to drive the liquid silicone device, completes the synchronous driving of interactive behavior and interactive voice, and collects real-time behavioral feedback data from the user during the execution process. The effectiveness of response strategies is evaluated based on real-time behavioral feedback data to obtain an effectiveness score. The lightweight behavior recognition model is fine-tuned locally based on the validity score to perform iterative optimization of the lightweight behavior recognition model.

[0067] In this embodiment, the motion control module drives the internal micro-motor or flexible drive structure according to the speed parameter in the control command, ranging from 0.1 m / s to 1.0 m / s, to realize the posture change and contact action of the liquid silicone device; the tactile feedback module outputs the corresponding pressure or deformation response according to the intensity parameter ranging from 0 to 100; the voice output module synchronously plays interactive voice according to the voice segment index and speech rate parameter, with the speech rate range typically set to 3 syllables per second to 7 syllables per second. To ensure synchronization, a unified time scheduling mechanism is adopted, mapping the behavior execution and voice playback to a millisecond-level time axis, for example, performing a state calibration every 50 milliseconds. During execution, the tactile sensor array, IMU inertial measurement unit, and vision acquisition unit collect real-time user behavior feedback data, including changes in user touch intensity, movement trajectory changes, and facial or posture reactions, with the sampling frequency typically set between 100Hz and 500Hz. All feedback data is timestamped and transmitted in real time to the AI ​​chip for caching and preliminary analysis to form a complete interactive data link.

[0068] The evaluation comprehensively considers three dimensions: behavioral consistency, user response intensity, and voice interaction matching degree. Behavioral consistency is assessed by comparing the deviation between the expected action and the actual action; for example, a posture angle error within 5 degrees is considered high consistency. User response intensity is quantified by analyzing the rate of change in tactile feedback, the frequency of user re-touch, and the dwell time; for example, a touch duration exceeding 2 seconds with stable pressure changes indicates a positive response. Voice interaction matching degree is assessed by analyzing the consistency between the user's emotional voice feedback and the voice output. Each indicator is assigned a weight; for example, behavioral consistency has a weight of 0.4, user response intensity 0.4, and voice matching degree 0.2. A weighted summation is used to calculate the effectiveness score of the response strategy, ranging from 0 to 1. A score greater than 0.8 indicates excellent strategy execution, between 0.5 and 0.8 indicates moderate effectiveness, and less than 0.5 indicates poor strategy effectiveness. The evaluation process employs a sliding window mechanism, updating the score every 500 milliseconds to ensure real-time and dynamic evaluation.

[0069] To incorporate the validity score as a feedback signal into the loss function adjustment process, for example, when the validity score is below 0.5, the weights of the corresponding modal channels are penalized to improve the model's sensitivity to this type of error scenario; when the score is above 0.8, the weight stability of the policy path is enhanced. The fine-tuning process only applies to a portion of the network layers, such as the last two convolutional layers and the fusion attention layer, to reduce computational overhead; the parameter update ratio is typically controlled between 1% and 5%. The learning rate is set relatively low, for example, between 0.0001 and 0.001, to avoid model oscillation. Simultaneously, an experience replay mechanism is employed, storing recent interaction feedback data in a cache pool, typically containing 1000 to 5000 data points, to enhance model stability. Through this local fine-tuning mechanism, the model can continuously adapt to changes in user behavior without undergoing full training, thereby improving the long-term interaction performance and recognition accuracy of the liquid silicone interactive robot.

[0070] In this embodiment, a liquid silicone device is provided for performing the AI ​​chip-based feedback parameter acquisition and processing method described above, including: The acquisition module is used to acquire multiple sensor parameters based on a liquid silicone device and generate a multimodal data stream; The encoding module is used to perform adaptive encoding processing on multimodal data streams and output multimodal sensing encoded sequences; The semantic parsing module is used to call the lightweight behavior recognition model built into the AI ​​chip to parse the semantic features of the multimodal perception encoding sequence one modality at a time and generate the interactive intent state. The requirements analysis module is used to perform interaction requirements analysis based on the interaction intent state and output behavior strategy items and interaction voice packs. The instruction compilation module is used to logically adjust the order of behavior policy entries and interactive voice packs and compile instructions to generate control instructions. The drive control module is used to drive the instruction execution unit of the liquid silicone device based on control commands, and to complete the synchronous driving operation of interactive behavior and interactive voice.

[0071] The liquid silicone device encapsulates multiple sensing nodes on its surface and in key internal interaction areas; the key internal interaction areas include a head, arms, and torso; the arms and torso encapsulate flexible tactile pressure sensors, temperature sensors, and inertial measurement units; the head encapsulates an AI chip, a visual acquisition unit, and a voice acquisition unit.

[0072] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0073] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for acquiring and processing feedback parameters based on an AI chip, characterized in that, Includes the following steps: Step S1: Acquire multi-sensor parameters based on the liquid silicone device and generate a multimodal data stream; Step S2: Perform adaptive encoding processing on the multimodal data stream and output a multimodal sensing encoded sequence; Step S3: Call the lightweight behavior recognition model built into the AI ​​chip to perform modal semantic feature parsing on the multimodal perception encoding sequence and generate the interaction intent state; Step S4: Analyze the interaction requirements based on the interaction intent state, and output the behavior strategy items and interaction voice packs; Step S5: Adjust the logical order of the behavior strategy items and interactive voice packages and compile the instructions to generate control instructions; Step S6: Based on the control commands, drive the instruction execution unit of the liquid silicone device to complete the synchronous driving operation of interactive behavior and interactive voice.

2. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, The specific steps of step S1 are as follows: Multi-sensor parameter acquisition based on liquid silicone device; extraction of user interaction dataset. The user interaction dataset includes touch feedback parameters, user voice signals, and visual images; Calculate the sampling delay difference and time drift between different sensing nodes; The user interaction dataset is subjected to multi-source synchronous calibration based on the sampling delay difference and time drift to generate a multimodal data stream.

3. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, The specific steps of step S2 are as follows: The multimodal data stream includes touch feedback parameters, user voice signals, and visual images; Deformation parameters of flexible tactile signals are identified based on touch feedback parameters; Calculate the motor vibration interference value of the touch feedback parameters; Dynamic filtering is performed based on the deformation parameters and the motor vibration interference value to generate filtered tactile parameters; The pressure distribution of the filtered tactile parameters is analyzed to extract the contact intensity, area of ​​action and rate of change, and to construct tactile distribution features. Semantic parsing and emotion analysis of user voice signals are performed to generate interactive voice features; Target detection and key point extraction are performed on visual images to obtain key points of user interaction; Visual behavior analysis is performed on key user interaction points to generate interaction behavior features; Adaptive encoding is performed on tactile distribution features, interactive speech features, and interactive behavior features to output a multimodal perception encoding sequence.

4. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, Step S3 is as follows: The lightweight behavior recognition model built into the AI ​​chip is invoked to perform modal semantic feature parsing on the multimodal perception coding sequence, generating multimodal semantic features; Cross-modal correlation analysis is performed on multimodal sensing encoded sequences to identify interactive correlation features between different modalities; Based on the aforementioned interaction association features, user interaction intent is parsed from the multimodal semantic features to generate user interaction intent; Perform time-series analysis on user interaction intent to obtain the trend of intent change; Calculate the confidence score for each intent category based on the user's interaction intent to obtain the confidence score for multiple intents; The trend of intent change is comprehensively evaluated based on the confidence level to generate the interaction intent state.

5. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, The specific steps of step S4 are as follows: Based on the interaction intent state, perform interaction requirement analysis to generate interaction action requirements; The interaction action requirements are input into a preset behavior strategy library for retrieval and matching, and behavior strategy entries are output. Deep understanding of dialogue semantics based on interactive voice features, generating structured interactive text and emotion tags; Based on emotion tags, the interaction parameters are adaptively matched, and the interaction voice style and speech rate rhythm parameters are output. Structured interactive text is encapsulated based on interactive speech style and speech rate rhythm parameters to generate interactive speech packages.

6. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, The specific steps of step S5 are as follows: The behavioral strategy items and interactive voice packages are broken down step by step to generate behavioral sequences and multiple voice sequences. Synchronous mapping is performed on behavioral sequences and multiple speech sequences, and the logical order is analyzed to obtain the logical sequence. Based on logical order, rhythm conflict and semantic inversion are identified, and logical rationality is adjusted to generate a set of behavioral actions with speech segments; The set of actions is compiled into instructions to generate control instructions.

7. The feedback parameter acquisition and processing method based on an AI chip according to claim 1, characterized in that, The specific steps of step S6 are as follows: The instruction execution unit, which is driven by control commands to drive the liquid silicone device, completes the synchronous driving of interactive behavior and interactive voice, and collects real-time behavioral feedback data from the user during the execution process. The effectiveness of response strategies is evaluated based on real-time behavioral feedback data to obtain an effectiveness score. The lightweight behavior recognition model is fine-tuned locally based on the validity score to perform iterative optimization of the lightweight behavior recognition model.

8. A liquid silicone device, characterized in that, The method for performing the feedback parameter acquisition and processing method based on an AI chip as described in claim 1 includes: The acquisition module is used to acquire multiple sensor parameters based on a liquid silicone device and generate a multimodal data stream; The encoding module is used to perform adaptive encoding processing on multimodal data streams and output multimodal sensing encoded sequences; The semantic parsing module is used to call the lightweight behavior recognition model built into the AI ​​chip to parse the semantic features of the multimodal perception encoding sequence one modality at a time and generate the interactive intent state. The requirements analysis module is used to perform interaction requirements analysis based on the interaction intent state and output behavior strategy items and interaction voice packs. The instruction compilation module is used to logically adjust the order of behavior policy entries and interactive voice packs and compile instructions to generate control instructions. The drive control module is used to drive the instruction execution unit of the liquid silicone device based on control commands, and to complete the synchronous driving operation of interactive behavior and interactive voice.

9. The liquid silica gel device according to claim 8, characterized in that, The liquid silicone device encapsulates multiple sensing nodes on its surface and in key internal interaction areas; the key internal interaction areas include a head, arms, and torso; the arms and torso encapsulate flexible tactile pressure sensors, temperature sensors, and inertial measurement units; the head encapsulates an AI chip, a visual acquisition unit, and a voice acquisition unit.

Images