vem - multimodal affective skin system

Abstract

The present application relates to the field of artificial intelligence, and discloses a VEM-multimodal emotional skin system, which adopts a micro electromagnet to realize expression simulation of skin through a magnetic peristalsis method in a soft magnetic colloid, and adopts various touch sensing methods to realize touch sensing of the skin, wherein the multimodal emotional skin system supports a VEM-Token rhythm model, a VEM-sync emotion synchronization model and a VEM-fe expression function model through expression instructions and touch functions which can be closed-loop and standardized, forms an innovative scheme of emotional skin, provides a multimodal emotional skin product of expression output and touch sensing input of a humanoid robot, realizes an emotional simulation function of 'blushing at a touch', and can be used for facial skin and body skin.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to the skin system structure and control of humanoid robots, embodied robots and robotic animals, especially to the subdivision field of skin with dynamic expressions and touch sensing for humanoid robots, aiming to realize skin beat peristalsis and touch response. BACKGROUND

[0002] With the development of humanoid robots and embodied robots industry, the limbs of robots have been widely used due to the innovation of various moving parts. However, in terms of the skin of robots, the current progress is still unsatisfactory, especially the face skin as the core component of the emotional expression of humanoid robots of artificial intelligence. So far, there is still no suitable technology and manufacturing scheme, and in fact this dilemma hinders the development and application of humanoid robots and embodied robots.

[0003] According to the search of the invention team, there are currently the following types of skin technology solutions:

[0004] 1. Material-driven type

[0005] Pneumatic / hydraulic-driven soft skin: using elastic materials such as silicone to embed micro pneumatic networks (pneumatic artificial muscles, PAM) inside. By controlling the air pressure of different cavities, the skin can be locally inflated, contracted, and bent to form smiles, frowns, and other expressions. The advantages are soft movements and high bionics; the disadvantages are relatively slow response speed, the need for a complex pneumatic control system, and the risk of air leakage.

[0006] Shape memory material:

[0007] Shape memory alloy (SMA) wire / spring: embedded in the skin base, shrinks after heating, and pulls the skin to produce deformation. The advantages are large force and compact structure; the disadvantages are high power consumption, difficult heat management, limited cycle life, and slow response speed.

[0008] Shape memory polymer (SMP): can be programmed to change shape through heat, light, etc. It is mainly used for overall contour changes, but it is challenging for local, fine, and fast expression simulation.

[0009] Electroactive polymer (EAP): such as dielectric elastomer (DE). It can produce large strain under the action of electric field, similar to "artificial muscle". The theoretical potential is huge, but in practical application, it needs extremely high driving voltage (kV level), and the material durability and packaging technology are still challenges.

[0010] Tendon-cable drive: Arrange biomimetic tendons (like high-strength cords or hoses) under the flexible skin, pulled by a distal motor, to achieve displacement of specific points on the skin. This is a classic method in bionics, with relatively direct control, but the system is complex and prone to mechanical wear and noise.

[0011] 2. Mechanical structure type

[0012] Multi-DOF micro motor / linear actuator array: Install micro servo mechanisms under key points on the face (such as the tip of the eyebrow, the corner of the mouth), directly pushing and pulling the skin. This is the mainstream solution for many humanoid robots (such as Sophia, Ameca) at present. The advantages are precise control and fast response; the disadvantages are that the face may appear mechanical, there is noise, and the skin deformation in the non-driven point area may not be natural.

[0013] Linkage mechanism: Design a precise mechanical linkage driven by one or a few motors, and achieve coordinated motion of multiple expression points through mechanism conversion. It can simplify the control system, but the mechanical design precision is extremely high, and the expression mode is fixed, with limited flexibility.

[0014] 3. Hybrid drive and integrated system

[0015] Rigid-flexible hybrid drive: Core areas (such as mouth opening and closing) use motors, while subtle expressions (such as cheek bulging, wrinkles) use soft drives such as pneumatic or EAP. Aim to balance power, speed, and naturalness.

[0016] Multi-layer composite skin:

[0017] Functional layering: The outermost layer is a silicone layer with biomimetic texture and color; the middle layer is a strain layer embedded with driving elements (such as SMA wires, fluid channels); the inner layer may be a base layer connected to the robot's facial skeleton.

[0018] Sensor integration: Integrate flexible pressure, strain, and even temperature sensors into the skin to form a "sensing-driving" closed loop, enabling the robot to perceive touch and respond more dynamically.

[0019] 4. VEM-Token model technology

[0020] The team first proposed a set of innovative VEM (Vocal-Emotion-Multimodal) technologies, including the following 6 patented inventions, which form a patent pool:

[0021] 4.1. VEM-Token vocal-emotion-multimodal model

[0022] This refers to the model in "VEM-Token Vocal Emotional Multimodal Tokenization Song and Accompaniment Deep Learning Method, CN120126506B", which specifically includes the following steps and methods 1-4:

[0023] (1) Record emotions using one or more modalities, label the vocal emotional multimodal as VEM, and construct VEM classification, VEM coordinate system, VEM function, and VEM library. Vocal emotions include one or a combination of joy, sadness, anger, fear, disgust, surprise, calm, anticipation, trust, love, hate, emotion, and enmity. Multimodal includes one or a combination of lyrics, song, accompaniment, vocal style, music, emotional basis, accompaniment instruments, video, and image. The VEM coordinate system includes a coordinate axis system established based on independent emotions, opposite emotion pairs, and related opposite emotion groups.

[0024] (2) Collect vocal samples according to VEM classification, and have human vocal experts evaluate the emotions of the vocal samples in song and accompaniment. Use supervised learning and deep learning to train VEM functions to obtain VEM parameters and add them to the VEM library.

[0025] (3) Use VEM processor to beat mark vocal files and separate song stream and accompaniment stream. Based on the beat, VEM-Token is segmented into VEM-Token1 sequence and VEM-Token2 sequence, and added to the preprocessing library.

[0026] (4) Use deep learning to generate dialogue expression spectrum, lyrics spectrum, VEM-Token song spectrum, VEM-Token accompaniment spectrum, and VEM-Token music score.

[0027] 4.2. VEM-Token beat capture and beat alignment model

[0028] This refers to the model in "Method for Building VEM-Token Beat Capture and Alignment Model, CN120748450B", which specifically includes the following steps and methods 5-7:

[0029] (5) For vocal files, set a beat model including beat capture and beat alignment based on the VEM-Token vocal emotional multimodal model. Capture the beat of the vocal file, divide the vocal file into VEM-Token sequences based on the beat, and mark the position of the start of the beat and the end of the beat in each VEM-Token.

[0030] (6) Set the start point alignment model, including:

[0031] The sample file included in the vocal file and the user file sung by the user imitating the sample file are divided into VEM-Token1 sequence and VEM-Token2 sequence respectively, and the starting point of each VEM-Token1 is used to adjust the starting point of the VEM-Token2 at the corresponding position by using the starting point fine-tuning step, so that the starting point of the VEM-Token1 at the corresponding position is aligned.

[0032] For the dialogue paragraph and the beat included in the vocal dialogue file in the social, the starting point of each VEM-Token of each segment is adjusted by using the starting point fine-tuning step, starting from the second segment, taking the first segment as a reference, so that the starting point of the VEM-Token at the corresponding position of the first segment is aligned, until the end of all the loop segments.

[0033] (7) Set the end point alignment model, including:

[0034] According to the end point of each VEM-Token1, the end point of the VEM-Token2 at the corresponding position is adjusted by using the end point fine-tuning step, so that the end point of the VEM-Token1 at the corresponding position is aligned.

[0035] For each segment of the dialogue segment, the end point of each VEM-Token of each segment is adjusted by using the end point fine-tuning step, starting from the second segment, taking the first segment as a reference, so that the end point of the VEM-Token at the corresponding position of the first segment is aligned, until the end of all the loop segments.

[0036] 4.3. VEM-Token vocal emotion multi-modal magic modification model

[0037] This refers to the model in "Method for constructing VEM-Token vocal emotion multi-modal magic modification model, CN120853611B", which specifically includes the following 8, 9 steps and methods:

[0038] (8) Collect sample files and user files, and obtain VEM-Token1 sequence of the sample file and VEM-Token2 sequence of the user file by using beat capture and VEM-Token segmentation according to the VEM-Token model, and generate VEM-Token2 sequence by using beat alignment according to VEM-Token1 sequence for all VEM-Token2 sequences.

[0039] (9) According to the VEM parameters included in the VEM-Token model, identify the VEM parameters of the VEM-Token1 sequence, determine the magic modification scheme by the user, process the VEM parameters of the VEM-Token2 sequence, and generate the VEM-Token2 sequence of the result file which conforms to the magic modification scheme of the style of the VEM-Token1 sequence.

[0040] 4.4. VEM-Token emotion synchronization function hierarchical fusion method

[0041] This refers to the model in "VEM-Token emotion synchronization function hierarchical fusion method, CN120913602B", which specifically includes the following 10, 11, and 12 steps and methods:

[0042] (10) Adopting VEM-Token vocal emotion multi-modal model, the song file is segmented into VEM-Token sequence by beat as unit, and the emotion synchronization function is set as VEM-sync, wherein VEM-sync includes VEM-sync vector sequence corresponding to and aligned with VEM-Token sequence, and VEM-sync vector includes synchronization content and synchronization pointer pointing to corresponding VEM-Token beat.

[0043] (11) According to VEM-Token sequence, the synchronization content of VEM-sync vector sequence is calculated and obtained, and the synchronization content includes beat attribute and emotion attribute, wherein the emotion attribute includes one or more emotion name, emotion value, and emotion weight.

[0044] (12) Emotion weight function includes: adopting hierarchical processing of song file to obtain corresponding VEM-sync vector sequence, and adopting one or combination of multi-layer weighted scanning, recurrent neural network, long short-term memory network, self-attention mechanism, and retrieval enhancement to forward propagate, backward propagate, and omnidirectional propagate the hierarchical emotion weight included in the emotion value of the emotion weight function, and fusion is obtained.

[0045] 4.5. VEM-Token world model robot expression function

[0046] This refers to "VEM-Token world model robot expression function construction method, CN120951102A", which specifically includes the following 13 and 14 steps and methods:

[0047] (13) Constructing VEM-fe expression function model, including: synthesizing one or more expression vectors representing emotion input into output synthetic expression, aligning expression vectors in the current beat, eliminating expression conflicts between two or more expression vectors, and transitioning from the synthetic expression of the current beat to the synthetic expression of the next beat, so that the robot can understand emotion and learn expression.

[0048] (14) The robot adopts entity expression generator or animation expression generator to constitute expression display, connects expression driving model by VEM-fe expression function model, transmits one or more synthetic expressions, and displays facial expression.

[0049] 4.6. VEM-4D dynamic skin structure and dynamic expression model construction method

[0050] This refers to "VEM-4D dynamic skin structure and dynamic expression model construction method, CN120951102A", which specifically includes the following 15 to 18 steps and methods:

[0051] (15) VEM-4D dynamic skin includes 4D dynamic skin and driving module and 4D expression model, wherein:

[0052] (16) 4D dynamic skin includes 2 or more electromagnets, electromagnets include parallel electromagnets composed of soft magnetic material and excitation coil, and parallel electromagnets are arranged side by side with a spacing A. The front of the 4D dynamic skin includes a skin adhered to the front magnetic pole of the parallel electromagnet. The skin is composed of an elastic and stretchable film.

[0053] (17) The driving module generates an attractive or repulsive force between the two ends of the electromagnet by applying excitation current to the excitation coil, and drives the skin to produce stretch deformation.

[0054] (18) VEM-4D dynamic skin also includes a 4D expression model, which provides static and dynamic expression instructions for the driving module to drive the skin to stretch and deform.

[0055] Disadvantages of prior art methods

[0056] (1) The prior art solution cannot solve the two-in-one demand of expression output and touch input.

[0057] (2) No matter whether it is the existing material driving type, mechanical structure type, hybrid driving or integrated system solution to solve the skin expression generation, no applicable product has been found so far. SUMMARY

[0058] The team of the present invention proposes a new "VEM-multimodal emotional skin system" to overcome the shortcomings of the prior art. This solution combines expression output and touch input together, has a simple structure, is easy to productize, and has low cost, and is easy to integrate with current artificial intelligence.

[0059] The purpose and intention of the present invention is achieved by using the following structure and method:

[0060] 1. VEM-multimodal emotional skin system

[0061] The present invention is a VEM-multimodal emotional skin system, which includes but is not limited to the following structure:

[0062] The VEM-multimodal emotional skin system includes but is not limited to electromagnets, soft magnetic colloids, and multimodal mutual inductance systems.

[0063] The electromagnet includes but is not limited to a soft magnetic core and an excitation coil, and an excitation current passes through the excitation coil to generate a magnetic pole of the electromagnet. Two or more electromagnets are arranged in a spaced manner to form an electromagnet array. The soft magnetic colloid fills the space to form a mutual inductance magnetic circuit, thereby forming the emotional skin.

[0064] The soft magnetic colloid is formed by mixing soft magnetic material particles and elastic colloid, and has a resilience to restore to the original state in a static state.

[0065] The multi-modal mutual inductance system receives and executes the expression instruction, provides an excitation current for the electromagnet, and generates a magnetic creep on the surface of the emotional skin to simulate the expression.

[0066] The multi-modal mutual inductance system also has a touch sensing function. When an external object applies a touch action on the surface, the touch action is converted and calculated to form a touch signal.

[0067] 2. The multi-modal mutual inductance system includes an expression loop.

[0068] On the basis of the foregoing basic scheme, the expression loop of the multi-modal mutual inductance system of the present application includes but is not limited to one or a combination of the following structures:

[0069] The expression loop includes but is not limited to an expression circuit and an expression magnetic circuit. The expression circuit executes the expression instruction, provides an excitation current for the electromagnet array, and generates a magnetic pole. The magnetic pole and the soft magnetic colloid form an expression magnetic circuit. Under the action of the magnetic attraction and repulsion force between the magnetic poles overcoming the resilience of the soft magnetic colloid, a dynamic change process of the magnetic circuit is generated between the magnetic poles and the adjacent soft magnetic colloid, resulting in a magnetic creep on the emotional skin and the surface to simulate a dynamic expression.

[0070] The expression circuit includes but is not limited to a circuit for series connection, parallel connection and conversion of two or more excitation coils, a circuit for current polarity conversion when each excitation coil is powered, a circuit powered in a constant current source and constant voltage source manner, and a logic module for accepting and controlling the expression instruction.

[0071] 3. The expression instruction:

[0072] On the basis of the foregoing scheme, the expression instruction of the present application includes but is not limited to one or a combination of the following:

[0073] The expression instruction is generated based on an expression function FE(x, y, z, t, i, p, v, τ), wherein x, y, and z in the vector (x, y, z, t, i, p, v, τ) are respectively the coordinates of the three-dimensional space of the electromagnet, t is a fourth-dimensional time value coordinate, i, p, v, and τ are respectively the current value, current polarity, electromagnet type, and step time running in the excitation coil. The step time is the time length of the transition from the previous step expression to the next step expression in the dynamic expression, and the step time is greater than the time value.

[0074] The expression instruction generates a magnetic creep sequence, including but not limited to a sequence according to a time value, driving electromagnets in a three-dimensional space coordinate in a stepping sequence, so that adjacent electromagnets are attracted and repelled in a stepping sequence, and finally magnetic creep is generated on the surface of the emotional skin to simulate the final expression.

[0075] The expression function supports a VEM-fe expression function model, specifically including but not limited to: synthesizing more than one expression vector representing an emotional input into a synthesized expression as output, aligning expression vectors in a current beat, eliminating expression conflicts between two or more expression vectors, and transitioning from a synthesized expression of a current beat to a synthesized expression of a next beat, wherein a step size uses a beat model.

[0076] The driving implementation of the expression function includes but is not limited to direct current pulses and pulse width modulation (PWM).

[0077] 4. Touch loop:

[0078] On the basis of the foregoing scheme, the present application includes but is not limited to one or a combination of the following in terms of the touch loop:

[0079] The touch loop includes but is not limited to a sensing and touch circuit one that multiplexes a touch magnetic circuit. When an external touch action is applied to the surface, the touch pressure overcomes the resilience of the soft magnetic colloid, the soft magnetic colloid is locally deformed due to the touch point and the touch path under the touch pressure, and a touch magnetic circuit is formed by disturbing the expression magnetic circuit, resulting in a dynamic change process of the magnetic circuit between the soft magnetic colloid and the adjacent magnetic poles. The touch circuit one senses the dynamic change process of the magnetic circuit through magnetic circuit multiplexing to generate electromagnetic induction in the excitation coil, and outputs a touch signal calculated by a multi-modal mutual inductance system sensing.

[0080] The touch loop includes but is not limited to a sensing and touch circuit two that multiplexes a touch coil. When an external finger touch action is applied to the surface, the excitation coil of the electromagnet at the touch point is in a capacitor mode, and the touch voltage is sensed by the multiplexed touch coil. The touch circuit two captures the change process of the excitation coil voltage, and outputs a touch signal calculated by a multi-modal mutual inductance system sensing.

[0081] The touch loop includes but is not limited to a sensing and touch circuit three that multiplexes a thin film resistance array. A flexible touch sensing thin film resistance array is arranged in the emotional skin, and the touch circuit three connects the thin film resistance array to collect each resistance value. When an external touch action is applied to the surface, the resistance value of the thin film resistance at the touch point changes, and a touch signal is output by a multi-modal mutual inductance system sensing.

[0082] 5. Touch circuit:

[0083] Based on the aforementioned solutions, the present invention, in terms of touch circuitry, includes, but is not limited to, one or more combinations of the following:

[0084] Touch circuit 1 includes, but is not limited to, a low-pass filter and a signal processor 1. The cutoff frequency of the low-pass filter is lower than the frequency of the excitation current. The signal processor 1 is connected to the circuit of the excitation coil. In the multiplexed touch magnetic circuit algorithm of the multi-mode mutual inductance system, including but not limited to, it detects and calculates the signal quantity of the touch signal. When the signal quantity of the touch signal is greater than the set minimum value, it determines and records the touch as a valid touch signal. The low-pass filter is used to isolate the excitation current, and the signal processor 1 is used to amplify and shape the touch signal.

[0085] Touch circuit two uses a high-pass filter and signal processor two to connect to the excitation coil. The cutoff frequency of the high-pass filter is higher than the frequency of the excitation current. In the multiplexed touch coil algorithm of the multi-mode mutual inductance system, including but not limited to, the signal quantity of the touch signal is detected and calculated. When the signal quantity of the touch signal is greater than the set minimum value, the touch signal is determined and recorded as a valid touch signal. The high-pass filter is used to eliminate interference signals, and the signal processor two is used to amplify and shape the touch signal.

[0086] The touch circuit three uses a thin-film resistor array signal processor three. In the multi-mode mutual inductance system, including but not limited to the thin-film resistor array algorithm, it detects and calculates the signal quantity of the touch signal. When the signal quantity of the touch signal is greater than the set minimum value, it determines and records the touch signal as a valid touch signal.

[0087] 6. Touch signal:

[0088] Based on the aforementioned solutions, the present invention, in terms of touch signals, includes, but is not limited to, one or more combinations of the following:

[0089] The touch signal includes at least the touch function TM(x, y, z, t), where x, y, and z in the vector (x, y, z, t) are the coordinates of the electromagnet or thin-film resistor array in three-dimensional space, and t is the fourth-dimensional time value coordinate.

[0090] A set of touch signals includes, but is not limited to, the collection of all touch signals over a period of time.

[0091] The touch signal also includes, but is not limited to, increased touch intensity, where the touch function includes TM(x, y, z, t, h), where h is the touch intensity.

[0092] Touch signals also include, but are not limited to, instructions submitted to the multimodal sensing system and subsequent systems to process and generate set facial expression instructions and subsequent control instructions.

[0093] The multi-modal mutual inductance system cyclically scans an electromagnet array or a thin film resistance array, detects, calculates, records coordinates, time values and effective touch signals of the electromagnet or the thin film resistance array, and generates a touch signal.

[0094] 7. The electromagnet:

[0095] On the basis of the foregoing scheme, the present application includes, but is not limited to, one or more combinations of the following in terms of electromagnets:

[0096] The bar-shaped sheet electromagnet is made of bar-shaped sheet soft magnetic material to make an electromagnet core, and a group of excitation coils wound on the core to generate magnetic poles at both ends of the bar-shaped sheet electromagnet, and two or more bar-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and is installed on the inner side of the skin to form emotional skin.

[0097] The cross-shaped sheet electromagnet is made of cross-shaped sheet soft magnetic material to make an electromagnet core, and excitation coils are wound on the four arms of the cross-shaped sheet to generate magnetic poles at the four arms, and two or more cross-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and is installed on the inner side of the skin to form emotional skin.

[0098] The parallel electromagnet is made of bar-shaped cylindrical or rectangular soft magnetic material to make an electromagnet core, and a group of excitation coils are wound on the bar-shaped sheet to generate magnetic poles at both ends of the bar-shaped sheet electromagnet, and two or more parallel electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is perpendicular to the skin and is installed on the inner side of the skin to form emotional skin.

[0099] The concave-convex electromagnet is installed on the back plate side of the emotional skin, and is used to attract and repel parallel electromagnets, bar-shaped sheet electromagnets and cross-shaped sheet electromagnets, and the concave-convex electromagnet includes a group of excitation coils to generate magnetic poles at both ends of the concave-convex electromagnet, and a mutual inductance magnetic circuit is formed by filling and wrapping the soft magnetic colloid to form emotional skin.

[0100] 8. The multi-modal mutual inductance system supports a beat model:

[0101] On the basis of the foregoing scheme, the present application supports a beat model, including but not limited to one or more combinations of the following:

[0102] In the touch function TM(x, y, z, t), the time value coordinates are aligned with the start beat and the end beat of the beat model through start point fine tuning and end point fine tuning, so that the collection of touch signals is generated according to the beat model, thereby generating a beat feeling.

[0103] In the expression function FE (x, y, z, t, i, p, v, tau), the time value coordinates are aligned with the start and end beats of the beat model through start and end fine-tuning, so that the effect of the expression function is generated according to the beat model, thereby generating a sense of beat.

[0104] The beat model includes but is not limited to the beat model in the compatible VEM-Token vocal emotion multi-modal model and the beat definition in music.

[0105] 9. The multi-modal mutual sensing system supports the VEM-sync emotion synchronization model:

[0106] On the basis of the foregoing scheme, the emotion synchronization model supported by the present application includes but is not limited to one or a combination of the following:

[0107] The emotion synchronization model function is set as VEM-sync, wherein the VEM-sync includes but is not limited to a VEM-sync vector sequence corresponding to and aligned with the VEM-Token sequence, and the VEM-sync vector includes synchronization content and an emotion synchronization pointer pointing to the corresponding VEM-Token beat.

[0108] In the touch function TM (x, y, z, t, h), the time value coordinates are aligned with the emotion synchronization pointers in the VEM-sync vector sequence through start and end fine-tuning, so that the collection of touch signals is generated according to the emotion synchronization model, thereby generating a sense of emotion.

[0109] In the expression function FE (x, y, z, t, i, p, v, tau), the time value coordinates are aligned with the emotion synchronization pointers in the VEM-sync vector sequence through start and end fine-tuning, so that the effect of the expression function is generated according to the emotion synchronization model, thereby generating a sense of emotion.

[0110] The emotion synchronization model includes but is not limited to the VEM-sync emotion synchronization model in the compatible VEM-Token emotion synchronization function hierarchical fusion method and the emotion definition in social interaction.

[0111] 10. The multi-modal mutual sensing system supports the VEM-fe expression function model:

[0112] On the basis of the foregoing scheme, the VEM-fe expression function model supported by the multi-modal mutual sensing system of the present application specifically includes but is not limited to one or a combination of the following:

[0113] Expression alignment includes but is not limited to two or more expression vectors or a synthetic expression, and start and end alignment is performed on the beat;

[0114] In the touch function TM (x, y, z, t, h), the time value coordinates are aligned with the expression vectors in the VEM-fe expression function model through the start point fine adjustment and the end point fine adjustment, so that the collection of the touch signal is generated in accordance with the expression synchronization, thereby generating the emotional simulation effect.

[0115] In the expression function FE (x, y, z, t, i, p, v, tau), the time value coordinates are aligned with the expression vectors in the VEM-fe expression function model through the start point fine adjustment and the end point fine adjustment, so that the action of the expression function is generated in accordance with the expression synchronization, thereby generating the emotional simulation effect.

[0116] The VEM-fe expression function model includes but is not limited to the VEM-fe expression function model in the method of constructing the robot expression function compatible with the VEM-Token world model and the expression definition in social interaction.

[0117] 11. A multimodal mutual sensing input system:

[0118] On the basis of the foregoing scheme, the present application on the multimodal mutual sensing input system specifically includes but is not limited to one or a combination of the following:

[0119] The multimodal mutual sensing input function is set as MMSI (x, y, z, t, s1, fx), wherein the vectors x, y and z are the position coordinates of the input sensors, t is the time value coordinate or the beat number, s1 is the input sensor type, and fx is the input value sequence.

[0120] The input sensor type includes but is not limited to sound sensing, temperature sensing, movement sensing and visual sensing, including one or a combination thereof.

[0121] The subcutaneous arrangement includes platinum resistance wire to transmit the temperature in the emotional sensing skin.

[0122] 12. A multimodal mutual sensing output system:

[0123] On the basis of the foregoing scheme, the present application on the multimodal mutual sensing output system specifically includes but is not limited to one or a combination of the following:

[0124] The multimodal mutual sensing output function is set as MMIO (x, y, z, t, s2, fx), wherein the vectors x, y and z are the position coordinates of the output sensors, t is the time value coordinate or the beat number, s2 is the output actuator type, and fx is the output value sequence.

[0125] The output actuator type includes but is not limited to the skin discoloration actuator, the local motion actuator and the overall motion actuator, including one or a combination thereof.

[0126] The skin includes, but is not limited to, being composed of elastic, wear-resistant, cosmetic and washable materials and soft magnetic colloidal adhesion.

[0127] The skin includes, but is not limited to, electric heating wires and temperature control systems to adjust the temperature of the skin.

[0128] 13. Invention purpose and intention

[0129] The VEM-multimodal emotional skin system of the present invention has the purpose and intention of:

[0130] (1) Invention of a multimodal emotional skin that can simulate dynamic skin expressions and solve multiple touch sensing at the same time.

[0131] (2) Support for emotional multimodal rhythm models, VEM-sync emotional synchronization models, and VEM-fe expression function models.

[0132] (3) Support for artificial intelligence control methods.

[0133] 14. Beneficial effects of the invention

[0134] (1) Simple structure, easy to mass-produce, and easy to develop secondarily.

[0135] (2) Can be used for facial skin of artificial intelligence robots with touch feeling, realizing, for example, "blushing at a touch" personification interaction function.

[0136] (3) Can be used for facial skin of machine pets and other intelligent organisms, easy to produce offline end-to-end products, and easy to access existing large model systems. BRIEF DESCRIPTION OF DRAWINGS

[0137] LIST OF DRAWINGS:

[0138] Figure 1 : VEM-multimodal emotional skin system schematic diagram

[0139] Figure 2 : Emotional skin structure schematic diagram

[0140] Figure 3 : Parallel electromagnet concave-convex electromagnet working schematic diagram

[0141] Figure 4 : A row of bar-shaped sheet electromagnets mutual attraction schematic diagram

[0142] Figure 5 : Bar-shaped sheet electromagnet array mutual attraction schematic diagram

[0143] Figure 6 : Cross-shaped sheet electromagnet mutual attraction schematic diagram

[0144] Figure 7Touch path and sensing diagram

[0145] Figure 8 Multiplexing sensing principle diagram of electromagnet

[0146] Figure 9 Beat model diagram

[0147] Figure 10 Emotion synchronization model diagram

[0148] Figure 11 Expression function diagram

[0149] Detailed description of the drawings:

[0150] Detailed description of each drawing, please see the following corresponding description of each drawing. It needs to be emphasized that these drawings are only one of the illustrations of the innovative ideas of the present application, and are only a kind of drawing. The system and structure in the figure are not marked with specific dimensions, and the user of the present application can give detailed design dimensions according to the specific application. Therefore, the drawings here are not a limitation of the innovative ideas of the present application. The user of the present application can draw other ways of drawings and their understanding and explanation according to the knowledge commonly used in the industry.

[0151] Figure 1 :

[0152] This is a VEM-multimodal emotion skin system diagram. The system includes two parts, the upper part is the emotion skin, which contains electromagnets, soft magnetic colloid and some structural parts, which constitute the main body of the emotion skin. The lower part is a multimodal mutual sensing system, which also contains expression circuit and touch circuit, under the support and management of the multimodal mutual sensing system, the control of the emotion skin is completed to realize the expression simulation output and the sensing of the skin touch.

[0153] Figure 2 :

[0154] This is an emotion skin structure diagram. It includes: back plate, sealing sleeve, epidermis, soft magnetic colloid, parallel electromagnet, excitation coil, front magnetic pole, back magnetic pole, transverse partition, transverse partition support, epidermis positioning column, concave-convex electromagnet, etc. Among them, the soft magnetic colloid is a mixture of soft magnetic material particles and elastic colloid. The mixture not only fills the space between the electromagnets, but also forms a magnetic circuit together with the magnetic poles of the electromagnets.

[0155] In the selection of electromagnet, 1, sheet-shaped electromagnet is mainly used for thin skin, such as eyelid, forehead and hand, which has simple expression simulation and can be simulated by simple magnetostriction. 2, parallel electromagnet is installed on thick skin due to its large size, and its advantage is that in the array structure of multiple parallel electromagnets, the number of front magnetic poles can be large, and different configurations of north and south poles can form complex magnetostriction on the skin to simulate delicate skin expression. 3, concave-convex electromagnet is mainly used to attract and repel the back magnetic poles of parallel electromagnet to form concave and convex on the skin to increase the expression simulation effect, such as local concave of skin to simulate "dimple" of face, and local convex of skin to simulate "smile" of face. In addition, the combination of these electromagnets can also be used to simulate the expression and reaction of other positions of skin, which is only an example here.

[0156] In addition, the heat dissipation device is an optional item. When heat dissipation is not required, the sealing shell is simply made of non-sealing material, and there is no need for heat dissipation liquid inlet and outlet. When heat dissipation is required, the sealing shell is simply made of sealing material, and the heat dissipation liquid inlet and outlet are required. It should be noted that the subcutaneous tissue is made of foamed silica gel material, and a high foaming ratio is adopted, so that in the 4D dynamic skin, most of the bubbles become interconnected spaces, in which the heat dissipation liquid is filled, and the circulating pump is connected to the heat dissipation liquid inlet and outlet to form a circulating heat dissipation.

[0157] In addition, the coils of parallel electromagnets and concave-convex electromagnets are led to the outside of the sealing sleeve and fixed on the 4D dynamic skin through the cluster connector. The skin uses an elastic film which can be washed and made up, and is adhered to the front magnetic poles of the parallel electromagnets. When the front electromagnets are peristaltic, they drive the skin to produce stretching deformation and movement.

[0158] The function of the transverse septum support is to maintain the fixed distance between the transverse septum and the back plate at this position. When a concave-convex electromagnet is energized to generate an attractive or repulsive force on the back magnetic poles of the nearby parallel electromagnets, the deformation of the transverse septum is limited within the range of the transverse septum support. The function of the skin positioning column is to position the skin in three-dimensional direction at this position, and it will not move due to the energization of the electromagnets.

[0159] Figure 3 :

[0160] This is a working schematic diagram of parallel electromagnets and concave-convex electromagnets. As shown in Figure 1 The same mark, as Figure 2 The same principle, in Figure 3The upper half of the face, the four parallel electromagnets two pole excitation current is opposite, resulting in the front of the magnetic pole from top to bottom, respectively, S, N, S, N, so mutual attraction, resulting in the front of the skin contraction, while the periphery of the skin back stretch, simulate the skin expression. In Figure 3 The lower half of the face, due to the back of the parallel electromagnets and the adjacent magnetic pole of the concave-convex electromagnets at this place are opposite sex attraction, so the magnetic force makes the diaphragm at this place bend, pull the parallel electromagnets at this place to make the skin produce concave, simulate the skin concave expression (such as the dimples of the human face).

[0161] In the expression circuit and emotional skin, including: input interface, processor subsystem, codec, current source matrix, switch matrix, output matrix, connecting the excitation coil of the parallel electromagnet matrix, the excitation coil of the concave-convex electromagnet matrix, wherein, according to the need, can also be selected with wireless communication module, filter circuit, temperature sensor and the connection end of circulating pump.

[0162] In addition, the expression circuit also includes a switch matrix, which includes on-off switches and polarity conversion switches to control the on-off and polarity of the excitation current of the excitation coil, so that the magnetic poles of the electromagnets change in magnetism and polarity. The filter circuit is used to improve the working condition of the electromagnet and reduce vibration and noise. The wireless communication module is used for communication with the outside world.

[0163] The expression circuit also includes a drive module and its input end, which is connected to the expression function vector from the expression model, and a drive power supply. The temperature sensor and circulating pump are used to connect the processor subsystem in the drive module to form a temperature control system.

[0164] Figure 4 :

[0165] This is a schematic diagram of a row of strip-shaped electromagnets attracting each other. As shown in the figure, when multiple strip-shaped electromagnets are in a static state (their coils are not powered) and the soft magnetic colloid around the expression is not magnetized (the residual magnetism is zero), in this case, the strip-shaped electromagnets are arranged in the original state with a large distance between each other. When the excitation current is loaded in the excitation coil, the strip-shaped electromagnets are magnetized as shown in Figure 4 Due to the repulsive force overcoming the resilience of the soft magnetic colloid, the original distance is reduced, thereby causing the skin at this place to contract, and further simulating the expression of the skin. If we change the polarity of the strip-shaped electromagnets, replacing the opposite sex attraction with opposite sex repulsion, the area on the skin will be stretched and stretched, producing the opposite expression effect.

[0166] Figure 5 :

[0167] This is a schematic diagram of a strip-shaped electromagnet array attracting each other. As shown in Figure 4The introduction, here is the array of multi-bar-shaped electromagnet, from the skin, the area of contraction is larger than Figure 4 the area of expansion. In this way, we can achieve more complex skin stretching and contraction by planning the size and shape of the skin contraction area, and using bar-shaped electromagnet array to simulate more complex expressions.

[0168] Figure 6 :

[0169] This is a cross-shaped electromagnet mutual attraction diagram. The cross-shaped electromagnet is characterized by having four groups of excitation coils on the same electromagnet, generating four magnetic poles. According to the principle of magnetic pole attraction and repulsion, we can design more complex patterns on the skin plane. For example Figure 6 the design in , produces an inclined contraction line. In this way, we can also design more complex contraction and expansion plane patterns to simulate corresponding expressions on the skin.

[0170] Figure 7 :

[0171] This is a touch path and sensing diagram. In Figure 7 , T1 to T6 is a touch path, where, under the skin, is the position of the electromagnet array, such as parallel electromagnets or sheet-shaped electromagnets or their combination, at this time, according to the touch path, the touch signal is captured by the multi-modal mutual sensing system. Further, these touch signals are used to perform some further operations to form interactive functions. For example, this skin is used for the face of a humanoid robot, when the outside touches the face, the humanoid robot produces excited, shy, and even red-faced emotional responses to increase interest and usability.

[0172] Figure 8 :

[0173] This is a diagram of the principle of electromagnet multiplexing sensing. Figure 8 , FE is the expression instruction, P is the pulse generator, which is used to drive the expression instruction, the driving pulse is loaded on the excitation coil of the electromagnet, so that the electromagnet L generates a magnetic field and a magnetic pole. SMCE is a soft magnetic elastomer colloid. C1 and C2 are capacitors to isolate the direct current in the excitation coil and achieve filtering, A is a signal processor or a combination of signal processors, when the external touch and sliding load on the soft magnetic colloid, the magnetic circuit changes, through the processing of C1, C2 and A, the touch signal function is formed, and the touch function is output from the TM end.

[0174] Figure 9 :

[0175] This is a beat model diagram. In Figure 9 , the token of the sample file (VEM-Token1) and the token of the user file (VEM-Token2) are included. Among them, the user file is a vocal file that the user imitates the sample file to sing and record. According to music theory, the user file should be consistent with the sample file as much as possible. In this application, this consistency is first that the beat (Beat) should be consistent, that is, the beat start point and the beat end point of the user file need to be aligned with the beat start point and the beat end point of the sample file. It should be noted that according to the working principle of Feng's computer serial array, the file token of the vocal file is in the form of a numerical sequence (such as an array), which is divided into a VEM-Token sequence according to the music beat format. Among them, the sample file is divided into a VEM-Token1 sequence, and the user file is divided into a VEM-Token2 sequence. Each token defines a start point and an end point, and the start point of the next token coincides with the end point of the previous token, and the token sequence is defined as shown in Figure 1 .

[0176] In this step, the beat start point alignment model and the beat end point alignment model are included. Among them, the beat start point alignment model and the beat end point alignment model actually also have the process of beat start point capture and beat end point capture. Because, in the music file, there may also be "pause, breathing", and such "pause, breathing" in the array signal (spectrum format file) may be represented by array elements with signal quantity of 0 or close to 0. Therefore, in order to avoid including such interference, the application adopts a start point fine-tuning step and an end point fine-tuning step to further accurately capture the beat start point and the beat end point.

[0177] In addition, in a music file, there may also be a "loop segment". In the loop segment, the music score and the beat of the next segment and the previous segment include the same part and individual different parts. For the same part, the beat start point and the beat end point of the next segment and the previous segment need to be aligned, and for the different part, it is processed according to the actual start point and end point of the sample file.

[0178] It should be noted that usually in the sample file, since it is assumed that the sample file is sung and recorded by a professional singer according to the rules of music theory, the beat is accurate. However, the user may not be a professional singer, or may not be a professional music lover, so the start point of the beat and the end point of the beat cannot be strictly sung according to the rules of music theory, that is, the start point of the beat and the end point of the beat are not strictly consistent with the sample file. Therefore, in this case, the capture and alignment of the beat start point and the beat end point of the user file need to be strictly checked and corrected.

[0179] Figure 10 :

[0180] This is a schematic diagram of the emotion synchronization model. Figure 10 The lower part is the VEM-Token sequence divided by the song file, and the file attribute is a spectrum format file. The upper part is the emotion synchronization function VEM-sync, Beat is the beat, and n is the number of the sequence. The design intends to:

[0181] 1. VEM-Token is a division of a song file in a certain structure, including the VEM-Token vocal emotion multi-modal tokenization song and accompaniment deep learning method CN120126506, the VEM-Token vocal emotion multi-modal division model of CN120748450, and other division methods such as MIDI and MCS models. On the VEM-Token sequence of the song file, a VEM-sync vector sequence is attached, where the VEM-sync vector sequence corresponds to the VEM-Token sequence one by one.

[0182] 2. The correspondence between the VEM-sync vector sequence and the VEM-Token sequence is constrained by the synchronization pointer. It should be noted that although Figure 10 The VEM-sync vector sequence and the VEM-Token sequence in the above-mentioned CN120748450 are "equal length", but this equal length is only logical and not physical array length, because with different emotional complexity of the VEM-Token sequence of the song, the parsed emotion name, emotion value and emotion weight are different, and the storage space size is also different.

[0183] 3. Since the VEM-Token sequence is divided according to the musical beat, and the beat is the most basic unit of musical language, the invention regards the beat unit as the beat layer, and above the beat layer, there are also bar layer, sentence layer and whole song layer. According to this layer division, the VEM-sync vector sequence is correspondingly synchronized into beat layer, bar layer, sentence layer and whole song layer. It should be noted that this physical and temporal division method according to bar layer, sentence layer and whole song layer is only one of the division levels of the invention, not the only one. The invention also includes emotional, logical and spatial division methods.

[0184] 4. The emotion synchronization function VEM-sync, the so-called "synchronization" refers to the correspondence between the VEM-sync vector and the corresponding VEM-Token; the so-called "emotional synchronization" refers to the emotional description in the VEM-sync vector, which is synchronized with the VEM-Token; the so-called "layered fusion" in the invention refers to the "layered" analysis and "fusion" according to the layers, so as to obtain the emotional analysis result of the whole song.

[0185] 5、Need to note that the VEM-sync vector is a quantity with module length and direction, which can be represented as a matrix or array in mathematics. Its spatial direction information has been included in the data structure. Therefore, in this invention, the VEM-sync vector, vector and array are considered as the same concept and are not distinguished.

[0186] Figure 11

[0187] This is the expression function schematic diagram. It is also a diagram of the VEM-fe expression function model, which is analyzed as follows:

[0188] On the left side of Figure 11 , there are several expression vectors input into the synthesized expression vector module. It needs to be noted that here the synthesis is for the synthesis according to the classification, for example, regarding the eyeball mode, it may involve the expression vector due to the nearby sound, it may be the expression vector when the front desk of the hotel greets the guest, and it may be the expression vector when the guest is speaking, so it is necessary to consider these several expression vectors comprehensively, that is, "synthesis". In addition, these several expression vectors may also become the input of other modes, such as head rotation. Therefore, there is not a one-to-one correspondence between the input of the expression vector and the output of the synthesized vector, but a many-to-many mapping relationship. It needs to be noted that in the module, the beat division of the expression vector can be set, and the beat label can be added. For details, please refer to the model generation beat label in the "VEM-Token beat capture and alignment model construction method, 202511249168.0". The user of this patent can also design the beat division method by himself.

[0189] From the synthesized expression vector to the aligned expression vector, from the aligned expression vector to the expression conflict elimination, from the expression conflict elimination to the transition synthesized expression, refer to the subsequent steps 2, 3, 4 and 5 to complete the corresponding step formula. After generating the synthesized expression, submit the entity expression generator of the spatial robot mode or the animation expression generator of the digital robot mode, adopt micro-expression simulation or real-time expression, and output to the spatial robot or real-time expression respectively to show the expression. Here, the spatial robot is the head and face of the entity robot to show the expression, and the neck below can be connected by the spatial robot; while the real-time expression is completed by the display to show the animation.

[0190] It needs to be noted that in the module, the beat division of the expression vector can be set, and the beat label can be added. For details, please refer to the model generation beat label in the "VEM-Token beat capture and alignment model construction method, 202511249168.0". The user of this patent can also design the beat division method by himself. DETAILED DESCRIPTION ​

[0191] The purpose and intent of the present invention is achieved with the following specific embodiments. It is important to note that each specific embodiment has a specific use and industrial applicability. Therefore, the following embodiments do not include all of the system features and implementation steps of the present invention, and the accompanying drawings are only one of the embodiments, and do not constitute a limitation on the present invention. The description of the claims of the present invention is the core summary of the invention.

[0192] The specific embodiments of the present invention are as follows:

[0193] Innovative VEM-multimodal emotional skin system

[0194] Diagram explanation

[0195] This embodiment mainly includes, but is not limited to, the following parts of the schematic diagram, which are as detailed in the accompanying drawings Figures 1 to 11 .

[0196] System implementation explanation

[0197] This embodiment mainly includes system implementation 1 to system implementation 12. Part of them is the basic system of the present application, and the other part is the optional combination item according to the actual application. The following is explained:

[0198] The logical line in this specification refers to a natural paragraph. The "further" in the logical line indicates that this logical line is the next optional item of the previous logical line, and the "preferably" indicates that the logical line is not an optional item, but an optional item in the optimization of some applications.

[0199] Unless otherwise specified, these systems are not completely required.

[0200] Unless otherwise specified, the order is also not required.

[0201] Unless otherwise specified, the selection of materials and the selection of design parameters are not required.

[0202] According to the specific task requirements, the patent implementer makes the preferred and further selection.

[0203] 1. VEM-multimodal emotional skin system:

[0204] The present invention as a VEM-multimodal emotional skin system includes but is not limited to the following main parts:

[0205] The VEM-multimodal emotional skin system includes but is not limited to electromagnets, soft magnetic colloids, and multimodal mutual inductance systems.

[0206] The electromagnet includes but is not limited to a soft magnetic core and an excitation coil, and an excitation current passes through the excitation coil to make the electromagnet generate a magnetic pole. Two or more electromagnets are arranged at a distance from each other to form an electromagnet array, and the distance is filled with a soft magnetic colloid to form a mutual inductance magnetic circuit, thereby forming the emotional skin.

[0207] The soft magnetic colloid is formed by mixing soft magnetic material particles and elastic colloid, and has a resilience to restore to its original state in a static state.

[0208] The multi-modal mutual inductance system receives and executes expression instructions, provides excitation current for the electromagnet, and generates magnetic creep on the surface of the emotional skin to simulate expressions.

[0209] The multi-modal mutual inductance system also has a touch sensing function. When an external object applies a touch action on the surface, the touch action is converted and calculated to form a touch signal.

[0210] The basic idea of this design is to use dynamic design of electromagnetic field to generate magnetic creep on the simulation surface to simulate expressions. The magnetic core of the electromagnet is made of soft magnetic material with high magnetic permeability, such as permalloy, which has a saturation magnetic induction of about 0.8T (Tesla), a coercive force of less than 0.2%, and a relative permeability of more than 100000. In the soft magnetic colloid, the soft magnetic powder can be carbonyl iron powder, and the colloid material can be silicone. At this time, the relative permeability of carbonyl iron powder is between 5-100, and the effective relative permeability of soft magnetic colloid is 15 to 30 when the volume filling rate of carbonyl iron powder is between 30% and 60%. This value is much larger than the relative permeability of air, which is 1. In this case, the soft magnetic colloid can take full advantage of its high magnetic permeability, and the electromagnet can generate a very obvious magnetic creep effect in the soft magnetic colloid.

[0211] It should be noted that when designing the attractive and repulsive forces between the poles of the electromagnet, the resilience of the soft magnetic colloid needs to be overcome. For example, Figure 2 As indicated in the above example, assuming that the static distance between the front poles of two parallel electromagnets is S0, the resilience of the soft magnetic colloid at this distance is f0, and the attractive or repulsive force generated by the front poles of the parallel electromagnets at the moment of energization is F0. To generate magnetic creep, F0 must be much greater than f0. In addition, it should be noted that in the case of attractive force, the attractive force will increase sharply as the distance decreases. In order to optimize the design, the excitation current of the excitation coil can be gradually reduced by pulse width modulation (PWM) to gradually reduce the attractive force, so that the magnetic creep is closer to the normal motion of the human face. In the case of repulsive force, gradually increasing the pulse width is adopted to adapt to the rapid decay of repulsive force due to the increase in distance.

[0212] It should be noted that the sensing principle of the touch sensor includes the following options:

[0213] (1) Magnetic induction sensing.

[0214] (2) Capacitive sensing.

[0215] (3) Resistance sensing.

[0216] Here refer to Figure 1 and Figure 8 .

[0217] 2. The multi-modal mutual sensing system includes an expression loop:

[0218] On the basis of the foregoing scheme, the mutual sensing system of the present application includes an expression loop, specifically including but not limited to one or a combination of the following structures:

[0219] The expression loop includes but is not limited to an expression circuit and an expression magnetic circuit. The expression circuit executes expression instructions, provides excitation current for the electromagnet array and generates magnetic poles. The magnetic poles form the expression magnetic circuit with the soft magnetic colloid. The magnetic attraction and repulsion force between the magnetic poles overcomes the resilience of the soft magnetic colloid, and a dynamic change process of the magnetic circuit is generated between the magnetic poles and the adjacent soft magnetic colloid, resulting in magnetic creep on the emotional skin and epidermis to simulate dynamic expressions.

[0220] The expression circuit includes but is not limited to: a circuit for series, parallel and conversion of two or more excitation coils, a circuit for current polarity conversion when each excitation coil is powered, a circuit powered by constant current source and constant voltage source, and a logic module controlled by expression instructions.

[0221] It should be noted here that in order to ensure that the attractive and repulsive force is much greater than the resilience, sufficient current-driven redundancy is required in the design of electromagnets and excitation current, and the viscosity of the soft magnetic colloid should be reduced as much as possible in the design of the resilience of the soft magnetic colloid. In order to meet the needs of expressions on the epidermis, different numbers of excitation coils or different sizes of excitation currents are used for electromagnets at different positions, and different pulse widths of PWM can also be used to ensure the attractive and repulsive force of electromagnets at different positions. In addition, the above requirements can be realized in the logic module and expression instructions.

[0222] Here refer to Figures 1 to 11 .

[0223] 3. Expression instructions:

[0224] On the basis of the foregoing scheme, the expression instructions of the present application include but are not limited to one or a combination of the following:

[0225] The expression instruction is generated based on an expression function FE(x, y, z, t, i, p, v, τ), wherein x, y, and z in the vector (x, y, z, t, i, p, v, τ) are respectively coordinates of three-dimensional space of an electromagnet, t is a time value coordinate of the fourth dimension, i, p, v, and τ are respectively a current value, a current polarity, an electromagnet type, and a step time in the electromagnet coil, and the step time is a time length for transition from a previous expression to a next expression in a dynamic expression, and the step time is greater than the time value.

[0226] The expression instruction generates a magnetic peristalsis sequence, including but not limited to driving the electromagnet in the three-dimensional space coordinates in a step sequence according to a time value sequence, so that magnetic poles of adjacent electromagnets are attracted and repelled according to the step sequence, and finally magnetic peristalsis is generated on the surface of the emotional skin to simulate a final expression.

[0227] Preferably, the expression function supports a VEM-fe expression function model, and specifically includes but is not limited to: synthesizing more than one expression vector representing an emotional input into a synthesized expression as an output, aligning expression vectors in a current beat, eliminating expression conflicts between two or more expression vectors, and transitioning from a synthesized expression of a current beat to a synthesized expression of a next beat, wherein a step uses a beat model.

[0228] Preferably, the driving implementation of the expression function includes but is not limited to direct current pulse and pulse width modulation (PWM).

[0229] Here, the expression instruction is used to simulate an expression by magnetic peristalsis according to a beat, the expression instruction is constituted by an expression function, and in the expression function FE(x, y, z, t, i, p, v, τ), the state of any point on the surface is determined by the expression function vector (x, y, z, t, i, p, v, τ). It should be noted that the 4D referred to in the present application refers to three-dimensional coordinates (x, y, z) of the state and a one-dimensional time value (t), which are combined into a four-dimensional space state, i.e., a 4D expression state. At the same point in the three-dimensional space on the surface, the state of the point can change continuously with the passage of time. In addition, this one-dimensional time change is measured according to the shortest time, i.e., a beat, so the change of the point on the surface, i.e., peristalsis, is also peristaltic according to the beat, resulting in a beat peristalsis process. The beat peristalsis mode is determined according to (i, p, v, τ).

[0230] It should be noted that the three-dimensional point coordinates (x, y, z) on the surface referred to in the present application are supported by electromagnets below the surface, and the expression function vector acts by causing beat peristalsis through the attractive or repulsive force of the electromagnet at the point.

[0231] Here, please refer to Figure 11 .

[0232] It should be noted that:

[0233] When the excitation current needs to be reduced, the current source can adopt pulse width modulation (PWM) mode, and can also adopt variable pulse width modulation, for example, at the beginning of the current source, the pulse width is increased so as to overcome the distance between the magnetic poles at the beginning of driving the magnetic poles, and after the distance between the magnetic poles becomes small, a smaller excitation current is adopted to maintain the smaller distance between the magnetic poles.

[0234] The VEM-fe expression function model here can preferably be compatible with the method of constructing a robot expression function of a VEM-Token world model, CN120951102A, as described in the background art.

[0235] The "preferably" logic lines here are optional structures, and the user of the present application selects execution according to respective applications. In the present embodiment, all optional items are selected.

[0236] 4. Touch loop:

[0237] On the basis of the foregoing scheme, the present application includes, but is not limited to, one or a combination of the following in terms of the touch loop:

[0238] The touch loop includes, but is not limited to, a sensing and touch circuit one that multiplexes a touch magnetic circuit. When an external touch action is applied to the skin, the touch pressure overcomes the resilience of the soft magnetic colloid, the soft magnetic colloid is locally deformed due to the touch point and the touch path under the touch pressure, a touch magnetic circuit is formed by disturbing the expression magnetic circuit, a dynamic change process of the magnetic circuit is caused between the soft magnetic colloid and the adjacent magnetic pole, the sensing and touch circuit one captures the dynamic change process of the magnetic circuit to generate electromagnetic induction in the excitation coil, and a touch signal is output by a multi-modal mutual inductance system sensing calculation.

[0239] Preferably, the touch loop includes, but is not limited to, a sensing and touch circuit two that multiplexes a touch coil. When an external finger touch action is applied to the skin, the excitation coil of the electromagnet at the touch point adopts a capacitor to sense the touch voltage by multiplexing the touch coil, the sensing and touch circuit two captures the change process of the excitation coil voltage, and a touch signal is output by a multi-modal mutual inductance system sensing calculation.

[0240] Preferably, the touch loop includes, but is not limited to, a sensing and touch circuit three that multiplexes a thin film resistance array. A flexible touch sensing thin film resistance array is arranged in the emotional skin, the sensing and touch circuit three connects the thin film resistance array to collect each resistance value, when an external touch action is applied to the skin, the resistance value of the thin film resistance at the touch point changes, and a touch signal is output by a multi-modal mutual inductance system sensing calculation.

[0241] It should be noted that the method of the touch circuit here includes at least the following three options, which are aimed at solving the method of touch sensing on the skin, and the user can adopt one of these methods or a combination of multiple methods.

[0242] (1) Magnetic induction sensing method.

[0243] When the magnetic pole and the soft magnetic colloid are in a stable state, the magnetic field under the skin is also in a stable state, and at this time when the external finger touches the skin, due to the touch pressure and the movement of the finger on the skin, the local morphology of the soft magnetic colloid is changed, thereby changing the magnetic circuit, and a weak voltage change is generated in the excitation coil due to magnetic induction. At this time, the voltage change is detected through the filter to realize the sensing of the touch action.

[0244] (2) Capacitive sensing method.

[0245] Since the skin under the skin contains an electromagnet array, and on the electromagnet, an excitation coil is contained, through the design of the filter circuit and the detection circuit, the capacitive induction generated on the excitation coil under the skin position when the finger touches can be realized. Detecting the change of this capacitance realizes the sensing of the touch action.

[0246] (3) Resistance sensing method.

[0247] A flexible thin film circuit matrix device is arranged under the skin, and a corresponding detection circuit is used to realize touch sensing.

[0248] Here, reference is made to Figure 7 and Figure 8 .

[0249] 5. Touch circuit:

[0250] On the basis of the foregoing scheme, the present application in the touch circuit includes but is not limited to one or a combination of the following:

[0251] The touch circuit includes but is not limited to a low-pass filter and a signal processor, the low-pass filter has a cutoff frequency lower than the frequency of the excitation current, and the signal processor is connected to the loop of the excitation coil. In a multiplexed touch magnetic circuit algorithm including but not limited to a multi-modal mutual inductance system, the signal amount of the touch signal is detected and calculated, and when the signal amount of the touch signal is greater than a set minimum value, it is determined and recorded that this touch is an effective touch signal. The low-pass filter is used to isolate the excitation current, and the signal processor is used to amplify and shape the touch signal.

[0252] Preferably, the touch circuit two adopts a high-pass filter and a signal processor two connected with the excitation coil, the high-pass filter has a cut-off frequency higher than the frequency of the excitation current, in a multi-modal mutual inductance system including but not limited to a multiplex touch coil algorithm, the signal amount of the touch signal is detected and calculated, when the signal amount of the touch signal is greater than a set minimum value, the touch signal is determined and recorded as a valid touch signal, wherein the high-pass filter is used to eliminate interference signals, and the signal processor two is used to amplify and shape the touch signal.

[0253] Preferably, the touch circuit three adopts a signal processor three of a thin film resistance array, in a multi-modal mutual inductance system including but not limited to a thin film resistance array algorithm, the signal amount of the touch signal is detected and calculated, when the signal amount of the touch signal is greater than a set minimum value, the touch signal is determined and recorded as a valid touch signal.

[0254] Herein, refer to Figure 7 and Figure 8 .

[0255] It should be noted that:

[0256] For the touch circuit one and the touch circuit two, Figure 8 In actual current design, the focus is on the filter, C1 and C2 in the figure are only a principle display, and a more practical filter circuit can be designed by a person skilled in the art according to the principle, and the key prompt point is that the touch circuit one needs to filter the pulse part of the excitation current, for example, filter the PWM fluctuation, and here, since the signal frequency of the PWM is high relative to the signal frequency of the finger touch, it is a high-frequency signal, therefore, it is suggested to adopt a low-pass filter. In the touch circuit two, based on the finger touch to the capacitive induction, the touch signal frequency should be selected to be relatively high at this time, relative to the excitation driving current, the filter here can select a high-pass filter. In the touch circuit three, the circuit is usually matched with the thin film resistance array.

[0257] 6. Touch signal:

[0258] On the basis of the foregoing scheme, in terms of touch signals, the present application includes but is not limited to one or a combination of the following:

[0259] The touch signal at least includes a touch function TM (x, y, z, t), wherein x, y, z in the vector (x, y, z, t) are respectively the coordinates of the three-dimensional space of the electromagnet or the thin film resistance array, and t is a fourth-dimensional time value coordinate.

[0260] A group of touch signals includes but is not limited to a collection of all touch signals within a period of time.

[0261] Preferably, the touch signal further includes but is not limited to an increased touch intensity, and at this time, the touch function includes TM (x, y, z, t, h), wherein h is the touch intensity.

[0262] Preferably, the touch signal also includes, but is not limited to, instructions submitted to the multi-modal mutual inductance system and subsequent systems to process the generated set expression instructions and subsequent control instructions.

[0263] The multi-modal mutual inductance system cyclically scans the electromagnet array or the thin film resistance array, detects and records the coordinates and time values of the electromagnet or the thin film resistance array, and generates the touch signal.

[0264] It should be noted that in the touch signal, the time value coordinate t of the fourth dimension is similar to the expression instruction, and the touch signal is also collected according to the beat. Only the beat here is different from the beat in the expression instruction, and the beat frequency is faster than the beat frequency in the expression instruction, so as to improve the sampling rate of the touch signal.

[0265] The subsequent instructions triggered by the touch signal are to make the skin produce corresponding reactions according to the touch signal, such as simulating a smile after touching the skin, or making the skin red to simulate a red face.

[0266] 7. Electromagnet:

[0267] On the basis of the foregoing scheme, the present application includes, but is not limited to, one or a combination of the following:

[0268] Bar-shaped sheet electromagnet: The electromagnet core is made of bar-shaped sheet soft magnetic material, and a group of excitation coils are wound on the core to generate magnetic poles at both ends of the bar-shaped sheet electromagnet. Two or more bar-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by a soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and is installed on the inside of the skin to form emotional skin.

[0269] Preferably, the cross-shaped sheet electromagnet is made of cross-shaped sheet soft magnetic material, and excitation coils are wound on the four arms of the cross-shaped sheet to generate magnetic poles at the four arms of the cross-shaped sheet. Two or more cross-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by a soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and is installed on the inside of the skin to form emotional skin.

[0270] Preferably, the parallel electromagnet is made of bar-shaped cylindrical or rectangular soft magnetic material, and a group of excitation coils are wound on the bar-shaped sheet to generate magnetic poles at both ends of the bar-shaped sheet electromagnet. Two or more parallel electromagnets constitute an electromagnet array and are filled and wrapped by a soft magnetic colloid to form a mutual inductance magnetic circuit, which is perpendicular to the skin and is installed on the inside of the skin to form emotional skin.

[0271] Preferably, the concave-convex electromagnet is installed on the back plate side of the emotional skin, and is used to attract and repel the parallel electromagnet, the strip-shaped electromagnet and the cross-shaped electromagnet. The concave-convex electromagnet includes a group of excitation coils, generates magnetic poles at two ends of the concave-convex electromagnet, fills and wraps the soft magnetic colloid to form a mutual inductance magnetic circuit, and constitutes the emotional skin.

[0272] Regarding the preferred option, it should be noted that different types of electromagnets are used at different positions of the skin. Here, not only the size of the electromagnet is considered, for example, the skin at the arm and eyelid is thin, and the strip-shaped electromagnet and the cross-shaped electromagnet can be used. At the mouth and cheekbone, the skin is thick, and the parallel electromagnet can be used to simulate a smile, and the concave-convex electromagnet can be used to simulate dimples.

[0273] 8. The multi-modal mutual inductance system supports the beat model:

[0274] On the basis of the foregoing scheme, the multi-modal mutual inductance system supports the beat model, including but not limited to one or a combination of the following:

[0275] In the touch function TM (x, y, z, t), the time value coordinates are aligned with the start beat and the end beat of the beat model through start point fine adjustment and end point fine adjustment, so that the collection of touch signals is generated according to the beat model, thereby generating a beat feeling.

[0276] Preferably, in the expression function FE (x, y, z, t, i, p, v, tau), the time value coordinates are aligned with the start beat and the end beat of the beat model through start point fine adjustment and end point fine adjustment, so that the effect of the expression function is generated according to the beat model, thereby generating a beat feeling.

[0277] Preferably, the beat model includes but is not limited to the beat model in the compatible VEM-Token vocal emotion multi-modal model and the beat definition in music.

[0278] As Figure 9 shown, it should be emphasized that the support and compatibility of the beat model here is a kind of equal or including relationship from the set. The reason why there is an including relationship is that the multi-modal mutual inductance system of the present application needs to have a set element that is included by itself but not by the beat model, that is, those features that are not necessarily possessed in the original VEM-Token vocal emotion multi-modal model.

[0279] 9. The multi-modal mutual inductance system supports the VEM-sync emotion synchronization model:

[0280] On the basis of the foregoing scheme, the multi-modal mutual inductance system supports the VEM-sync emotion synchronization model, including but not limited to one or a combination of the following:

[0281] The emotion synchronization model function is set as VEM-sync, wherein the VEM-sync includes but is not limited to a VEM-sync vector sequence corresponding to and aligned with the VEM-Token sequence, and the VEM-sync vector includes a synchronization content and an emotion synchronization pointer pointing to a corresponding VEM-Token beat.

[0282] In the touch function TM (x, y, z, t, h), the time value coordinates thereof are aligned with the emotion synchronization pointers in the VEM-sync vector sequence through the start point fine adjustment and the end point fine adjustment, so that the collection of the touch signal is generated according to the emotion synchronization model, thereby generating the emotion feeling.

[0283] Preferably, in the expression function FE (x, y, z, t, i, p, v, τ), the time value coordinates thereof are aligned with the emotion synchronization pointers in the VEM-sync vector sequence through the start point fine adjustment and the end point fine adjustment, so that the action of the expression function is generated according to the emotion synchronization model, thereby generating the emotion feeling.

[0284] The emotion synchronization model includes but is not limited to a VEM-sync emotion synchronization model in a method of compatible VEM-Token emotion synchronization function hierarchical fusion and an emotion definition in social interaction.

[0285] As shown in Figure 10 It is emphasized that the support and compatibility of the VEM-sync emotion synchronization model are equal to or include the beat model in terms of set, and the reason why the relationship is included is that the multi-modal mutual sensing system needs to have a set element that the VEM-sync emotion synchronization model does not include, that is, those features that the original VEM-sync emotion synchronization model does not necessarily have.

[0286] 10. The multi-modal mutual sensing system supports the VEM-fe expression function model:

[0287] On the basis of the foregoing scheme, the VEM-fe expression function model is supported on the multi-modal mutual sensing system, and specifically includes but is not limited to one or a combination of the following:

[0288] The expression alignment includes but is not limited to two or more expression vectors or a synthetic expression, and the start point alignment and the end point alignment are performed in the beat.

[0289] In the touch function TM (x, y, z, t, h), the time value coordinates thereof are aligned with the emotion synchronization pointers in the VEM-sync vector sequence through the start point fine adjustment and the end point fine adjustment, so that the collection of the touch signal is generated according to the emotion synchronization model, thereby generating the emotion feeling.

[0290] Preferably, in the expression function FE (x, y, z, t, i, p, v, tau), its time value coordinates are aligned with the expression vector in the VEM-fe expression function model through the start fine-tuning and the end fine-tuning, so that the effect of the expression function is generated in accordance with the expression synchronization, thereby generating the emotional simulation effect.

[0291] The VEM-fe expression function model includes but is not limited to the VEM-fe expression function model in the method of constructing a robot expression function compatible with the VEM-Token world model and the expression definition in social interaction.

[0292] As shown in Figure 11 It is emphasized that the support and compatibility of the VEM-fe expression function model here is an equal or including relationship from the set, and the reason for the including relationship is that the multi-modal mutual sensing system of the application needs to have its own including set elements, but the VEM-fe expression function model does not necessarily include them, that is, those features that the original VEM-fe expression function model does not necessarily have.

[0293] 11. Multi-modal mutual sensing input system:

[0294] On the basis of the foregoing scheme, the multi-modal mutual sensing input system of the application specifically includes but is not limited to one or a combination of the following:

[0295] The multi-modal mutual sensing input function MMSI (x, y, z, t, s1, fx) is set, wherein the vectors x, y, and z are the position coordinates of the input sensors, t is the time value coordinate or the beat number, s1 is the input sensor type, and fx is the input value sequence.

[0296] The input sensor type includes but is not limited to sound sensing, temperature sensing, movement sensing, and visual sensing, including one or a combination thereof.

[0297] Preferably, the subcutaneous arrangement includes platinum resistance wire to transmit the temperature in the emotional sensing skin.

[0298] Regarding the input sensor, it is set according to the application scenario of the application. For example, when the application is applied to a humanoid robot, it is possible to detect the temperature of the skin, it is also possible to detect the proximity of the external human body or object, and it is also possible to install "eyes" for the application system of the application. These auxiliary requirements constitute the input sensor of the application. Those skilled in the art should understand that these sensors can be installed and connected to the multi-modal mutual sensing system as a conventional application, and applied through the unified management of the multi-modal mutual sensing system.

[0299] MMSI is a multimodal mutual sensing input function (MMSI: Multimodal mutual sensing input).

[0300] 12. A multimodal mutual sensing output system:

[0301] On the basis of the foregoing solutions, the present application is on a multimodal mutual sensing output system, and specifically includes but is not limited to one or a combination of the following:

[0302] The multimodal mutual sensing output function MMIO(x, y, z, t, s2, fx) is set, wherein the vectors x, y, and z are the position coordinates of the output sensors, t is the time value coordinate or the beat number, s2 is the output actuator type, and fx is the output value sequence.

[0303] The output actuator type includes but is not limited to a skin color change actuator, a local motion actuator, and a whole body motion actuator, including one or a combination thereof.

[0304] Preferably, the skin includes but is not limited to a material that is elastic, wear-resistant, cosmetically acceptable, and washable, and is bonded with a soft magnetic colloid.

[0305] Preferably, the skin includes but is not limited to an electric heating wire and a temperature control system to adjust the temperature of the skin.

[0306] Regarding the output system, it is set according to the application scenarios of the present application. For example, when the present application is applied to a humanoid robot, in order to increase the emotional mutual sensing function, an emotional response can be set for the application of the skin, such as skin contraction, face skin redness (i.e., blushing response), or even body retreat. Therefore, the present application can set some corresponding actuators. Those skilled in the art should understand that these actuators can be installed and connected to the multimodal mutual sensing system as a conventional application, and are applied through the unified management of the multimodal mutual sensing system.

[0307] MMIO is a multimodal mutual inductance output function (MMIO: Multimodal mutual inductance output).

Claims

1. A VEM - Multimodal Emotional Skin System, characterized in that, Comprise: VEM - multi-modal emotional skin system, comprising electromagnet, soft magnetic colloid and multi-modal mutual inductance system; The electromagnet comprises a soft magnetic core and an excitation coil, and an excitation current passes through the excitation coil to generate a magnetic pole of the electromagnet. Two or more electromagnets are arranged at a distance from each other to form an electromagnet array, and the distance is filled with soft magnetic colloid to form a mutual inductance magnetic circuit, thereby forming an emotional skin; The soft magnetic colloid is made of soft magnetic material particles and elastic colloid, and has a restoring force to restore to its original shape when static; The multi-modal mutual inductance system receives and executes expression instructions, provides excitation current for the electromagnet, and generates magnetic creep on the surface of the emotional skin to simulate expression; The multi-modal mutual inductance system also has a touch sensing function, which converts and calculates touch signals when external objects apply touch actions on the surface.

2. The system according to claim 1, characterized in that The multi-modal mutual inductance system comprises an expression loop, wherein: The expression loop comprises an expression circuit and an expression magnetic circuit. The expression circuit executes expression instructions, provides excitation current for the electromagnet array and generates magnetic poles. The magnetic poles and the soft magnetic colloid form an expression magnetic circuit. Under the action of magnetic attraction and repulsion between the magnetic poles, the soft magnetic colloid generates a dynamic change process of the magnetic circuit between the magnetic poles and the adjacent soft magnetic colloid, resulting in magnetic creep on the emotional skin and the surface to simulate dynamic expression; The expression circuit comprises: a circuit for series, parallel and conversion of two or more excitation coils, a circuit for current polarity conversion when each excitation coil is powered, a circuit powered by constant current source and constant voltage source, and a logic module for accepting expression instructions for control.

3. The system according to claim 2, characterized in that, The expression instructions comprise: The expression instructions are generated based on an expression function FE(x, y, z, t, i, p, v, τ), wherein x, y, z in the vector (x, y, z, t, i, p, v, τ) are respectively the coordinates of the three-dimensional space of the electromagnet, t is the fourth-dimensional time value coordinate, i, p, v, τ are respectively the current value, current polarity, electromagnet type and step time running in the excitation coil, and the step time is the time length of the transition from the previous step expression to the next step expression in the dynamic expression, and the step time is greater than the time value; The expression instruction generates a magnetic creep sequence, which comprises driving the electromagnets in the three-dimensional space coordinates in a step-by-step sequence according to the time value sequence, so that the magnetic poles of adjacent electromagnets are attracted and repelled in a step-by-step manner, and finally generate magnetic creep on the surface of the emotional skin to simulate the final expression; and / or, The expression function supports VEM-fe expression function model, which specifically comprises: synthesizing one or more expression vectors representing emotional input into output, aligning the expression vectors in the current beat, eliminating expression conflicts between two or more expression vectors, and transitioning from the synthesized expression of the current beat to the synthesized expression of the next beat, wherein the step length adopts a beat model; and / or, The driving implementation of the expression function comprises direct current pulse and pulse width modulation (PWM).

4. The system according to claim 3, characterized in that The multi-modal mutual inductance system comprises a touch loop, wherein: The touch circuit includes a sensing and touch circuit one multiplexing a touch magnetic circuit. When an external touch action is applied to the skin, the touch pressure overcomes the resilience of the soft magnetic colloid, the soft magnetic colloid is locally deformed by the touch point and the touch path under the touch pressure, and a touch magnetic circuit is formed by disturbing the expression magnetic circuit, resulting in a dynamic change process of the magnetic circuit between the soft magnetic colloid and the adjacent magnetic pole. The touch circuit one captures the dynamic change process of the magnetic circuit by magnetic circuit multiplexing sensing to generate electromagnetic induction in the excitation coil, and outputs a touch signal calculated by a multi-modal mutual inductance system sensing; and / or, The touch circuit includes a sensing and touch circuit two multiplexing a touch coil. When an external touch action is applied to the skin, the excitation coil of the electromagnet at the touch point is in a capacitor mode, and the touch voltage is sensed by the multiplexing touch coil. The touch circuit two captures the change process of the excitation coil voltage to output a touch signal calculated by a multi-modal mutual inductance system sensing; and / or, The touch circuit includes a sensing and touch circuit three of a thin film resistance array. A flexible touch sensing thin film resistance array is arranged in the emotional skin, and the touch circuit three connects the thin film resistance array to collect each resistance value. When an external touch action is applied to the skin, the resistance value of the thin film resistance at the touch point changes, and a touch signal is calculated by a multi-modal mutual inductance system sensing.

5. The system according to claim 4, characterized in that The touch circuit further includes: The touch circuit one includes a low-pass filter and a signal processor one. The low-pass filter has a cutoff frequency lower than the frequency of the excitation current, and the signal processor one is connected to the loop of the excitation coil. In the multiplexing touch magnetic circuit algorithm included in the multi-modal mutual inductance system, the signal amount of the touch signal is detected and calculated. When the signal amount of the touch signal is greater than a set minimum value, it is determined and recorded that this touch is an effective touch signal. The low-pass filter is used to isolate the excitation current, and the signal processor one is used to amplify and shape the touch signal; and / or, The touch circuit two uses a high-pass filter and a signal processor two connected to the excitation coil. The high-pass filter has a cutoff frequency higher than the frequency of the excitation current. In the multiplexing touch coil algorithm included in the multi-modal mutual inductance system, the signal amount of the touch signal is detected and calculated. When the signal amount of the touch signal is greater than a set minimum value, it is determined and recorded that the touch signal is an effective touch signal. The high-pass filter is used to eliminate interference signals, and the signal processor two is used to amplify and shape the touch signal; and / or, The touch circuit three uses a signal processor three of a thin film resistance array. In the thin film resistance array algorithm included in the multi-modal mutual inductance system, the signal amount of the touch signal is detected and calculated. When the signal amount of the touch signal is greater than a set minimum value, it is determined and recorded that the touch signal is an effective touch signal.

6. The system according to claim 5, characterized in that The touch signal specifically includes: The touch signal at least includes a touch function TM(x, y, z, t), wherein x, y, z in the vector (x, y, z, t) are respectively the coordinates of the three-dimensional space of the electromagnet or the thin film resistance array, and t is the fourth-dimensional time value coordinate; A group of touch signals includes a collection of all touch signals in a period of time; and / or, The touch signal also includes an increased touch intensity, and the touch function includes TM(x, y, z, t, h), where h is the touch intensity; and / or, The touch signal also includes instructions submitted to the multi-modal mutual inductance system and subsequent systems to process the generated expression instructions and subsequent control instructions; The multi-modal mutual inductance system cyclically scans the electromagnet array or the thin film resistance array, detects and calculates the coordinates, time values and effective touch signals of the electromagnet or the thin film resistance array, and generates the touch signal.

7. The system according to claim 6, characterized in that The electromagnet types include: Bar-shaped sheet electromagnet, which is made of bar-shaped sheet soft magnetic material to make the electromagnet core, and a group of excitation coils wound on the core to generate magnetic poles at both ends of the bar-shaped sheet electromagnet, and two or more bar-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and installed on the inside of the skin to form emotional skin; Cross-shaped sheet electromagnet, which is made of cross-shaped sheet soft magnetic material to make the electromagnet core, and excitation coils are wound on the four arms of the cross-shaped sheet to generate magnetic poles at the four arms of the cross-shaped sheet, and two or more cross-shaped sheet electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is parallel to the skin and installed on the inside of the skin to form emotional skin; and / or, Parallel electromagnet, which is made of bar-shaped cylindrical or rectangular soft magnetic material to make the electromagnet core, and a group of excitation coils are wound on the bar-shaped sheet to generate magnetic poles at both ends of the bar-shaped sheet electromagnet, and two or more parallel electromagnets constitute an electromagnet array and are filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit, which is perpendicular to the skin and installed on the inside of the skin to form emotional skin; and / or, Rough and smooth electromagnet, which is installed on the back plate side of the emotional skin and used to attract and repel parallel electromagnets, bar-shaped sheet electromagnets and / or cross-shaped sheet electromagnets, and a group of excitation coils are included on the rough and smooth electromagnet to generate magnetic poles at both ends of the electromagnet, and the electromagnet is filled and wrapped by soft magnetic colloid to form a mutual inductance magnetic circuit to form emotional skin.

8. The system according to claim 6, characterized in that The multi-modal mutual inductance system supports the beat model, specifically including: In the touch function TM(x, y, z, t, h), the time value coordinates are aligned with the start beat and end beat of the beat model through start point fine tuning and end point fine tuning, so that the collection of touch signals is generated according to the beat model, thereby generating a sense of beat; and / or, In the expression function FE(x, y, z, t, i, p, v, τ), the time value coordinates are aligned with the start beat and end beat of the beat model through start point fine tuning and end point fine tuning, so that the effect of the expression function is generated according to the beat model, thereby generating a sense of beat; and / or, The beat model is compatible with the beat model in the VEM-Token vocal emotion multi-modal model and the beat definition in music.

9. The system according to claim 6, characterized in that The multi-modal mutual inductance system supports the emotion synchronization model, specifically including: The emotion synchronization model function is set as VEM-sync, where VEM-sync includes a sequence of VEM-sync vectors corresponding to and aligned with the VEM-Token sequence, and the VEM-sync vector includes synchronization content and an emotion synchronization pointer pointing to the corresponding VEM-Token beat; In the touch function TM(x, y, z, t, h), the time value coordinates are aligned with the emotion synchronization pointer in the VEM-sync vector sequence through the start point fine-tuning and the end point fine-tuning, so that the collection of the touch signal is generated according to the emotion synchronization model, thereby generating the emotion feeling; and / or, In the expression function FE(x, y, z, t, i, p, v, τ), the time value coordinates are aligned with the expression vector in the VEM-fe expression function model through the start point fine-tuning and the end point fine-tuning, so that the action of the expression function is generated according to the expression synchronization, thereby generating the emotion simulation. The emotion synchronization model is compatible with the VEM-sync emotion synchronization model in the method of VEM-Token emotion synchronization function hierarchical fusion and the emotion definition in social interaction.

10. The system according to claim 6, characterized in that The multi-modal mutual sensing system supports the VEM-fe expression function model, specifically including: The expression alignment includes two or more expression vectors or a composite expression, and the start point alignment and the end point alignment are performed on the beat; In the touch function TM(x, y, z, t, h), the time value coordinates are aligned with the emotion synchronization pointer in the VEM-sync vector sequence through the start point fine-tuning and the end point fine-tuning, so that the collection of the touch signal is generated according to the emotion synchronization model, thereby generating the emotion feeling; and / or, In the expression function FE(x, y, z, t, i, p, v, τ), the time value coordinates are aligned with the expression vector in the VEM-fe expression function model through the start point fine-tuning and the end point fine-tuning, so that the action of the expression function is generated according to the expression synchronization, thereby generating the emotion simulation. The VEM-fe expression function model is compatible with the VEM-fe expression function model in the method of constructing the robot expression function of the VEM-Token world model and the expression definition in social interaction.

11. The system according to any of claims 6 to 10, characterized in that The multi-modal mutual sensing input system includes: The multi-modal mutual sensing input function is set as MMSI(x, y, z, t, s1, fx), where the vectors x, y, and z are the position coordinates of the input sensors, t is the time value coordinate or the beat number, s1 is the input sensor type, and fx is the input value sequence; The input sensor type includes sound sensing, temperature sensing, movement sensing, and visual sensing, including one or a combination thereof; and / or, The subcutaneous arrangement includes platinum resistance wire to transmit the temperature in the emotional sensing skin.

12. The system according to any one of claims 6 to 10, characterized in that, The multi-modal mutual sensing output system includes: The multi-modal mutual sensing output function is set as MMIO(x, y, z, t, s2, fx), where the vectors x, y, and z are the position coordinates of the output sensors, t is the time value coordinate or the beat number, s2 is the output actuator type, and fx is the output value sequence; The output actuator type includes a skin color changing actuator, a local motion actuator, and a whole body motion actuator, including one or a combination thereof; and / or, The skin is composed of a material that is elastic, wear-resistant, makeupable, and washable, and is bonded with a soft magnetic colloid; The skin includes an electric heating wire and a temperature control system to adjust the temperature of the skin.

Images