Teleoperation control method and system for cloth, medium and terminal

Abstract

The invention provides a teleoperation control method and system for cloth, a medium and a terminal. The method comprises the following steps: acquiring image information and spectral information of the cloth; acquiring a cloth type based on the image information, and acquiring mechanical parameters of the cloth based on the spectral information; obtaining the current contact force distribution of teleoperation, and obtaining a multi-mode tactile feedback signal based on the cloth type, the mechanical parameters and the current contact force distribution; and controlling the teleoperation of the cloth based on the multi-mode tactile feedback signal. According to the application, the material and mechanical parameters of the cloth can be identified in real time, the spatial distribution state of the contact force in the teleoperation process is sensed, and multi-mode sensing is performed according to the material and mechanical parameters of the cloth and the spatial distribution state of the contact force so as to control the teleoperation action of the cloth.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of cloth, and relates to a cloth remote operation control method, system, medium and terminal. BACKGROUND

[0002] With the rapid development of robotics, dexterous hands are increasingly widely used in industrial automation, service robots, medical assistance, and special operations. As an important part of the robot end effector, one of the core tasks of the dexterous hand is to achieve precise grasping and operation of flexible objects (such as cloth, fabric, etc.). However, due to the high flexibility, deformability and complex physical properties of cloth, traditional rigid clamping methods are difficult to meet the operation requirements, resulting in poor grasping stability and low operation efficiency.

[0003] In actual application scenarios, cloth operation often requires high flexibility and perception ability, for example, in clothing manufacturing, warehouse logistics or home services, robots need to complete complex tasks such as grasping clothes, laying cloth, folding fabric, etc. In order to achieve these functions, the dexterous hand not only needs to have multi-degree-of-freedom motion ability, but also needs to combine advanced perception systems such as vision, force sensation, and tactile sensors to obtain the state information of the cloth, and through efficient control algorithms for real-time feedback and adjustment.

[0004] SUMMARY

[0005] The purpose of the present application is to provide a cloth remote operation control method, system, medium and terminal, which controls the remote operation of cloth based on multi-modal perception information in real time.

[0006] In a first aspect, the present application provides a cloth remote operation control method, the method comprising:

[0007] obtaining image information and spectral information of the cloth;

[0008] obtaining the cloth type based on the image information, and obtaining the mechanical parameters of the cloth based on the spectral information;

[0009] obtaining the current contact force distribution of the remote operation, to obtain a multi-modal tactile feedback signal based on the cloth type, the mechanical parameters and the current contact force distribution;

[0010] controlling the remote operation of the cloth based on the multi-modal tactile feedback signal.

[0011] In an implementation form of the first aspect, obtaining the cloth type based on the image information comprises:

[0012] encoding the image information to extract cloth texture features;

[0013] calculate a cosine similarity between the fabric texture feature and each material feature in a preset material feature library;

[0014] determine the most matched fabric type based on the cosine similarity.

[0015] In an implementation form of the first aspect, the obtaining the mechanical parameter of the fabric based on the spectral information comprises:

[0016] extracting a characteristic absorption peak based on the spectral information;

[0017] calculating a spectral index based on the characteristic absorption peak; the spectral index comprises a crystallinity index and an orientation index;

[0018] predicting a tensile strength and a friction coefficient of the fabric based on the spectral index.

[0019] In an implementation form of the first aspect, the obtaining the current contact force distribution of the teleoperation comprises:

[0020] obtaining a current elastic deformation image of the fabric under the current contact force of the teleoperation;

[0021] obtaining the current contact force distribution corresponding to the current elastic deformation image based on a mapping relationship between the elastic deformation image and the spatial force distribution map;

[0022] obtaining position information, a current contact force size and a current contact force direction of the contact point based on the current contact force distribution.

[0023] In an implementation form of the first aspect, the obtaining the multi-modal haptic feedback signal based on the fabric type, the mechanical parameter and the current contact force distribution comprises:

[0024] calculating a maximum contact force of the teleoperation based on the mechanical parameter;

[0025] obtaining a contact force feedback signal based on the maximum contact force, the current contact force distribution and a set contact force; and

[0026] modulating a vibration frequency of the teleoperation based on the fabric type to obtain a haptic perception feedback signal.

[0027] In an implementation form of the first aspect, the controlling the teleoperation of the fabric based on the multi-modal haptic feedback signal comprises:

[0028] controlling a contact force of the teleoperation of the fabric in real time based on the contact force feedback signal;

[0029] controlling a vibration mode of the teleoperation of the fabric in real time based on the haptic perception feedback signal.

[0030] In an implementation form of the first aspect, the method further comprises predicting the teleoperation intention based on a long short-term memory network to perform feedforward compensation on the teleoperation delay.

[0031] In a second aspect, the present application provides a teleoperation control system for cloth, the system comprising:

[0032] a first obtaining module configured to obtain image information and spectral information of the cloth;

[0033] a second obtaining module configured to obtain a cloth type based on the image information and obtain a mechanical parameter of the cloth based on the spectral information;

[0034] a feedback module configured to obtain a current contact force distribution of the teleoperation to obtain a multi-modal haptic feedback signal based on the cloth type, the mechanical parameter and the current contact force distribution;

[0035] a control module configured to control the teleoperation of the cloth based on the multi-modal haptic feedback signal.

[0036] In a third aspect, the present application provides a terminal, comprising a processor and a memory;

[0037] the memory is configured to store a computer program;

[0038] the processor is configured to execute the computer program stored in the memory, so that the terminal executes the cloth teleoperation control method described above.

[0039] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, which is executed by a terminal to implement the cloth teleoperation control method described above.

[0040] As described above, the cloth teleoperation control method, system, medium and terminal of the present application have the following beneficial effects:

[0041] (1) The present application can accurately and in real time identify a variety of cloth types and their mechanical parameters (such as friction coefficient, tensile strength), ensuring the adaptability and accuracy of the teleoperation strategy.

[0042] (2) The present application can finely perceive the spatial distribution state of the contact force in the teleoperation process, improving the accuracy of force perception.

[0043] (3) The present application performs operation feedback based on the cloth characteristics and the current teleoperation state in real time to dynamically control the teleoperation action, effectively preventing the occurrence of cloth tearing or slipping phenomenon. In addition, the present application generates corresponding vibration patterns according to different cloth materials, so that users can obtain almost consistent haptic experience as actual operation.

[0044] (4) The material identification accuracy of the application is ≥95%, and the force control error is <0.3N, which greatly reduces the probability of fabric damage caused by improper operation and reduces the risk of fabric damage.

[0045] (5) The application shortens the feedback delay and reduces the haptic feedback delay to within 40ms, which is more than 60% shorter than the existing system, greatly improving the operation immersion and realism of the user.

[0046] (6) The application can easily expand new material types and is suitable for a variety of fabric types, with strong flexibility and use range. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart shows the remote operation control method of the fabric of the application in an embodiment.

[0048] Figure 2 The flowchart shows the process of obtaining the fabric type based on the image information of the application.

[0049] Figure 3 The flowchart shows the process of obtaining the current contact force distribution of the remote operation of the application.

[0050] Figure 4 The structural diagram shows the remote operation control system of the fabric of the application in an embodiment.

[0051] Figure 5 The structural diagram shows the terminal of the application in an embodiment. DETAILED DESCRIPTION

[0052] The embodiments of the application are described below through specific, concrete examples, and those skilled in the art can easily understand other advantages and effects of the application from the disclosure of the specification. The application can also be implemented or applied in different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the application in a schematic manner, and only the components related to the application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change in type, number and proportion, and the layout pattern of the components may also be more complex.

[0054] In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features or implicitly indicating the number of the technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0055] Figure 1 A flowchart of a remote operation control method of cloth provided by an embodiment of the present application is shown. As shown in Figure 1 The remote operation control method of cloth of the present application includes steps S1 to S4.

[0056] Step S1, obtaining image information and spectral information of cloth.

[0057] In some embodiments, the image information of the cloth surface is obtained by a high-resolution camera, and the spectral information of the cloth is obtained based on a near-infrared spectrometer.

[0058] Step S2, obtaining cloth type based on the image information, and obtaining mechanical parameters based on the spectral information.

[0059] In some embodiments, the cloth texture features can be extracted from the image information by a deep learning algorithm to identify the cloth type, such as cotton, silk, etc. Figure 2 A flowchart of a remote operation control method of cloth provided by an embodiment of the present application is shown. As shown in Figure 2 Obtaining cloth type based on the image information includes steps S21 to S23.

[0060] Step S21, encoding the image information to extract cloth texture features.

[0061] Specifically, the image information is preprocessed, including image normalization, adaptive histogram equalization and denoising processing, to obtain a standard image, and a convolution network such as a ConvNeXt model is used for encoding to obtain the fabric texture features. The ConvNeXt model is a modern convolutional neural network architecture, which can realize efficient fabric texture feature encoding by inputting the image into the pre-trained ConvNeXt network and extracting the output of a specific layer. The process first standardizes the input image (adjusts to 224x224 resolution and performs ImageNet mean / variance normalization), and then uses the feature extraction layer of the model (usually retained before the global average pooling layer) to convert the fabric image into a fixed-dimensional feature vector (such as the output of ConvNeXt-Tiny 768-dimensional features), that is, the extracted fabric texture features, so as to capture the multi-scale texture, pattern and structure information of the fabric.

[0062] Step S22, calculate the cosine similarity between the fabric texture features and each material feature in the preset material feature library.

[0063] In some embodiments, the method of obtaining the preset material feature library includes: obtaining corresponding fabric images of a plurality of fabrics and pre-processing. Then, the extracted features are encoded by a model such as ConvNeXt, and the extracted features are stored in the material feature library together with the corresponding material labels. At the same time, by saving the model weight and generating a deployable model file, the fabric texture features can be quickly obtained based on the image information in the subsequent process.

[0064] Further, since the preset material feature library has been obtained, which saves the material features of various fabrics and the corresponding material labels. Therefore, in order to identify the obtained fabric texture features, the cosine similarity between the fabric texture features and each material feature in the preset material feature library can be calculated by the Qwen-VL-B-Instruct model. For example, assuming that there are N material features in the preset material feature library, the cosine similarity scores between the fabric texture features and the N material features are calculated, that is, N cosine similarity scores.

[0065] Step S23, determine the most matched fabric type based on the cosine similarity.

[0066] In the above embodiment, the Qwen-VL-B-Instruct model obtains the material feature with the highest score according to the N cosine similarity scores, matches the material label corresponding to the material feature, determines the corresponding fabric type, and completes the identification process of the fabric material.

[0067] Further, the spectral information of different fabrics may actually be different. In some embodiments, the method for obtaining the mechanical parameters of the fabric based on the spectral information comprises: extracting characteristic absorption peaks based on the spectral information; calculating spectral indicators based on the characteristic absorption peaks; the spectral indicators comprise a crystallinity index and an orientation index; and predicting the tensile strength and friction coefficient of the fabric based on the spectral indicators.

[0068] Specifically, first, the reflection or transmission spectrum of the fabric is collected by a near-infrared spectrometer (900-1700 nm range), and characteristic absorption peaks are extracted for different fiber types of the fabric, such as the 1150 nm crystallinity-related peak of cotton and the 1710 nm ester group vibration peak of polyester, and a crystallinity index representing the order degree of molecular chains and an orientation index representing the arrangement direction of fibers are calculated based on the characteristic absorption peaks, so as to predict the tensile strength and friction coefficient of the fabric through the crystallinity index and the orientation index.

[0069] Further, in order to realize the prediction of the tensile strength and friction coefficient through the spectral indicators, it is actually necessary to establish a quantitative correlation model between the spectral indicators and the mechanical parameters. In some embodiments, the spectral indicators and the mechanical parameters obtained by laboratory standard tests (such as ASTM D5034 tensile test) can be modeled by PLS regression or random forest algorithm, and finally the tensile strength and friction coefficient of the fabric can be predicted through the crystallinity index and the orientation index.

[0070] It should be noted that the method for obtaining the corresponding relationship between the spectral indicators and the mechanical parameters can also be established by other deep learning models or other methods, which are not limited in the present application.

[0071] Step S3, obtaining the current contact force distribution of the teleoperation, to obtain a multi-modal haptic feedback signal based on the fabric type, the mechanical parameters, and the current contact force distribution.

[0072] In fact, the teleoperation of the fabric is performed by a robot arm and a dexterous hand. The robot arm can be teleoperated using an exoskeleton device, or the robot arm can be teleoperated using a motion capture device, and the dexterous hand can also be teleoperated using visual capture. When teleoperating the fabric, a certain contact force is required for operation. In order to achieve precise grasping and other operations, the current contact force distribution needs to be obtained to better control the teleoperation action. Figure 3 The flowchart of the fabric teleoperation control method provided by the embodiments of the present application is shown. As shown in Figure 3 The current contact force distribution of the teleoperation includes steps S31 to S33.

[0073] Step S31, obtaining the current elastic deformation image of the fabric under the current contact force of the teleoperation.

[0074] In step S32, the current contact force distribution corresponding to the current elastic deformation image is obtained based on the mapping relationship between the elastic deformation image and the spatial force distribution map.

[0075] In step S33, the position information of the contact point, the current contact force size and the current contact force direction are obtained based on the current contact force distribution.

[0076] In some embodiments, a three-dimensional force sensor such as a tactile sensor array can be arranged on a dexterous hand or a robotic arm to obtain a current elastic deformation image of the cloth under the action of the current contact force. Then, based on the mapping relationship between the elastic deformation image and the spatial force distribution map, the current contact force distribution corresponding to the current elastic deformation image is obtained, and then the three-dimensional coordinates of the contact point and the current contact force size and direction of the contact point are obtained.

[0077] In some embodiments, the mapping relationship between the elastic deformation image and the spatial force distribution map can be learned by training a ResNet model. ResNet (Residual Neural Network) is a kind of deep convolutional neural network model, which solves the training degradation problem caused by gradient vanishing / explosion in traditional deep network by introducing skip connection to directly superimpose the input on the output of the deep network. In some embodiments, in the data collection stage, the surface of the three-dimensional force sensor is contacted by force probes of different positions and directions (simulating the contact between the sensor and the cloth), and the contact force vector, the contact position and the camera internal image are recorded. Then, the data is preprocessed, and the ResNet model is trained to learn the mapping relationship between the contact force distribution and the image features, so that the ResNet model can directly output the three-dimensional coordinates of the contact point and the normal contact force and shear force components of the point based on the deformation image. Moreover, the spatial force distribution map covering the entire sensor surface can also be generated by convolution layer processing.

[0078] Further, after obtaining the cloth type, the mechanical parameters and the current contact force distribution, a multi-modal tactile feedback signal can be obtained based on the cloth type, the mechanical parameters and the current contact force distribution, so as to control the teleoperation action in real time. In some embodiments, obtaining the multi-modal tactile feedback signal based on the cloth type, the mechanical parameters and the current contact force distribution includes: calculating the maximum contact force of teleoperation based on the mechanical parameters; obtaining the contact force feedback signal based on the maximum contact force, the current contact force distribution and the set contact force; and modulating the vibration frequency of teleoperation based on the cloth type to obtain the tactile perception feedback signal.

[0079] Specifically, the calculation of the maximum contact force of teleoperation based on the mechanical parameters can be represented by formula (1):

[0080] F max = a * (μN) + β * T_strength

[0081] Wherein, μ is the friction coefficient, T_strength is the tensile strength, unit is N; N is the normal contact force; a and β are weight coefficients.

[0082] Specifically, the contact force feedback signal can be obtained based on the maximum contact force, the current contact force distribution and the set contact force by formula (2):

[0083]

[0084] Wherein, K p is a proportional gain, which provides a control amount proportional to the current error, the greater the error, the greater the output affects the response speed of the system; K i is an integral gain, which eliminates steady-state error; F error is the error force, which is the output difference between the set contact force and the current contact force F actual ; F actual is the current contact force, which can be obtained from the current contact force distribution.

[0085] It should be noted that the current contact force of the cloth can also be obtained by a piezoresistive sensor, a piezoelectric sensor, etc.

[0086] Specifically, the implementation manner of modulating the vibration frequency of the remote control based on the cloth type to obtain the tactile perception feedback signal includes: modulating the vibration frequency of the mechanical arm or the dexterous hand, which can simulate the roughness of the cloth texture, for example, high frequency modulation (such as 200Hz) can generate delicate concave-convex texture (such as the tiny wrinkles of silk), and low frequency modulation (such as 50Hz) can produce more gentle undulation (such as the thick and solid texture of coarse linen), thereby obtaining the tactile perception feedback signal, i.e. the vibration frequency to be reached.

[0087] Step S4, controlling the remote operation of the cloth based on the multi-modal tactile feedback signal.

[0088] In some embodiments, controlling the remote operation of the cloth based on the multi-modal tactile feedback signal includes: controlling the contact force of the remote operation of the cloth in real time based on the contact force feedback signal; controlling the vibration mode of the remote operation of the cloth in real time based on the tactile perception feedback signal.

[0089] Wherein, the contact force feedback signal is used to modulate the mechanical arm or the finger force of the dexterous hand in real time, that is, the contact force is adjusted in real time according to the contact force feedback signal, so as to ensure that the cloth is neither damaged nor slipped. The linear resonant actuator (LRA) is used to generate a corresponding vibration mode based on the tactile perception feedback signal, so that the user can obtain a tactile experience almost consistent with the actual operation.

[0090] In addition, in some embodiments, temperature feedback can also be added. That is, the target temperature is set according to the thermal conductivity coefficient of the material of the cloth type, so that the user can feel the texture and temperature of the material without directly contacting the material.

[0091] In some embodiments, the teleoperation has a certain delay. Therefore, the teleoperation intention can be predicted based on the long short-term memory network to perform feedforward compensation on the teleoperation delay. The long short-term memory network (LSTM) effectively learns the time sequence dependence in teleoperation through its gating mechanism (input gate, forget gate, and output gate), and can capture the potential intention pattern of the operator from the time sequence data such as historical motion trajectory and multi-modal tactile feedback signal. The network uses a memory unit to long-term retain key state features (such as operation habits), while filtering noise interference, and predicts the operation instruction (such as the movement direction or force of the mechanical arm) at the next moment in real time, which significantly improves the response speed and fluency of teleoperation.

[0092] The protection scope of the cloth teleoperation control method described in the embodiments of the present application is not limited to the execution order of the steps listed in the embodiments. Any scheme realized by adding, replacing or modifying the steps of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0093] The embodiments of the present application also provide a cloth teleoperation control system, which can implement the cloth teleoperation control method described in the present application. However, the implementation device of the cloth teleoperation control system described in the present application includes but is not limited to the structure of the cloth teleoperation control system listed in the embodiments. Any structure modification and replacement of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0094] As shown in FIG. 1, Figure 4 In an embodiment, the cloth teleoperation control system of the present application includes a first acquisition module 91, a second acquisition module 92, a feedback module 93, and a control module 94.

[0095] The first acquisition module 91 is configured to acquire image information and spectral information of the cloth;

[0096] The second acquisition module 92 is configured to acquire the cloth type based on the image information and acquire the mechanical parameter of the cloth based on the spectral information;

[0097] The feedback module 93 is configured to acquire a current contact force distribution of the teleoperation, to acquire a multi-modal haptic feedback signal based on the cloth type, the mechanical parameter and the current contact force distribution;

[0098] The control module 94 is configured to control the teleoperation of the cloth based on the multi-modal haptic feedback signal.

[0099] The first acquisition module 91, the second acquisition module 92, the feedback module 93 and the control module 94 correspond to the steps of the method for controlling the teleoperation of the cloth one by one in structure and principle, and thus will not be described here.

[0100] In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, or method can be implemented in other ways. For example, the apparatus embodiments described above are only schematic. The division of the modules / units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed modules can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0101] The modules / units described as separate components can or can not be physically separate, and the components shown as modules / units can or can not be physical modules, i.e., can be located in one place or distributed on a plurality of network units. Part or all of the modules / units can be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in each embodiment of the present application can be integrated into a processing module, or each module / unit can be physically independent, or two or more modules / units can be integrated into one module / unit.

[0102] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0103] The embodiments of the present application further provide a computer readable storage medium. Those skilled in the art can understand that all or part of the steps of the method described above can be instructed by a program to complete the processor, and the program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, for example, a random access memory, a read-only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc and any combination thereof. The storage medium can be any available medium accessible by a computer or a data storage device such as a server, a data center and the like, which includes one or more available medium sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)) or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0104] The embodiments of the present application further provide a terminal. The terminal comprises a processor and a memory.

[0105] The memory is used to store a computer program.

[0106] The memory comprises a ROM, a RAM, a disk, a U disk, a memory card or an optical disc and the like various medium capable of storing program codes.

[0107] The processor is connected with the memory, and is used to execute the computer program stored in the memory, so that the terminal executes the remote control method of the cloth described above.

[0108] Preferably, the processor can be a general processor, including a central processing unit (CPU), a network processor (NP) and the like; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0109] As Figure 5As shown, the terminal of the present application is in the form of a general- purpose computing device. The components of the terminal can include, but are not limited to, one or more processors or processing units 101, a main memory 102, and a bus 103 that couples various system components including the memory 102 to the processing unit 101.

[0110] The bus 103 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics bus (e.g., AGP or Accelerated Graphics Port bus), and a local bus using any of a variety of bus architectures. By way of example, these architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0111] The terminal typically includes a variety of computer system readable media. Such media can be any available media that is located either in or out of the computing device, including both volatile and nonvolatile media, removable and non-removable media.

[0112] The memory 102 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1021 and / or cache memory 1022. The terminal can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 1023 can be provided for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 5 Not shown, a removable / non-removable interface can also be coupled to the bus 103 through which a removable storage unit 1024 can be attached and detached, such as a magnetic disk, optical disk or tape cartridge. A storage interface can also be coupled to the bus 103 through which a non-removable, non-volatile memory unit 1025 can be connected and disconnected, such as a solid state drive or other persistent storage unit. Figure 5 In alternative embodiments, a storage device 1024 can be connected to the bus 103 by a storage interface (not shown) that can also appear as a media device and a storage controller. Similar to the storage interface that was described previously, the storage interface can also be configured to read from and write to a removable, non-removable, or constant presence storage medium 1024 such as a floppy disk, a CD-ROM, a DVD-ROM, or another digital medium. The memory 102 can include a number of program products including a program 1024 that can be configured to implement embodiments of the present application. The program 1024 utilized by the memory 102 can also include computer readable program instructions that can implement one or more embodiments provided by the various other embodiments of the present application.

[0113] Program / utility 1024 having a set (at least one) of program modules 10241 can be stored in memory 102 by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, can include implementation of a network environment. Program modules 10241 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0114] The terminal can also be in communication with one or more external devices (e.g., a keyboard, a pointing device, a display, etc.) and can communicate with one or more devices that enable a user to interact with the terminal. Further, the terminal can communicate with one or more devices that enable the terminal to Figure 5 communicate with other computing devices. Such communication can occur via Input / Output (I / O) interface 104. Still yet, the terminal can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet) through network adapter 105. As depicted, network adapter 105 communicates with the other components of the terminal via bus 103. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with the terminal. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0115] The above-described embodiments are merely illustrative for the principles of the present application and the effects achieved by the present application, and are not used to limit the present application. Any modification or change to the above-described embodiments made by any person skilled in the art, without departing from the spirit and scope of the present application, shall be covered by the claims of the present application.

Claims

1. A remote control method for fabrics, characterized in that: The method comprises: Obtain image information and spectral information of the fabric; Acquire the type of the fabric based on the image information, and acquire mechanical parameters of the fabric based on the spectral information; obtaining a current contact force distribution of the teleoperation to obtain a multimodal tactile feedback signal based on the fabric type, the mechanical parameters, and the current contact force distribution; The remote operation of the cloth is controlled based on the multimodal tactile feedback signal.

2. The remote control method for fabric according to claim 1, characterized in that: Acquiring the type of fabric based on the image information includes: encoding the image information to extract fabric texture features; Calculating the cosine similarity between the fabric texture feature and the material features in the preset material feature library; The cloth type that best matches the fabric is determined based on the cosine similarity.

3. The remote control method for fabric according to claim 1, characterized in that: Obtaining the mechanical parameters of the fabric based on the spectral information includes: Extracting characteristic absorption peaks based on the spectral information; Calculating spectral indices based on the characteristic absorption peak; the spectral indices include a crystallinity index and an orientation index; The tensile strength and friction coefficient of the fabric are predicted based on the spectral index.

4. The remote control method for fabric according to claim 1, characterized in that: Obtaining the current contact force distribution of the teleoperation includes: Obtain the current elastic deformation image of the cloth under the current contact force of the teleoperation; Based on a mapping relationship between the elastic deformation image and the spatial force distribution map, obtaining the current contact force distribution corresponding to the current elastic deformation image; The position information of the contact point, the current contact force magnitude and the current contact force direction are obtained based on the current contact force distribution.

5. The remote control method for fabric according to claim 1, characterized in that: Acquiring a multimodal tactile feedback signal based on the cloth type, the mechanical parameters, and the current contact force distribution includes: Calculating a maximum contact force of teleoperation based on the mechanical parameters; obtaining a contact force feedback signal based on the maximum contact force, the current contact force distribution, and a set contact force; and The vibration frequency of the remote control is modulated based on the fabric type to obtain a tactile feedback signal.

6. The remote control method for fabric according to claim 5, characterized in that: Controlling the remote operation of the fabric based on the multimodal tactile feedback signal includes: Controlling the contact force of the remote operation of the cloth in real time based on the contact force feedback signal; The vibration mode of the remote operation of the cloth is controlled in real time based on the tactile perception feedback signal.

7. The remote control method for fabric according to claim 1, characterized in that: The method further comprises: Teleoperation intention is predicted based on long short-term memory network to perform feedforward compensation for teleoperation delay.

8. A remote control system for fabrics, characterized in that: The system comprises: a first acquisition module, configured to acquire image information and spectral information of the fabric; a second acquisition module, configured to acquire the type of the fabric based on the image information, and acquire mechanical parameters of the fabric based on the spectral information; a feedback module configured to obtain a current contact force distribution of the teleoperation to obtain a multimodal tactile feedback signal based on the cloth type, the mechanical parameters, and the current contact force distribution; The control module is configured to control the remote operation of the cloth based on the multimodal tactile feedback signal.

9. A terminal, characterized in that: The terminal includes: a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, so as to enable the terminal to execute the remote control method for fabric according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a terminal, the remote control method for fabric according to any one of claims 1 to 7 is realized.

Images