Adaptive gradient soft-wire one-step buckle prediction method based on virtual-real fusion

Abstract

This invention discloses a one-step snap-fit ​​prediction method for flexible flat cable based on virtual-real fusion and adaptive gradient. First, in the real operating environment of mobile phone flexible flat cable assembly, real tactile images are acquired using a tactile sensor. Then, based on the real tactile images, the Material Point Method (MPM) is used to simulate the real tactile sensation in a virtual environment, generating a virtual tactile image. Real and virtual tactile sensations together constitute a digital twin environment. By performing adaptive gradient calculation on the virtual and real tactile images in the digital twin environment, a one-step prediction model based on adaptive gradients is established. The model uses a CNN+LSTM network to predict the tactile images, generating predicted tactile images, which are then compared with ideal tactile images. Then, by encoding and decoding the tactile images, a one-step snap-fit ​​guidance strategy is obtained and applied in the real operating environment. The one-step prediction model based on adaptive gradients is trained and iterated in the digital twin environment.

Description

Technical Field

[0001] This invention belongs to the field of industrial intelligent assembly in artificial intelligence. It is a method for generating soft flat cable fastening strategies based on a virtual-real fusion environment, which is used by intelligent agents to generate soft flat cable fastening strategies based on multimodal perception. Background Technology

[0002] In today's society, robots have been widely used in various assembly tasks in industry. Robots can help humans complete a number of tasks, such as assembling cars ([Zachares PA, Lee MA, Lian W, et al. Interpreting contact interactions to overcome failure in robot assembly tasks[A]. In: 2021 IEEE International Conference on Robotics and Automation (ICRA)[C]. IEEE, 2021. 3410-3417.]), assembling furniture ([Lee Y, Hu ES, Lim J J. IKEA furniture assembly environment for long-horizon complex manipulation tasks[A]. In: 2021 IEEE international conference on robotics and automation (ICRA)[C]. IEEE, 2021. 6343-6349.]), and machine assembly ([Kimble K, Van Wyk K, Falco J, et al. Benchmarking protocols for evaluating small parts robotic assembly systems[J]. IEEE robotics and automation letters, 2020, 5(2): 883-889.]). Robots are gradually replacing humans in the assembly of small electronic devices, such as the assembly of flexible cables for mobile phones. The task of assembling flexible cables requires robots to correctly assemble the flexible cables on both sides with as few experiments as possible. This is a highly complex assembly task that also requires a high degree of precision in operation.

[0003] In the process of assembling mobile phone flexible flat cables, there are currently errors in the robot's perception of the operating scene. There are still significant errors in the acquisition and processing of multimodal data, including tactile, visual, and auditory data. For example, when the robotic arm is fastening tiny parts like mobile phone flexible flat cables, some parts of the parts are not visible due to the robotic arm's obstruction, making them difficult to measure and causing assembly failures. Moreover, repeated failures may even damage the parts.

[0004] However, if robot skill learning can also be conducted in a virtual environment, the integration of real and virtual worlds allows for the measurement of data that is difficult to measure in the real world, significantly improving the efficiency and accuracy of robot skill learning. Therefore, using a virtual-real fusion approach for robot skill learning is a current hot research direction. In the process of learning skills through virtual-real fusion, using multimodal interaction methods to acquire data offers advantages such as strong real-time performance and high interactivity. Consequently, multimodal interaction has been widely applied in robot skill learning in recent years.

[0005] Simulation also plays a crucial role in robotics research. Real-world experiments are expensive and time-consuming, and pose potential risks of wear and tear and accidents to robots. Simulation, however, can provide data for preliminary experiments without these risks. Furthermore, data-driven methods such as neural networks and deep learning are used in robotics for sensing and control. These methods require vast amounts of data, and simulation offers an efficient way to collect such data, potentially enabling the transfer of well-trained robotic agents to the real world.

[0006] In recent years, various simulation methods have been proposed for object simulation, such as the finite element method (FEM) and the key point method. Several commonly used robot simulators exist, such as Gazebo, Pybullet, and MuJoCo. While they are capable of simulating robotic components such as robotic arms and grippers, tactile sensor simulation remains a challenging task, hindering the simulation of physical robots with tactile perception.

[0007] Tactile sensing is essential for contact control in robots, and many tactile sensors have been developed in recent years. Optical tactile sensors, which can generate high-resolution tactile images, and some that use cameras to capture the deformation of soft elastomers, are favored in robotics research. Due to their low cost and high resolution, optical tactile sensors have been widely used in practical experiments, but the challenge of simulating the deformation of their soft elastomer layers has hindered their simulation.

[0008] Adaptive control has a wide range of applications in the current process of robots performing industrial assembly tasks. Adaptive robots can make self-adjustments in real time to adapt to changes in the environment. By using the gradient changes of virtual and real tactile images, an adaptive method can effectively obtain a one-step fastening strategy for soft flat cables. Summary of the Invention

[0009] To address the aforementioned issues, this invention proposes an adaptive gradient soft flat cable snapping prediction method based on virtual-real fusion. By performing tactile simulation in a virtual environment, the method reduces perception errors and effectively improves the success rate of soft flat cable snapping.

[0010] This invention relates to a one-step prediction method for soft flat cable snapping based on virtual-real fusion and adaptive gradient, which includes a digital twin environment for soft flat cable snapping, a one-step prediction model based on adaptive gradient, and a virtual-real tactile image comparison model.

[0011] The digital twin environment for the flexible flat cable fastening is used to generate virtual and real tactile images. The specific method is as follows:

[0012] a) In a real operating environment, the robotic arm begins to fasten the flexible cable.

[0013] b) The tactile sensor on the end effector of the robotic arm senses the real tactile image.

[0014] c) Analyze real tactile images using the material point method.

[0015] d) Implement virtual tactile deformation simulation in a virtual environment.

[0016] e) Generate virtual haptic images in a virtual environment.

[0017] f) The real tactile image and the virtual tactile image together constitute the digital twin environment for the flexible flat cable fastening.

[0018] The specific method for establishing the one-step prediction model based on adaptive gradient is as follows:

[0019] First, a virtual-real tactile prediction model is established. By comparing the real tactile images and virtual tactile images in the current experiment with the real tactile images and virtual tactile images under the ideal successful fastening condition, the prediction result of the virtual-real fastening condition is obtained.

[0020] Secondly, an adaptive gradient model is established. After the virtual-real tactile prediction model obtains the prediction result, if the prediction result is successful snapping, no adjustment is required. If the prediction result is snapping failure, the step size and direction of the soft cable pose adjustment need to be obtained according to the adaptive gradient method. The adjustment process needs to be compared with the ideal tactile image.

[0021] Finally, a virtual-real tactile image comparison model is established. Based on the virtual-real adjustment step size and direction obtained by the one-step prediction model based on adaptive gradient, the optimal virtual-real tactile image adjustment strategy is obtained by training iteration in the digital twin environment. The weights are used to obtain the final adjustment strategy, which enables the soft flat cable to reach the position before successful fastening through one-step pose adjustment.

[0022] The advantages of this invention are:

[0023] 1. The present invention is an adaptive gradient soft flat cable snapping prediction method based on a virtual-real fusion environment, which reduces the perception error of soft flat cable snapping scenarios in real environment. Currently, there are errors in the perception process of robot operation scenarios, and there are still large errors in the acquisition and processing of tactile, visual and auditory multimodal data. The present invention reduces the perception error by performing tactile simulation in a virtual environment.

[0024] 2. The present invention is based on an adaptive gradient one-step fastening prediction method for flexible flat cables in a virtual-real fusion environment. This method reduces the assembly tolerance of small parts such as flexible flat cables. When the robotic arm fastens small parts like mobile phone flexible flat cables, some parts of the parts are not visible due to the obstruction of the robotic arm, making them difficult to measure and causing assembly failure. Moreover, multiple failures may even damage the parts. The present invention obtains a prediction strategy through an adaptive gradient method, which reduces the occurrence of fastening failure.

[0025] 3. The present invention provides an adaptive gradient soft flat cable snapping prediction method based on a virtual-real fusion environment, which improves the snapping success rate of soft flat cables. By adding tactile simulation in the virtual environment and fusing the snapping strategies obtained from virtual and real tactile data, the present invention effectively improves the snapping success rate of soft flat cables. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the one-step snapping prediction method for adaptive gradient soft flatbed cable based on virtual-real fusion of the present invention.

[0027] Figure 2 This is a 3D tactile image and gradient annotation of the present invention.

[0028] Figure 3 This is a curve showing the change in force at the contact point in a tactile sensor over time.

[0029] Figure 4 Before and after mesh segmentation.

[0030] Figure 5 This is a schematic diagram of the adaptive gradient model based on virtual-real fusion of the present invention.

[0031] Figure 6 This is a flowchart of the execution process of an adaptive gradient model based on virtual-real fusion. Detailed Implementation

[0032] The present invention will now be described in further detail with reference to the accompanying drawings.

[0033] This invention relates to an adaptive gradient one-step prediction method for soft flat cable snapping based on a virtual-real fusion environment. The method includes a digital twin environment for soft flat cable snapping, a one-step prediction model based on adaptive gradients, and a virtual-real tactile image comparison model. Figure 1 As shown.

[0034] The virtual and real tactile images are generated in the digital twin environment where the flexible flat cable is fastened. The specific process is as follows:

[0035] 1. Acquisition and analysis of real tactile data.

[0036] The actual tactile data is obtained from the contact between the 4*11 dimensional tactile sensor mounted on the end of the robotic arm assembling the mobile phone flexible flat cable and the fastening end of the cable. The tactile data at each moment is a 4*11 dimensional matrix. The tactile data is analyzed and fitted from two perspectives:

[0037] A. Analyze the 4*11 tactile data points at each moment;

[0038] B. Analyze tactile data from the same force acquisition point at multiple times.

[0039] For A, by plotting a three-dimensional graph of the force tactile data at different times from the 44 touch points of a 4*11 dimensional tactile sensor, and analyzing its statistical characteristics, the force distribution pattern can be clearly observed from the three-dimensional graph, such as... Figure 2 As shown.

[0040] For B, it can be divided into three stages:

[0041] The first stage is the contact stage, where the flexible flat cable begins to make contact with the mobile phone, and the force continuously increases. When the snap-fit ​​end of the flexible flat cable enters the slot on the mobile phone, the resistance decreases rapidly, causing the contact force to drop instantly, and the next stage begins.

[0042] The second stage is the fastening stage, which is the process of the flexible flat cable fastening end entering the slot until it touches the bottom. When the fastening process ends, the rate of force increase decreases significantly, and the next stage begins.

[0043] The third stage is the pressing stage, where the end of the flexible cable has reached the bottom, but the robotic arm continues to apply pressure. At this stage, the rate of force increase is significantly slower than in the fastening stage.

[0044] like Figure 3As shown, the slopes of the force-time curves at the contact points in the three stages of the tactile sensor are significantly different. A piecewise linear method can be used for fitting to obtain the piecewise curves of the flexible flat cable fastening process. Therefore, the current fastening status can be determined based on the tactile data, guiding the flexible flat cable fastening process.

[0045] 2. Simulate virtual tactile data using real tactile data.

[0046] Collecting multimodal data from virtual environments offers significant advantages over real-world environments. Firstly, it accelerates and improves the speed and efficiency of robot skill transfer and learning, for example, allowing for faster and more accurate guidance to the final position. The quality of tactile data simulation in a virtual environment fundamentally depends on the realism of object deformation simulation. Traditional simulation environments, limited by the contradiction between algorithm complexity and real-time simulation, struggle to achieve high-precision real-time simulation of physical deformations generated during interaction. However, applying the Material Point Method (MPM) to the Unity simulation environment enables high-precision real-time simulation of elastic body deformation.

[0047] Taking the assembly of flexible flat cables in 3C assembly as an application scenario, the material point method (MPM) is used to calculate the deformation of the flexible flat cable during the fastening process with the female connector of the mobile phone. It can be assumed that the male connector of the flexible flat cable is elastic during the pressing process, while the female connector of the mobile phone is approximately a rigid body.

[0048] First, the virtual soft wire model (usually a mesh) needs to be processed to transform it into a group of particles with mass, position, and velocity. These particles interact through the virtual mesh and have different velocities and positions, which are updated in each simulation step. The envelope of the particles and their respective velocities reflect the changes in the shape of the elastic body and the stress level inside the body.

[0049] Subsequently, voxelization was used to segment the previously processed mesh, the position of each voxel was recorded, and they were assigned mass and velocity. The effects before and after segmentation are as follows: Figure 4 As shown.

[0050] The specific method is as follows:

[0051] In each simulation step, particle information is first transferred to the grid by calculating the momentum and mass of each grid node. This can be viewed as simulating a portion of the objects around the grid node through quadratic b-spline weighted interpolation; specifically, it involves collecting the mass and momentum of nearby particles. The mass M of the i-th grid node... i for:

[0052]

[0053] In the formula, m p Let G be the mass of the p-th particle.i P is a 3×3×3 grid containing the i-th grid node and its adjacent grid nodes. j For the particle in the j-th grid node, ω ij The weighting parameters for the weighted interpolation of the i-th grid node and the j-th particle are calculated using quadratic β-splines. By applying these weighting parameters, the near-particle pair M... i The contribution of the particle is greater than that of the distant particle. This invention follows the principle of giving mass and the following momentum calculations the same weighting parameters.

[0054] Calculate the momentum MM generated by the motion of the particle. i and the momentum ME generated by elasticity i The grid momentum MG of the i-th grid node can be obtained. i :

[0055] MG i =MM i +ME i (2)

[0056] MM is calculated by collecting the velocities and affine velocities of nearby particles. i :

[0057]

[0058] Where, x j Indicates the position of the j-th adjacent node of the i-th grid node; x p Indicates the position of the p-th particle; v P C represents particle velocity. p The affine velocity matrix is ​​used to record the velocities of adjacent particles. Its purpose is to reduce information loss when particles exchange information with grid nodes. The affine matrix is ​​initialized as a three-dimensional zero matrix because the particles are initially at rest.

[0059] ME i It can be obtained through the following formula:

[0060]

[0061] Where Δt is the time interval between two adjacent simulation steps, and Δx is the grid node interval. Let S be the initial particle volume. p Let be the elastic force of the p-th particle.

[0062] Find M respectively i and MG i Then, the value of the i-th grid node V can be calculated. i The velocity of the nearby object is:

[0063]

[0064] After transforming the deformation of a deformable body into the motion of particles within a particle ensemble in the MPM-based simulation, the particle motion contains all the force information generated by the deformation of the body. Therefore, after achieving virtual tactile deformation simulation using the Material Point Method (MPM), force acquisition algorithms can be used to collect tactile data from the deformation of the elastic body, thereby generating virtual tactile images. This allows for high-precision deformation simulation and the generation of realistic tactile images while ensuring real-time performance. Like real tactile images, virtual tactile images are of great significance for robot operation guidance. Virtual tactile deformation simulation and the generation of virtual tactile images lay the foundation for subsequently establishing a one-step prediction model based on adaptive gradients.

[0065] For the complex structural elastomers addressed in this invention, the main force information generated is due to the different force distributions caused by structural complexity and varying contact postures. The purpose of the tactile sensor is to present this force distribution in a certain way. In real-world physical environments, the tactile sensor is a 4*11 force acquisition matrix. Each force acquisition point in the matrix can sense the locally generated contact force; therefore, the sensor output is a 4*11 tactile image, where each pixel represents the force magnitude at that point. For consistency, the designed virtual sensor maintains the same shape and structure as the real sensor. Next, this invention designs an algorithm to statistically analyze the local force information of each force acquisition point on the virtual sensor deployed in the virtual environment. The specific method is as follows:

[0066] The sensor is deployed on the base of the deformable object. Considering any force acquisition point, a ray is emitted from that point as the origin towards the normal direction facing the deformable object. This ray passes through several voxels of the deformable object. The information carried by these voxels during their motion comprehensively reflects the force information at that point. The force is synthesized using the following formula:

[0067]

[0068] Assume the ray passes through n voxels; m i and v i Let be the mass and velocity of the i-th voxel; Δt be the simulation step size; k be the equivalent elastic coefficient of the material in the normal direction; x n The indentation displacement in the ray direction is the voxel furthest from the sampling point in the ray direction compared to before the simulation started; k1 and k2 are a pair of empirical harmonic parameters, and their relative magnitudes reflect the microscopic and macroscopic biases in the statistics. k1+k2=1, and theoretically, the forces modified by k1 and k2 are of equivalent magnitude.

[0069] Therefore, the force distribution collected by the tactile sensor can be represented as follows:

[0070]

[0071] Each element in the matrix is ​​obtained using formula (6).

[0072] The final step in sensor simulation is to represent the acquired force matrix in some form. The first considered representation is a grayscale image. Therefore, each sampling point is mapped to a display pixel, and the force magnitude at each sampling point is related to the grayscale value of the pixel. Specifically, the grayscale value of each sampling point is calculated as follows:

[0073]

[0074] Among them, G vm The maximum grayscale value is usually set to 255, F. i Let be the force at the i-th sampling point. It can be seen that at least one sampling point will obtain the maximum gray value in a single sampling.

[0075] The purpose of establishing a digital twin environment is to reduce data errors caused by factors such as occlusion in the real environment by establishing a virtualization model. By simulating tactile sensation in the virtual environment, virtual tactile images are obtained. Then, by comparing and integrating the adjustment direction and step size obtained after processing the virtual tactile images together with the real tactile images through an adaptive model, the optimal result is obtained.

[0076] The structure of the one-step prediction model based on adaptive gradient is as follows: Figure 5 As shown, the generation method is as follows:

[0077] 1. Establishment of a virtual-real tactile prediction model

[0078] After completing the construction of the digital twin environment for the flat cable fastening and generating virtual and real tactile images from the digital twin environment, it is necessary to analyze the generated virtual and real tactile images and use a CNN+LSTM network to predict the flat cable fastening result through the virtual and real tactile images. That is, the CNN+LSTM network constitutes the virtual and real tactile prediction model.

[0079] The virtual-real haptic prediction model first requires obtaining a virtual haptic image, called the ideal haptic image, beforehand, which can be successfully matched. Then, it compares the currently obtained virtual-real haptic image with the previously obtained ideal haptic image. Through image encoding and decoding, a CNN+LSTM network is used to extract image features. CNNs can extract latent features from sample data by using convolutional kernels, while Long Short-Term Memory (LSTM) networks process and predict important events with long time intervals in the sample sequence. The CNN+LSTM network satisfies the function of extracting image features and also conforms to the temporal requirements of robot operation tasks.

[0080] If the prediction result obtained by the CNN+LSTM network is that the matching is successful, then no further adjustment process using the adaptive gradient method is required; if the prediction result is that the matching is unsuccessful, then the following adaptive gradient method needs to be used for adjustment.

[0081] 2. Establishment of the Adaptive Gradient Model

[0082] After the virtual-real tactile prediction model obtains the prediction result, if the prediction result indicates successful snapping, no adjustment is needed; if the prediction result indicates snapping failure, the step size and direction of the soft cable pose adjustment need to be obtained according to the adaptive gradient method. The adjustment process needs to be compared with the ideal tactile image, such as... Figure 6 As shown, the specific process is as follows:

[0083] a) Predict the fastening result using a realistic tactile prediction model;

[0084] b) If the expected engagement is successful, no adjustment is needed;

[0085] c) If the predicted fastening fails, the adjustment step size and direction are determined by the real tactile adaptive gradient model;

[0086] d) Predict the fastening result using a realistic tactile prediction model;

[0087] e) If the engagement is expected to be successful, no adjustment is needed;

[0088] f) If the predicted engagement fails, the adjustment step size and direction are determined by the real tactile adaptive gradient model.

[0089] When comparing with an ideal tactile image, two main aspects are considered: first, the location where the gradient is zero, representing the center of successful fastening; and second, the direction of the maximum gradient, representing the rotation direction of successful fastening. Based on the comparison results, the position of the flexible cable is adjusted, and then prediction continues using LSTM until the error is within a set threshold range.

[0090] Meanwhile, when comparing with the ideal tactile image, the distance between the robotic arm and the phone needs to be standardized. The sampling time order cannot be followed, as the duration of each experiment is not fixed, and following the sampling time order would lead to inconsistent positions. Based on the criteria of a gradient of 0 and a maximum gradient, the prediction step size is calculated by encoding and decoding the tactile image.

[0091] 3. Establishment of a comparative model of virtual and real tactile images;

[0092] The virtual-real tactile image comparison model, based on the virtual-real adjustment step size and direction obtained from the one-step prediction model based on adaptive gradients, obtains the optimal virtual-real tactile image adjustment strategy by using weights through training iterations in a digital twin environment, resulting in the final adjustment strategy. This allows the soft cable to reach the position before successful fastening through a one-step pose adjustment. Specifically:

[0093] 1) Data import: The adjustment step size and direction obtained by comparing the real tactile image obtained from the adaptive model with the ideal real tactile image, and the adjustment step size and direction obtained by comparing the virtual tactile image with the ideal virtual tactile image, are input into the virtual-real tactile image comparison model;

[0094] 2) Training of the neural network: After obtaining the virtual and real tactile adjustment strategy, the neural network needs to be trained in the digital twin environment to obtain the final network weights and determine the confidence of the virtual and real tactile adjustment strategy.

[0095] 3) Execution of the final strategy: Based on the confidence level of the virtual and real tactile adjustment strategy, the final adjustment strategy is determined and fed back into the real environment. With one adjustment, the robotic arm can be adjusted to the position before the successful engagement.

Claims

1. A one-step snapping prediction method for adaptive gradient soft flatbed cable based on virtual-real fusion, characterized in that: It includes a digital twin environment for soft flat cable fastening, a one-step prediction model based on adaptive gradients, and a virtual-real tactile image comparison model; The digital twin environment for the flexible flat cable fastening is used to generate virtual and real tactile images. The specific method is as follows: a) In a real operating environment, the robotic arm begins to perform soft cable fastening; b) The tactile sensor on the end effector of the robotic arm senses a real tactile image; c) Analyze real tactile images using the material point method; d) Implement virtual tactile deformation simulation in a virtual environment; e) Generate virtual haptic images in a virtual environment; f) Real tactile images and virtual tactile images together constitute the digital twin environment for the flexible flat cable fastening; The specific method for establishing the one-step prediction model based on adaptive gradient is as follows: First, a virtual-real tactile prediction model is established. By comparing the real and virtual tactile images in the current experiment with the real and virtual tactile images under the ideal successful snapping condition, the prediction result of the virtual-real snapping condition is obtained. The virtual-real tactile prediction model is composed of a CNN+LSTM network. When comparing images, the CNN+LSTM network is used to extract image features by encoding and decoding the images. The CNN extracts the latent features from the sample data by using convolutional kernels, while the Long Short-Term Memory (LSTM) network processes and predicts important events with long time intervals in the sample sequence. Secondly, an adaptive gradient model is established. After the virtual-real tactile prediction model obtains the prediction result, if the prediction result is successful fastening, no adjustment is needed; if the prediction result is failed fastening, the step size and direction of the soft cable pose adjustment need to be obtained according to the adaptive gradient method. The adjustment process needs to be compared with the ideal tactile image. Specifically, the adaptive gradient model is compared with the ideal tactile image from two aspects: first, the position where the gradient is 0, which represents the center position of successful fastening; second, the direction of the maximum gradient, which represents the rotation direction of successful fastening. At the same time, when comparing with the ideal tactile image, the distance between the robotic arm and the mobile phone needs to be unified. Based on the judgment of gradient 0 and maximum gradient, the step size of one prediction step is calculated by encoding and decoding the tactile image. Finally, a virtual-real tactile image comparison model is established. Based on the virtual-real adjustment step size and direction obtained by the one-step prediction model based on adaptive gradient, the optimal virtual-real tactile image adjustment strategy is obtained by training iteration in the digital twin environment. The weights are used to obtain the final adjustment strategy, which enables the soft flat cable to reach the position before successful fastening through one-step pose adjustment.

2. The one-step snapping prediction method for adaptive gradient soft flatbed cable based on virtual-real fusion as described in claim 1, characterized in that: In the process of generating virtual and real tactile images, real tactile data is analyzed and fitted from two perspectives: A. Analyze the 4*11 tactile data points at each moment; B. Analyze tactile data from the same force acquisition point at multiple times; For A, by drawing a three-dimensional graph composed of force tactile data from 44 touch points of a 4*11 dimensional tactile sensor at different times, and analyzing the statistical characteristics, the force distribution pattern can be determined from the three-dimensional graph. For B, it can be divided into three stages: The first stage is the contact stage, where the flexible flat cable begins to make contact with the mobile phone, and the force continuously increases. When the snap-fit ​​end of the flexible flat cable enters the slot on the phone, the resistance decreases rapidly, causing the contact force to drop instantly, and the process enters the next stage. The second stage is the fastening stage, which is the process of the flexible flat cable fastening end entering the slot until it touches the bottom. When the fastening process ends, the rate of force increase decreases significantly, and the next stage begins. The third stage is the pressing stage, where the end of the flexible cable has reached the bottom, but the robotic arm continues to apply pressure. At this stage, the rate of increase in force is significantly reduced compared to the fastening stage. The slopes of the force-time curves at the touch points in the three stages of the tactile sensor described above are significantly different. Piecewise linear fitting was used to obtain the piecewise curves of the soft flat cable fastening process.

3. The one-step snapping prediction method for adaptive gradient soft flatbed cable based on virtual-real fusion as described in claim 1, characterized in that: In the process of generating virtual and real tactile images, the specific method for analyzing real tactile images using the material point method and generating virtual tactile images is as follows: In each simulation step, particle information is first transferred to the mesh by calculating the momentum and mass of each mesh node, and the mass of the i-th mesh node. for: （1） In the formula, Let p be the mass of the p-th particle. Let be a 3 × 3 × 3 grid node containing the i-th grid node and its adjacent grid nodes. For the particle in the j-th grid node, The weighting parameters for the weighted interpolation of the i-th grid node and the j-th particle are calculated using quadratic β-splines; by applying the weighting parameters, the near-particle pairs... The contribution is greater than that of distant particles; The weighting parameter is the same as the momentum calculation described below; Calculate the momentum generated by the motion of a particle. Momentum generated by elasticity The grid momentum of the i-th grid node can be obtained. : （2） MM is calculated by collecting the velocities and affine velocities of nearby particles. i : （3） in, This indicates the position of the j-th adjacent node of the i-th grid node; The particle velocity; An affine velocity matrix for recording the velocities of adjacent particles; It is obtained through the following formula: （4） in, The time interval between two adjacent simulation steps. x is the grid node spacing. The initial particle volume, Let be the elastic force of the p-th particle; Find M respectively i and MG i After that, the i-th grid node V i The velocity of the nearby object is: （5） After simulating virtual tactile deformation using the material point method, a force acquisition algorithm is used to collect tactile data from the deformation of the elastic body, thereby generating a virtual tactile image; specifically: The sensor is deployed on the base of the deformable body. Considering any force acquisition point, a ray is emitted from that point as the origin towards the normal direction facing the sensor towards the deformable body. This ray passes through several voxels of the deformable body. The information carried by these voxels during their motion comprehensively reflects the force information at that point. The force is synthesized using the following formula: （6） Assuming the ray passed through a total of Individual factors; and Let the mass and velocity of the i-th voxel be denoted as . This is for simulating step size; is the equivalent elastic modulus of the material in the normal direction; The indentation displacement in the ray direction of the voxel farthest from the sampling point compared to before the simulation started; and For a pair of empirically based harmonic parameters; Therefore, the force distribution collected by the tactile sensor is represented as follows: ； Each element in the matrix is ​​obtained using formula (6); The collected force matrix is ​​presented in grayscale image form, and the grayscale value of each sampling point is calculated as follows: ； in, This is the maximum grayscale value, usually set to 255. Let be the force at the i-th sampling point.

4. The one-step snapping prediction method for adaptive gradient soft flatbed cable based on virtual-real fusion as described in claim 1, characterized in that: The specific steps of the virtual-real tactile image comparison model are as follows: 1) Data import: The adjustment step size and direction obtained by comparing the real tactile image obtained from the adaptive model with the ideal real tactile image, and the adjustment step size and direction obtained by comparing the virtual tactile image with the ideal virtual tactile image, are input into the virtual-real tactile image comparison model; 2) Training of the neural network: After obtaining the virtual and real tactile adjustment strategy, the neural network needs to be trained in the digital twin environment to obtain the final network weights and determine the confidence of the virtual and real tactile adjustment strategy. 3) Execution of the final strategy: Determine the final adjustment strategy based on the confidence level of the virtual and real tactile adjustment strategy, feed it back into the real environment, and adjust the robotic arm to the position before successful engagement through one adjustment.

Images