Multi-dimensional force decoupling method and system, storage medium and computer program product

Abstract

The invention provides a multi-dimensional force decoupling method and system, a storage medium and a computer program product, and the method comprises the steps: obtaining at least one sensing feature which is used for representing the deformation of a contact surface; at least one sensing feature is input into a decoupling model to output each force component of the multi-dimensional force, a total loss function of the decoupling model is configured as a combination of a plurality of sub-losses, and each sub-loss corresponds to a prediction error of one force component. The invention provides a scheme for improving the accuracy of multi-dimensional force decoupling.

Description

Technical Field

[0001] This application relates to the field of mechanical sensing technology, and in particular to a multidimensional force decoupling method and system, storage medium, and computer program product. Background Technology

[0002] In many cutting-edge applications such as robotics, biomechanical analysis, and human-computer interaction, accurately measuring the multi-dimensional forces on the contact interface of an object, that is, including the normal force perpendicular to the contact surface and the shear force parallel to the contact surface, is a core prerequisite for realizing complex operations and fine perception.

[0003] However, since the components of force in different dimensions can interfere with each other and produce crosstalk effects, accurately decoupling multidimensional forces has always been a recognized technical challenge in this field. Summary of the Invention

[0004] This application provides a solution to improve the accuracy of multidimensional force decoupling.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] Firstly, a multidimensional force decoupling method is provided, which is applied to a terminal device, or a chip or chip module in a terminal device. The multidimensional force decoupling method includes: acquiring at least one sensing feature, the sensing feature being used to characterize the deformation of the contact surface; inputting the at least one sensing feature into a decoupling model to output the force components of the multidimensional force, wherein the total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

[0007] Optionally, the decoupling model includes multiple regression heads, each used to predict a force component, and each regression head corresponds to a sub-loss.

[0008] Optionally, the total loss function further includes a feature similarity loss, which is used to characterize the difference in output features of at least one corresponding intermediate layer among the plurality of regression heads.

[0009] Optionally, the decoupling model further includes a backbone network, which includes multiple pathways, each pathway being used to process a sensing feature. The backbone network fuses the feature vectors output by the multiple pathways and outputs the fused feature vector to the multiple regression heads.

[0010] Optionally, acquiring at least one sensing feature includes: acquiring multiple sensing features, wherein the multiple sensing features have different sensitivities to different force components.

[0011] Optionally, the multiple sensing features include at least two of the following: optical flow matrix, color image of the contact surface, grayscale image of the contact surface, HSV image, and arrow diagram.

[0012] Optionally, the sensing features can be of multiple types, and inputting the at least one sensing feature into the decoupling model includes: fusing multiple sensing features and inputting the fused sensing features into the decoupling model.

[0013] Secondly, this application also discloses a multidimensional force decoupling device, which includes: an acquisition module for acquiring at least one sensing feature, the sensing feature being used to characterize the deformation of the contact surface; and a decoupling module for inputting the at least one sensing feature into a decoupling model to output the force components of the multidimensional force, wherein the total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

[0014] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to perform the method provided in the first aspect.

[0015] Fourthly, a force sensing device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to perform a method provided in the first aspect.

[0016] Fifthly, a computer program product is provided, on which a computer program is stored, the computer program being executed by a processor to perform the method provided in the first aspect.

[0017] Sixthly, embodiments of this application also provide a chip that stores a computer program, which, when executed by the chip, implements the steps of the above-described method.

[0018] In a seventh aspect, embodiments of this application also provide a system chip for use in a terminal. The system chip includes at least one processor and an interface circuit. The interface circuit and the at least one processor are interconnected via a line. The at least one processor is used to execute instructions to perform a method provided in the first aspect.

[0019] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0020] In this application's technical solution, at least one sensing feature is input into a decoupling model. The pre-trained decoupling model then decouples multidimensional forces, obtaining each force component. Since the total loss function of the decoupling model is configured as a combination of multiple sub-losses, with each sub-loss corresponding to the prediction error of a force component, independent prediction of each force component is achieved, improving the accuracy of multidimensional force decoupling.

[0021] Furthermore, in the technical solution of this application, the decoupling model includes multiple regression heads, each used to predict a force component, and the regression heads share at least a portion of the model parameters. The output of the neural network model in this technical solution is designed as multiple independent output heads. Each output head is an independent regression task, specifically responsible for predicting a specific component in a multidimensional force vector; for example, one regression head predicts pressure, and another predicts shear force. Through this multi-task regression output architecture, structured learning of the coupling relationship between force components is achieved, reconstructing the complex multidimensional force decoupling problem into a parallel multi-task regression learning problem. Because each regression task becomes simpler and its objective more clearly defined, the network can learn the mapping from visual features to individual force components more stably and accurately.

[0022] Furthermore, in the technical solution of this application, the multi-task regression output architecture allows for the setting of feature sharing or feature fusion mechanisms before the final output results of different regression tasks (output heads). This means that the network not only predicts each force component in isolation, but also actively learns and understands the inherent coupling relationship between different force components in a shared feature space. It can learn from the data how the characteristics of shear force change accordingly when the normal pressure changes (and vice versa). In this way, the model transforms the "coupling" between multidimensional forces from an interference term that needs to be suppressed into useful information that can be utilized and reflects physical laws, thereby achieving a deeper and more accurate decoupling.

[0023] Furthermore, in the technical solution of this application, since different sensing features have significantly different sensitivities to different mechanical components, by inputting multiple types of sensing features into the decoupling model in parallel, the maximum and optimal utilization of sensing information is achieved, thereby improving the accuracy of multidimensional force decoupling.

[0024] Furthermore, this application systematically solves the bottlenecks of existing technologies from two dimensions: information utilization and model structure, through multi-path optimization at the input end and task decomposition and coordination at the output end, thereby constructing a more efficient, robust and accurate multi-dimensional force decoupling method. Attached Figure Description

[0025] Figure 1 This is a flowchart of a multidimensional force decoupling method provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the structure of a force sensing device provided in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of a decoupling model provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of various sensing features provided in the embodiments of this application;

[0029] Figure 5 This is a schematic diagram of the structure of a multidimensional force decoupling device provided in an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of the hardware structure of a multidimensional force decoupling device provided in an embodiment of this application. Detailed Implementation

[0031] As described in the background section, since the components of forces in different dimensions can interfere with each other and produce crosstalk effects, accurately decoupling multidimensional forces has always been a recognized technical challenge in this field.

[0032] This application systematically solves the bottlenecks of existing technologies from two dimensions: information utilization and model structure, through multi-path optimization at the input end and task decomposition and coordination at the output end, thereby constructing a more efficient, robust and accurate multi-dimensional force decoupling method.

[0033] The force component (also referred to as mechanical component) in this embodiment refers to the force in various directions perpendicular to and parallel to the contact surface. For example, the force component may include: normal force perpendicular to the contact surface and shear force in multiple directions parallel to the contact surface.

[0034] The decoupling model referred to in this embodiment can be a neural network model, specifically a deep learning model, and can be constructed using any appropriate algorithm. This application does not impose any restrictions on this.

[0035] In this embodiment, the contact surface refers to the plane on which an external object applies force. Specifically, it can be the surface of the sensing layer, which has embedded visual markers (markers) on its surface or internally embedded. Specifically, an image sensor, such as an RGB camera, is typically deployed below the sensing layer to capture the state of the markers in real time. When an external object contacts the sensing layer and applies force, the flexible sensing layer deforms, causing the internal markers to shift. The image sensor below captures this deformation process and records it as a digital image or video stream.

[0036] The sensing features referred to in this embodiment are features used to characterize contact deformation information. Specifically, the raw image data acquired by the sensing device is processed to extract sensing features that can effectively characterize deformation information. Sensing features can take various forms, such as directly using the raw color or grayscale image, or they can be calculated using optical flow algorithms, describing the optical flow matrix or its visualization form that can represent the motion vector field of the marker.

[0037] Different sensing features represent visual sensing feature data with different physical meanings. For example, sensing feature 1 is used to reflect the structural deformation of the contact surface (such as a grayscale image), while sensing feature 2 is used to reflect the vector displacement of the contact surface (such as an arrow image or an optical flow field).

[0038] All user data involved in the technical solution of this application has been obtained with user authorization. For example, the user is clearly informed of the purpose, scope and permissions of the data through the interactive interface, and valid authorization is obtained after the user actively checks the box to agree or completes the electronic signature.

[0039] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0040] This application provides a multidimensional force decoupling method, referring to... Figure 1 The following will provide a detailed explanation through specific steps.

[0041] It is understood that, in specific implementations, the multidimensional force decoupling method can be implemented using software programs, which run within a processor integrated into the chip or chip module. This method can also be implemented using a combination of software and hardware; this application does not impose any limitations on this approach.

[0042] Step 101: Acquire at least one sensing feature, which is used to characterize the deformation of the contact surface.

[0043] Step 102: Input at least one sensing feature into the decoupling model to output the force components of the multidimensional force. The total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

[0044] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0045] The multidimensional force decoupling method in this embodiment can be used in force sensing devices (also known as visual-tactile sensors or vision-based force sensing platforms). The force sensing device can acquire at least one sensing feature and output the decomposed force components. Please refer to [link / reference] for details. Figure 2The force sensing device may include a sensing module 201, a data acquisition module 202, a data processing module 203, and a decoupling model 204. The following section describes the multidimensional force decoupling method using the force sensing device as an example.

[0046] The sensing module 201 can specifically be a flexible sensing layer serving as the contact surface. This sensing layer contains or has a large number of trackable visual markers embedded within it. When an external object comes into contact with the sensing layer and applies a force, the flexible sensing layer deforms, causing the internal markers to shift. This force is typically a resultant force, which will subsequently be decomposed into individual force components.

[0047] The data acquisition module 202 can be located below the sensing module 201, i.e., the sensing layer. Specifically, it can be an image sensor, such as an RGB camera, used to capture the state of the marker in real time. When the flexible sensing layer deforms, the data acquisition module 202 can capture this deformation process and record it as a digital image or video stream.

[0048] The data processing module 203 can process the digital images or video streams acquired by the data acquisition module 202 to extract sensing features that can effectively characterize deformation information. Sensing features can take various forms, such as directly using the original color or grayscale images, or they can be calculated using optical flow algorithms, describing the optical flow matrix or its visualization of the marker's motion vector field.

[0049] Specifically, the sensing features can be selected from: optical flow matrix, color image of the contact surface, grayscale image of the contact surface, HSV image, and arrow image. Please refer to [reference needed]. Figure 4 , Figure 4 It demonstrates a variety of sensing features. Figure 4 This demonstrates the performance of different sensing characteristics under the same external force load. The two images on the left and right show different magnitudes of shear stress under a 10N pressure. The shear stress in the left image is 3.6N, while that in the right image is 5N.

[0050] In typical computer vision tasks, optical flow usually refers to the relative motion of objects, surfaces, or edges in a visual scene. Optical flow is a vector field estimated from a sequence of 2D images, representing the size and orientation of moving objects, and can be used for motion detection, object segmentation, and scene understanding. Mathematically, optical flow estimation is based on the principle of constant brightness, meaning that pixel intensity remains constant between frames. To improve detection accuracy (such as capturing complex multidimensional force interactions like foot-to-ground contact forces in gait), dense optical flow methods based on the Dense Inverse Search (DIS) algorithm can be used. The DIS algorithm estimates the per-pixel displacement vector field describing the motion of individual pixels in an image, providing a complete and detailed representation of the motion of the entire sensing layer. For each external force load, the sensing layer image is compared with the unloaded image using the DIS algorithm to calculate a multidimensional matrix (size 480×640×2), which is the optical flow matrix. The last dimension represents the size and orientation of the optical flow field. Displacement is measured in pixels, and the optical flow direction is represented in polar coordinates (0-2π). Specifically, a dense optical flow estimation algorithm (such as the DIS algorithm) is used to calculate the displacement vector field, i.e., the optical flow matrix, of each pixel in the image by comparing the current stressed image with a pre-stored reference image under no-load conditions. Then, to make the features more suitable for neural network learning, the dense optical flow matrix is ​​sparsified and visualized, for example, by sampling at intervals of 20 pixels, and the displacement vector (including magnitude and direction) of each sampled point is plotted as an arrow to generate the final arrow map.

[0051] Visualizing the optical flow field is crucial for understanding the motion patterns of the sensing layer caused by external forces. The original optical flow matrix is ​​difficult to interpret directly due to its high-dimensional and complex structure. Therefore, the optical flow field can be converted into the more intuitive HSV (hue, saturation, brightness) color space, i.e., an HSV image (…). Figure 4 As shown in Figure a). Hue (H) represents the direction of motion (represented by a cyclic angle), saturation (S) is fixed at a maximum value to ensure vibrant colors, and lightness (V) encodes the amplitude of motion (the larger the value, the greater the displacement). Specifically, the lightness (V) is obtained by summing and normalizing the pixel motion vectors (u,v), the motion direction is calculated using arctan and normalized to obtain hue (H), and saturation is set to 1. Finally, the HSV image is converted to the RGB color space (size 480×640×3). Figure 4 As shown in Figure a, the HSV diagram visually displays the differences in direction and intensity of the motion field in the sensing layer. The variations in saturation and hue range of the red and purple tones in the right figure accurately reflect the greater shear stress.

[0052] Optical flow is itself a dense vector field, but it can be represented by arrows with magnitude and direction characteristics, thus conveying complex motion information in a simple and intuitive way. For example... Figure 4The arrow diagram shown in Figure b visualizes the dense optical flow field by sampling at 20-pixel intervals. In the left figure, larger pressures exhibit a more centrally symmetrical arrow distribution, while larger shear stresses in the right figure show a more pronounced axisymmetric pattern.

[0053] Color images ( Figure 4 (As shown in c) can be the original image acquired by the data acquisition module 202. Deep learning methods can directly extract features from the original image to achieve multidimensional force detection. For example... Figure 4 As shown in Figure c, particles distributed across multiple layers (red / purple / white) undergo interlayer relative motion under stress. Smaller shear forces only cause movement in the surface white particles; as the force increases, the middle purple particles and the bottom red particles begin to shift. Variations in shear stress under different pressures lead to significant interlayer shear displacement.

[0054] To eliminate the interference of color differences on subtle features, the original image can also be converted to a grayscale image, such as... Figure 4 As shown in Figure d, the binarized grayscale image represents the projection of markers at different depths onto the focal plane. Using grayscale features can not only verify the feasibility of force decoupling based on the original particle image, but also evaluate the effectiveness of the depth information of the colored particle layer.

[0055] Continue to refer to Figure 2 The data processing module 203 outputs at least one sensing feature to the decoupling model 204. The decoupling model 204 can decouple forces and output each force component. Based on the input one or more sensing features, the decoupling model 204 accurately infers the component values ​​of the force in each dimension and the location of the point of application. In a specific implementation, the decoupling model 204 can be a trained neural network model.

[0056] In this embodiment, the total loss function of the decoupling model is configured as a combination of multiple sub-losses, each sub-loss corresponding to the prediction error of a force component. That is, the total loss function of the decoupling model is a composite loss, which is composed of a weighted or unweighted combination of at least two independent sub-losses. Therefore, it achieves independent prediction of each force component and improves the accuracy of multidimensional force decoupling.

[0057] For example, the two sub-losses correspond to pressure and shear force, respectively. This embodiment emphasizes specialization and efficiency by separating the processing of pressure and shear stress characteristics into different branches, thereby enhancing the network's ability to capture unique force characteristics.

[0058] In a non-limiting embodiment, please refer to Figure 3 , Figure 3A decoupling model is illustrated. The decoupling model includes multiple regression heads, each used to predict a force component. This embodiment uses arrow diagrams and grayscale images as sensing features, and illustrates the decoupling model with two regression heads as an example.

[0059] In this embodiment, the decoupled model includes an input layer, a feature extraction layer 301, a regression head (also known as an output head) 302, and a regression head 303. The input layer is used to receive sensing features. The input layer and the feature extraction layer 301 can be referred to as the backbone network.

[0060] The feature extraction layer 301 comprises multiple pathways, each processing a specific sensor feature. These pathways can process various sensor features in parallel. While the pathways in feature extraction layer 301 share the same structure, their parameters (weights) are independent. Specifically, during decoupled model training, each pathway can perform gradient updates independently, allowing each pathway to learn the most suitable features for its input data type. For instance, the pathway for processing grayscale images focuses on extracting spatial features such as texture and edges, while the pathway for processing optical flow maps focuses on extracting temporal dynamic features such as motion and deformation direction.

[0061] Specifically, please refer to Figure 3 Each path of the feature extraction layer 301 includes three convolutional layers, one max pooling layer, three convolutional layers, one max pooling layer, three convolutional layers, one max pooling layer, one flatten layer, and one fully connected layer (dense layer).

[0062] It should be noted that the number of convolutional layers, pooling layers, flattening layers and fully connected layers in this embodiment is only an example. The specific number can be set according to the actual application scenario, and this application does not limit it.

[0063] For each sensor feature, the feature data is sequentially passed through three convolutional layers, followed by a max-pooling layer. This sequence is repeated three times. Afterward, the features are flattened by a flattening layer and then processed by a fully connected layer. The two sensor features (arrow image and grayscale image) are then fused after passing through the fully connected layer.

[0064] Specifically, the features output from multiple pathways of the feature extraction layer 301 can be fused using element-wise summation. This involves adding multiple feature vectors with the same dimensions at corresponding positions to form a new feature vector that includes all modal information. This fusion method effectively combines static spatial information, dynamic change information, and other features to create a more comprehensive and robust description of the current stress state.

[0065] Specifically, the fused feature vector will serve as the sole input for subsequent decoupling tasks. The fused features are then input to regression heads 302 and 303 respectively for force component prediction. Regression head 302 outputs normal force, and regression head 303 outputs shear force. Regression heads 302 and 303 perform different regression tasks. Regression heads 302 and 303 have the same structure but independent parameters (weights).

[0066] like Figure 3 As shown, regression heads 302 and 303 include three convolutional layers, one max pooling layer, one flattening layer, and one fully connected layer.

[0067] In this embodiment, the decoupling model decomposes the force decoupling task into multiple independent subtasks, each handled by a separate output head. The internal parameters (weights) of different regression heads are independent of each other, allowing each regression head to focus on learning the specific mapping relationship of its respective task, thereby significantly reducing the mutual interference of different physical quantities during the calculation process.

[0068] For example, regression header 302 and regression header 303 can be selected from:

[0069] Pressure Head: A separate subnetwork used to regress positive pressure (Pressure or Fz);

[0070] Shear Force Head: An independent subnetwork used to regress shear stress or two shear force components (Fx, Fy);

[0071] Torque Head: A separate subnetwork used to regress torsional torque (Torsion or Mz).

[0072] The above method is highly flexible and can be configured with different regression heads according to measurement needs.

[0073] This embodiment creatively decomposes a complex multidimensional force decoupling problem into a set of parallel, independent regression tasks targeting a single force component (such as pressure, X-axis shear force, Y-axis shear force, etc.) at the algorithmic level. This simplifies each regression task and makes the objective more explicit, enabling the network to learn the mapping from visual features to individual force components more stably and accurately.

[0074] In a preferred embodiment, in order to ensure that multiple regression tasks can still utilize common underlying physical information while being separated, since all forces and torques originate from the same physical contact event, a feature sharing mechanism can be introduced to ensure the physical correlation of each force component, avoid the information loss caused by completely independent regression tasks, and improve the generalization ability of the model.

[0075] In this embodiment, feature sharing is achieved by adding an auxiliary loss term, namely feature similarity loss, to the total loss function of the model.

[0076] In one specific implementation, the feature similarity loss (Ls) can be used to measure the difference in output features of at least one corresponding intermediate layer in any two or more regression heads. For example, this difference can be represented using cosine similarity or the reciprocal of the Euclidean distance. During model training, minimizing the feature similarity loss constrains each regression head to maintain a certain level of similarity and correlation while learning its own specific features. That is, the weights of at least one corresponding intermediate layer in any two or more regression heads should be the same or close, where closeness means the error is less than a certain preset threshold.

[0077] For example, please refer to Figure 3 Feature similarity loss can characterize the difference in output features of the first convolutional layer in regression head 302 and regression head 303.

[0078] In another specific embodiment, at least one intermediate layer, such as a fully connected layer, in regression head 302 and regression head 303 is the same.

[0079] In another specific implementation, the output of one regression head is used as one of the input information for another regression head to achieve information flow between tasks and assist decision-making. That is, the output of regression head 302 can be used as the input of regression head 303, or the output of regression head 303 can be used as the input of regression head 302.

[0080] It should be noted that, depending on the number of force components to be output, the decoupling model may also have three or more regression heads, and this application does not impose any restrictions on this.

[0081] In a non-limiting embodiment, the input to the decoupling model includes two or more sensing features. For example... Figure 3 As shown, the decoupling model includes a feature extraction layer that extracts features for each sensing feature. The decoupling model extracts features separately for each sensing feature and then performs feature fusion.

[0082] This embodiment employs a neural network architecture that simultaneously receives and processes two or more different types of visual sensing features as parallel inputs. This architecture allows the model to learn from the optimal feature sources separately based on the characteristics of different force components, for example, using a two-branch or multi-branch structure. This enables the model to simultaneously learn the static spatial distribution of forces, dynamic application trends, and other potential information, thereby obtaining richer and more discriminative information than with a single input.

[0083] In an optimal combination, the sensing features of the input decoupling model include arrow maps and grayscale images. The arrow map, primarily reflecting the arrows in the marker's motion vector field, exhibits the highest sensitivity to changes in shear force and is the optimal feature for decoupling shear force. The grayscale image, having removed color information and better highlighting marker structure and texture variations, is most sensitive to changes in normal pressure and is the optimal feature for decoupling pressure. This specific combination demonstrated extremely high decoupling accuracy in the tests conducted in this invention (shear force linearity 0.961, pressure linearity 0.981).

[0084] In this embodiment, the arrow image and grayscale image are processed separately through dedicated channels. Therefore, when decoupling shear force, the decoupling model primarily relies on the arrow image, which has the richest information; while when decoupling normal force, it primarily relies on the grayscale image, which has the richest information. This fundamentally solves the information bottleneck caused by a single feature, ensuring that each force component decoupling task receives optimal quality information input, thereby significantly improving decoupling performance.

[0085] It should be noted that the combination of sensing features in the input decoupling model can also be any other type or an appropriate number of sensing features, and this application does not impose any restrictions on this.

[0086] In a non-limiting embodiment, multiple sensing features can be directly input into the decoupled model.

[0087] In another non-limiting embodiment, multiple sensing features can be fused, and the fused sensing features can be input into the decoupled model.

[0088] For example, a single-channel grayscale image and two channels representing arrow diagram information (such as the amplitude and angle channels of optical flow) are stacked to form a pseudo-color image containing three or more channels, which is then input into the decoupling model.

[0089] In this embodiment, the input data tensor input to the decoupled model is composed of at least two sensing features with different physical meanings. For example, one feature mainly reflects the structural deformation of the contact surface (such as a grayscale image), while the other feature mainly reflects the vector displacement of the contact surface (such as an arrow image or an optical flow field).

[0090] In one specific embodiment, the decoupling model needs to be trained first, and then used to perform actual multidimensional force component decoupling. During the model training phase, sensor calibration needs to be completed and a high-precision decoupling model needs to be trained; in the online real-time prediction phase, the trained model is used to perform real-time multidimensional force decoupling.

[0091] During the model training phase, a high-quality dataset needs to be created, and a decoupling model capable of accurately decoupling multidimensional forces needs to be trained based on this dataset. The system used to build the dataset may include: a multi-degree-of-freedom industrial robot arm, a high-precision six-axis force / torque sensor, a rigid indenter for applying force, and force sensing devices.

[0092] Specifically, a rigid pressure head, controlled by a multi-degree-of-freedom industrial robot arm, applies a series of precisely known forces to the flexible sensing layer of a force-sensing device in a preset pattern to simulate real-world application scenarios. For example, simultaneously increasing pressure and shear force simulates the "heel-to-ground" phase of the foot, or increasing shear force under constant pressure simulates the "toe-off" phase. At the instant each force is applied, a high-precision six-axis force sensor records the ground truth label of the current force, which is typically a vector containing normal force and X / Y axis shear force. While the six-axis force sensor records the ground truth value, a camera below the force-sensing device captures an image frame. The system immediately processes this image to generate at least one sensing feature, such as an arrow diagram and a grayscale image.

[0093] Repeat the force-acquisition process described above, for example, to collect 4300 sets of data. Pair at least one sensing feature acquired each time with its corresponding ground truth force label and store them to form a dataset. Use this dataset to train the pre-built decoupling model.

[0094] In this embodiment, the decoupling model learns from the above dataset to establish a complex nonlinear mapping relationship from visual features to multidimensional force vectors.

[0095] Please refer to Figure 5 , Figure 5 A multidimensional force decoupling device 50 is shown, which may include:

[0096] The acquisition module 501 is used to acquire at least one sensing feature, which is used to characterize the deformation of the contact surface;

[0097] The decoupling module 502 is used to input at least one sensing feature into the decoupling model to output the force components of the multidimensional force. The total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

[0098] In specific implementation, the aforementioned multidimensional force decoupling device 50 may correspond to a chip with force decoupling function in a force sensing device, such as a system-on-a-chip (SOC), a baseband chip, etc.; or to a chip module in a force sensing device that includes a force decoupling function; or to a chip module with a data processing function chip; or to a force sensing device.

[0099] Other relevant descriptions of the multidimensional force decoupling device 50 can be found in the descriptions in the foregoing embodiments, and will not be repeated here.

[0100] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on the processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented through a software program that runs on the processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0101] This application also discloses a storage medium, which is a computer-readable storage medium storing a computer program thereon. When the computer program is executed, it can perform the steps of the method shown in the foregoing embodiments. The storage medium may include read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0102] Please refer to Figure 6 This application also provides a schematic diagram of the hardware structure of a communication device. The device includes a processor 601, a memory 602, and a transceiver 603.

[0103] Processor 601 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program according to the present application. Processor 601 may also include multiple CPUs, and processor 601 can be a single-core processor or a multi-core processor. Here, processor can refer to one or more devices, circuits, or processing cores used to process data (e.g., computer program instructions).

[0104] The memory 602 can be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. This application embodiment does not impose any limitations on this. The memory 602 can exist independently (in this case, the memory 602 can be located outside or inside the device) or it can be integrated with the processor 601. The memory 602 may contain computer program code. The processor 601 is used to execute the computer program code stored in the memory 602, thereby implementing the method provided in this application embodiment.

[0105] The processor 601, memory 602, and transceiver 603 are connected via a bus. The transceiver 603 is used to communicate with other devices or communication networks.

[0106] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0107] In the embodiments of this application, "multiple" refers to two or more.

[0108] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0109] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0110] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0111] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0115] The integrated unit implemented as a software functional unit described above can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of this application.

[0116] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. A multidimensional force decoupling method, characterized in that, include: Acquire at least one sensing feature, said sensing feature being used to characterize the deformation of the contact surface; The at least one sensing feature is input into the decoupling model to output the force components of the multidimensional force. The total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

2. The multidimensional force decoupling method according to claim 1, characterized in that, The decoupling model includes multiple regression heads, each used to predict a force component, and each regression head corresponds to a sub-loss.

3. The multidimensional force decoupling method according to claim 2, characterized in that, The total loss function also includes feature similarity loss, which is used to characterize the difference in output features of at least one corresponding intermediate layer among the plurality of regression heads.

4. The multidimensional force decoupling method according to claim 2, characterized in that, The decoupling model also includes a backbone network, which includes multiple pathways, each of which processes a sensing feature. The backbone network fuses the feature vectors output by the multiple pathways and outputs the fused feature vector to the multiple regression heads.

5. The multidimensional force decoupling method according to claim 1, characterized in that, The acquisition of at least one sensing feature includes: Multiple sensing features are acquired, and these multiple sensing features have different sensitivities to different force components.

6. The multidimensional force decoupling method according to claim 5, characterized in that, The multiple sensing features include at least two of the following: optical flow matrix, color image of the contact surface, grayscale image of the contact surface, HSV image, and arrow diagram.

7. The multidimensional force decoupling method according to claim 1, characterized in that, There are various types of sensing features, and the input of at least one sensing feature into the decoupling model includes: Multiple sensing features are fused, and the fused sensing features are input into the decoupled model.

8. A multidimensional force decoupling device, characterized in that, include: An acquisition module is used to acquire at least one sensing feature, the sensing feature being used to characterize the deformation of the contact surface; The decoupling module is used to input the at least one sensing feature into the decoupling model to output the force components of the multidimensional force. The total loss function of the decoupling model is configured as a combination of multiple sub-losses, and each sub-loss corresponds to the prediction error of a force component.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the multidimensional force decoupling method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the multidimensional force decoupling method according to any one of claims 1 to 7.

11. A force sensing device, comprising a storage module and a processing module, wherein the storage module stores a computer program executable on the processing module, characterized in that, When the processing module runs the computer program, it performs the steps of the multidimensional force decoupling method according to any one of claims 1 to 7.

Images