Methods for determining the deformation stiffness of flexible bodies and methods for correcting the simulation model of flexible bodies.

Abstract

This invention relates to the field of measurement technology, specifically providing a method for determining the stiffness parameters of a flexible body and a method for correcting a simulation model of a flexible body, aiming to solve the problem that existing flexible body morphology measurement technologies cannot measure physical properties of flexible bodies such as bending stiffness and torsional stiffness. To this end, the method for determining the stiffness of a flexible body according to this invention includes: reconstructing the three-dimensional centerline of the flexible body based on multi-view images of the flexible body in a robotic arm clamping state; acquiring deformation rate information distributed along the arc length of the three-dimensional centerline; collecting force information of the flexible body in the robotic arm clamping state based on sensors installed on the robotic arm; and determining the deformation stiffness information of the flexible body based on the deformation rate information and the force information.

Description

Technical Field

[0001] This invention relates to the field of measurement technology, specifically providing a method for determining the deformation stiffness of a flexible body and a method for correcting a simulation model of a flexible body. Background Technology

[0002] A flexible body is an object that can deform under the action of external forces and has a stress-strain relationship.

[0003] Currently, flexible body morphology measurement technology can use industrial cameras to measure planar dimensions (such as length / diameter) or employ binocular structured light cameras to acquire point cloud data of the flexible body, and then obtain the dimensional parameters of the flexible body based on the point cloud data. Existing flexible body morphology measurement technology can only acquire the geometric appearance parameters of the flexible body and cannot measure the physical properties of the flexible body, such as bending stiffness and torsional stiffness, under non-contact conditions, resulting in measurement results that cannot be used for the calibration of simulation parameters of the flexible body. Summary of the Invention

[0004] The present invention aims to solve the above-mentioned technical problems, namely, the problem that existing flexible body morphology measurement technologies cannot measure physical properties of flexible bodies such as bending stiffness and torsional stiffness.

[0005] In a first aspect, the present invention provides a method for determining the deformation stiffness of a flexible body, comprising:

[0006] Based on multi-view images of the flexible body in the gripping state of the robotic arm, the three-dimensional centerline of the flexible body is reconstructed.

[0007] Obtain the deformation rate information distributed along the arc length of the three-dimensional centerline;

[0008] Based on the sensors installed on the robotic arm, the force information of the flexible body under the gripping state of the robotic arm is collected;

[0009] Based on the deformation rate information and the force information, the deformation stiffness information of the flexible body is determined.

[0010] In some embodiments of the present invention, reconstructing the three-dimensional centerline of the flexible body based on multi-view images of the flexible body in the gripping state of the robotic arm includes:

[0011] Semantic segmentation is performed on the multi-view images, and the skeleton information of the flexible body is determined based on the semantic segmentation results;

[0012] Based on the skeleton information of the flexible body, a three-dimensional reconstruction is performed to obtain a three-dimensional reconstruction model of the flexible body.

[0013] Based on the three-dimensional reconstruction model of the flexible body, the three-dimensional centerline of the flexible body is determined.

[0014] In some embodiments of the present invention, the step of performing three-dimensional reconstruction based on the skeleton information of the flexible body to obtain a three-dimensional reconstruction model of the flexible body includes:

[0015] Using a preset global cost function, a combination of positive and negative multi-view curves is generated based on the skeleton information of the flexible body;

[0016] The combination with the smallest reprojection error is selected from the positive and negative combinations of the multi-view curves, and three-dimensional reconstruction is performed using spline interpolation to obtain the three-dimensional reconstruction model of the flexible body.

[0017] In some embodiments of the present invention, the step of performing three-dimensional reconstruction based on the skeleton information of the flexible body to obtain a three-dimensional reconstruction model of the flexible body includes:

[0018] In response to the fact that the flexible body is a bifurcated flexible body with a bifurcation structure, the endpoints and bifurcation points of the bifurcated flexible body are determined based on the skeleton information;

[0019] Based on the endpoints and bifurcation points of the bifurcation flexible body, a multi-segment sub-curve of the bifurcation flexible body is generated;

[0020] The multiple sub-curves are reconstructed and stitched together in three dimensions between the multiple perspectives corresponding to the multi-view images to obtain the three-dimensional reconstruction model of the bifurcated flexible body.

[0021] In some embodiments of the present invention, determining the deformation stiffness information of the flexible body based on the deformation rate information and the force information includes:

[0022] The mechanical equilibrium relationship of the flexible body is constructed based on a preset model;

[0023] Based on the mechanical equilibrium relationship, the deformation rate information, and the force information, the deformation stiffness information of the flexible body is obtained by inversion.

[0024] In some embodiments of the present invention, the preset model is a Cosserat rod model or an equivalent flexible body mechanical model.

[0025] In some embodiments of the present invention, it further includes:

[0026] During the movement of the flexible body held by the robotic arm, the maximum deflection point of the flexible body is located in real time, and a curve of the coordinates of the maximum deflection point changing over time is generated.

[0027] In some embodiments of the present invention, the multi-view images are acquired by a non-collinear multi-view visual acquisition device, which includes at least three cameras arranged non-collinearly in space. The optical axes of the at least three cameras form a non-collinear spatial distribution, and a unified visual coordinate system is established through joint extrinsic parameter calibration.

[0028] In a second aspect of the present invention, a method for correcting a flexible body simulation model is provided, comprising:

[0029] The deformation stiffness information of the flexible body is determined according to the method described in the first aspect above;

[0030] The deformation stiffness information of the flexible body is provided to the flexible body simulation model, and the actual measurement results and simulation results of the flexible body under preset motion conditions are obtained.

[0031] Based on the difference between the simulation results and the actual measurement results, the parameters of the flexible body simulation model are adjusted.

[0032] In some embodiments of the present invention, obtaining the actual measurement results of the flexible body under preset motion conditions includes:

[0033] Control the robotic arm to interact with the mechanism under test along a preset trajectory;

[0034] During the interactive action performed by the robotic arm, force information during the interaction is collected by sensors installed on the robotic arm, and the posture information of the robotic arm at a preset position is obtained.

[0035] Based on the force information and the attitude information, a mechanical curve characterizing the mechanical properties of the interaction process is generated.

[0036] In some embodiments of the present invention, the mechanical curves include force-time curves and / or force-displacement hysteresis curves.

[0037] A third aspect of the present invention provides an electronic device comprising:

[0038] Memory, used to store computer program products;

[0039] A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, to implement the method described in the first or second aspect above.

[0040] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method described in the first or second aspect above.

[0041] A fifth aspect of the present invention provides a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the method described in the first or second aspect.

[0042] By employing the aforementioned technical solutions, this invention combines visual and mechanical analysis to easily measure the stiffness parameters of flexible bodies using a non-contact method, overcoming the limitation of existing technologies that cannot invert the physical properties of flexible bodies through visual data under non-contact conditions. Furthermore, for bifurcate flexible bodies, this invention uses a bifurcation point topology matching algorithm to effectively solve the problems of self-occlusion and 3D reconstruction fracture in complex wire harnesses. Moreover, this invention, through the reverse injection of real data, endows the simulation environment with realistic physical properties, improving the authenticity and reliability of the simulation data. Attached Figure Description

[0043] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, in which:

[0044] Figure 1 This is a flowchart illustrating the method for determining the stiffness parameters of a flexible body in some embodiments of the present invention;

[0045] Figure 2 This is a schematic diagram of the hardware structure for determining the deformation stiffness of a flexible body in one example of the present invention;

[0046] Figure 3 This is a schematic diagram of extracting a flexible skeleton from multi-view images in one example of the present invention;

[0047] Figure 4 This is a schematic diagram of decoupling a bifurcated flexible body into multiple sub-curves in one example of the present invention;

[0048] Figure 5 This is a schematic diagram of the present invention for reconstructing the three-dimensional centerline of a single flexible body without a branching structure;

[0049] Figure 6 This is a schematic diagram of the three-dimensional centerline of a bifurcated flexible body in one example of the present invention;

[0050] Figure 7 This is a flowchart illustrating the correction method for a flexible body simulation model in some embodiments of the present invention;

[0051] Figure 8 These are structural block diagrams of electronic devices in some embodiments of the present invention. Detailed Implementation

[0052] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0053] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0054] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0055] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0056] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0057] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0058] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0059] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0060] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0061] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0062] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0063] Here we will first explain some of the terms used in this application.

[0064] Cosserat beam model: a continuum mechanics model based on geometrically accurate beam theory. The Cosserat model simultaneously considers the displacement of the centerline and the independent rotation of the cross section, and can accurately describe the large deformation, bending, torsion and shear behavior of slender flexible bodies in three-dimensional space.

[0065] Non-collinear multi-view visual acquisition device: A visual measurement system comprising at least three visual acquisition units whose optical axes are non-collinearly distributed in space, for acquiring image information from different spatial directions.

[0066] Morphological erosion and skeletonization: refers to a digital image processing technique that peels away the boundary pixels of foreground objects in a binary image layer by layer until only the center line (i.e., skeleton line) with a single pixel width that retains the original topology and connectivity is preserved.

[0067] Six-dimensional force / torque: refers to the force exerted on an object in space along the X, Y, and Z axes. ) and the torque about these three axes ( The general term for ).

[0068] Virtual-real benchmarking: This refers to establishing a mapping relationship between the physical real world and the computer simulation world. In this invention, it specifically refers to using physically measured parameters (such as stiffness and friction coefficient) to correct the simulation model, and comparing the differences in motion and mechanical response between the two under the same working conditions to verify the accuracy of the simulation.

[0069] Spline fusion algorithm based on minimizing reprojection error: A data fusion strategy that refers to a method in multi-view geometric reconstruction that uses spline interpolation algorithm to fit spatial points and aims to maximize the goodness of fit or smoothness, while combining epipolar constraints to select the optimal spatial curve.

[0070] Figure 1 This is a flowchart illustrating the method for determining the stiffness parameters of a flexible body in some embodiments of the present invention. For example... Figure 1 As shown, the method for determining the stiffness parameters of a flexible body includes the following steps:

[0071] S1: Reconstruct the three-dimensional centerline of the flexible body based on multi-view images of the flexible body in the gripping state of the robotic arm. The flexible body can be a cable, wire harness, conduit, hose, or other slender flexible body.

[0072] In one embodiment of the present invention, the multi-view images are acquired by a non-collinear multi-view visual acquisition device, which includes at least three cameras arranged non-collinearly in space. The optical axes of the at least three cameras form a non-collinear spatial distribution, and a unified visual coordinate system is established through joint extrinsic parameter calibration.

[0073] Figure 2 This is a schematic diagram of a hardware structure for determining the deformation stiffness of a flexible body in one example of the present invention. Figure 2 As shown, in some examples of this invention, the following hardware structure is employed:

[0074] 1. At least three high-resolution industrial cameras are used to construct an all-encompassing measurement field to capture the morphology of the flexible body from different perspectives.

[0075] 2. A six-axis robotic arm is used, with a six-dimensional force / torque sensor installed at its end.

[0076] 3. The end effector of the robotic arm is equipped with a quick-switchable gripper and a standard plug clamp.

[0077] In this embodiment of the invention, the calibration of multi-camera intrinsic and combined extrinsic parameters is performed as follows: A high-precision aerospace-grade aluminum checkerboard calibration plate is used, positioned at multiple angles within the common field of view of the cameras. Zhang's calibration method is used to calculate the intrinsic parameter matrix (focal length) of each camera. Main point ) and distortion coefficient (radial distortion) Tangential distortion Using stereo vision epipolar constraints, the rotation and translation matrix of each camera relative to the master calibration center is calculated. ), and construct a unified visual measurement coordinate system.

[0078] In some embodiments of the present invention, after acquiring multi-view images of the flexible body in the gripping state of a robotic arm using at least three high-resolution industrial cameras, the three-dimensional centerline of the flexible body is obtained in the following manner:

[0079] S1-1: Perform semantic segmentation on multi-view images and determine the skeleton information of the flexible body based on the semantic segmentation results.

[0080] Figure 3 This is a schematic diagram illustrating the extraction of a flexible skeleton from multi-view images, as an example of the present invention. Figure 3 As shown, semantic segmentation is performed on multi-view images. Based on the semantic segmentation results, the foreground region (i.e., flexible body) and background region (i.e., non-flexible body region) in the multi-view images can be identified. After semantic segmentation, a morphological erosion algorithm can be used to extract the flexible body skeleton with a single pixel width.

[0081] S1-2: Perform three-dimensional reconstruction based on the skeleton information of the flexible body to obtain the three-dimensional reconstruction model of the flexible body.

[0082] To address the directional ambiguity of three-dimensional curves, a global directional cost function is constructed. By enumerating the positive and negative combinations of multi-view curves, the combination with the smallest reprojection error is selected. A spatial rectangular coordinate system is established with the intersection of camera viewpoints as the spatial center to perform three-dimensional reconstruction of the flexible body.

[0083] In some embodiments of the present invention, for a single flexible body without a branching structure, S1-2 may include the following steps:

[0084] S1-2-A-1: Using a preset global cost function, generate a combination of positive and negative multi-view curves based on the skeleton information of the flexible body.

[0085] Multi-view curve resampling point coordinates .

[0086] Define direction indicator variable , where 1 represents keeping the original order and -1 represents reversing the order.

[0087] Constructing the global consistency cost function:

[0088]

[0089] Where N represents the total number of curve resampling points.

[0090] S1-2-A-2: Select the combination with the smallest reprojection error from the positive and negative combinations of multi-view curves, and use spline interpolation to perform three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the flexible body.

[0091] Index mapping function By enumerating 4 combinations, select the one that makes Minimum combination .

[0092] The formula for blending three-dimensional centerlines is:

[0093] .

[0094] Final three-dimensional centerline point .

[0095] In other embodiments of the present invention, for a bifurcated flexible body with a bifurcated structure, S1-2 may include the following steps:

[0096] S1-2-B-1: In response to the flexible body being a bifurcated flexible body with a bifurcated structure, the endpoints and bifurcation points of the bifurcated flexible body are determined based on the skeleton information.

[0097] Based on skeleton extraction, neighborhood analysis operators are used to locate bifurcation points and endpoints. Endpoint location: pixels with a degree of 1 within a 3x3 neighborhood are considered endpoints. Bifurcation point location: if multiple segments of white pixels separated by at least one black pixel appear within the same circle of a 5x5 neighborhood, the center point is considered a bifurcation point.

[0098] S1-2-B-2: Generate multi-segment curves of the bifurcation flexible body based on the endpoints and bifurcation points of the bifurcation flexible body.

[0099] Figure 4 This is a schematic diagram illustrating the decoupling of a bifurcated flexible body into multiple sub-curves in one example of the present invention. For example... Figure 4 As shown, the bifurcation flexible body is decoupled into multiple sub-curves consisting of "endpoint-bifurcation point" or "bifurcation point-bifurcation point" segments. Specifically, the steps for generating multiple sub-curves include:

[0100] The extension direction of each sub-curve at the bifurcation point is determined using the 5x5 neighborhood operator. After determining the starting point of the skeleton, the numbering of each bifurcation point (connecting bifurcation points to each other) and endpoint (connecting bifurcation points to each endpoint) is confirmed counterclockwise.

[0101] The bifurcation points and endpoints in each viewpoint are matched according to the corresponding numbers, and then fitted in three-dimensional space.

[0102] The curves between the bifurcation points and endpoints are split according to the bifurcation point and endpoint numbers, and the sub-curves of each viewpoint are matched accordingly and fitted in three-dimensional space.

[0103] S1-2-B-3: Perform three-dimensional reconstruction and stitching processing on multiple sub-curves between multiple perspectives corresponding to multi-view images to obtain a three-dimensional reconstruction model of the bifurcated flexible body.

[0104] S1-3: Based on the three-dimensional reconstruction model of the flexible body, determine the three-dimensional centerline of the flexible body.

[0105] Figure 5 This is a schematic diagram illustrating the reconstruction of the three-dimensional centerline of a single flexible body without a branching structure, as per the present invention. Figure 5 As shown, for a three-dimensional reconstruction model of a single flexible body without a bifurcation structure, the high-precision three-dimensional centerline of the flexible body (the line connecting the center points of each section of the flexible body in three-dimensional space) can be generated by combining the maximum spline interpolation fusion model.

[0106] Figure 6 This is a schematic diagram of the three-dimensional centerline of a bifurcated flexible body in one example of the present invention. Figure 6 As shown, by using the bifurcation point as a strong feature constraint, sub-curve matching is performed across multiple perspectives, and the sub-curves are stitched together after 3D reconstruction to finally restore the complex topology.

[0107] S2: Obtain deformation rate information distributed along the arc length of the three-dimensional centerline.

[0108] Based on the three-dimensional centerline of the flexible body, calculate the geometric curvature of each point along the arc length of the 3D centerline. ) and torque ( ).

[0109] S3: Based on the sensors installed on the robotic arm, collect the force information of the flexible body when it is held by the robotic arm.

[0110] Using a six-dimensional force / torque sensor on a robotic arm, force / torque information of the flexible body under the gripping state of the robotic arm is collected, namely the force along the X, Y, and Z axes. ) and the torque about these three axes ( ).

[0111] S4: Based on the deformation rate information and force information, determine the deformation stiffness information of the flexible body. The deformation stiffness information of the flexible body includes its bending stiffness and torsional stiffness.

[0112] In some embodiments of the present invention, step S4 includes the following steps:

[0113] S4-1: Construct the mechanical equilibrium relationship of the flexible body based on the preset model. The preset model is the Cosserat rod model or an equivalent flexible body mechanical model.

[0114] S4-2: Based on the mechanical equilibrium relationship, deformation rate information and force information, inversion is performed to obtain the deformation stiffness information of the flexible body.

[0115] Considering distributed gravity load and end-point six-dimensional load (force) torque ),in The linear weight density (gravity per unit length) of a flexible material.

[0116] Geometric kinematic parameters (tangent vector, curvature, torsion) , , ,in Arc length parameter along the 3D centerline (from the root) To the end ).

[0117] Internal force Inverse integration: using the equilibrium equations of forces From the end Integrating towards the root: .

[0118] internal moment Inverse integration: using the torque balance equation Combining discrete difference: Boundary conditions ,in The integral dummy variable representing the arc length, This represents the external torque scalar (torque along the tangential direction).

[0119] Decomposition of bending torque: Torque scalar: Bending moment vector: .

[0120] Bending stiffness Inversion: based on constitutive relations The solution is obtained through statistical methods (using the median for robust noise resistance): Thus, Young's modulus is obtained. ,in , Indicates the radius of the flexible body's cross-section. This represents the curvature threshold.

[0121] Internal torsion calculate: ,in This represents the total physical torsion angle of the flexible body's end relative to its root.

[0122] Torsional stiffness Inversion: based on constitutive relations Inverse torsional stiffness : .

[0123] In some embodiments of the present invention, the method for determining the deformation stiffness of the flexible body further includes the following steps: during the movement of the flexible body held by the robotic arm, the maximum deflection point of the flexible body is located in real time, and a curve showing the change of the coordinates of the maximum deflection point over time is generated.

[0124] During the movement of the flexible body, the point of maximum deflection is located in real time, and a coordinate-time variation curve is generated. Based on the three-dimensional centerline of the flexible body fitted in space, with the end point as the point of maximum deflection, a coordinate-time variation curve is plotted according to the coordinates of this point in space.

[0125] In some embodiments of the present invention, a timestamp-based B-spline interpolation algorithm is used to downsample (downsample) the high-frequency six-dimensional force / torque data to the corresponding time of the visual frame.

[0126] Figure 7 This is a flowchart illustrating the correction method for the flexible body simulation model in some embodiments of the present invention. For example... Figure 7 As shown, the method for correcting the simulation model of a flexible body includes the following steps:

[0127] A: Determine the deformation stiffness information (i.e., bending stiffness and / or torsional stiffness) of the flexible body according to the above method for determining the deformation stiffness of the flexible body.

[0128] B: Provide the deformation stiffness information of the flexible body to the flexible body simulation model, and obtain the actual measurement results and simulation results of the flexible body under preset motion conditions.

[0129] A simulation scene corresponding to the physical test bench at a 1:1 scale is constructed in the simulation environment. Deformation stiffness information is input into the flexible body simulation model, and a robotic arm is used to control the flexible body to move according to preset motion conditions in both the real scene and the simulation environment. The actual measurement results and simulation results of the flexible body under the preset motion conditions are collected. The simulation environment is a physical simulation platform that supports flexible body modeling and multibody dynamics calculation.

[0130] In some embodiments of the present invention, step B, obtaining the actual measurement results of the flexible body under preset motion conditions, includes the following steps:

[0131] B-1: Control the robotic arm to interact with the interactive mechanism under test along a preset trajectory.

[0132] Control a robotic arm with at least six degrees of freedom to make contact with the interactive mechanism under test along a preset or adaptive trajectory and perform switching, insertion or removal actions.

[0133] In some embodiments of the present invention, the robotic arm executes an interactive trajectory along a force-controlled or position-force hybrid control method to simulate the flexible contact and tolerance behavior of a human hand during operation.

[0134] B-2: During the interactive action performed by the robotic arm, the force information during the interaction is collected by the sensors installed on the robotic arm, and the posture information of the robotic arm at the preset position is obtained.

[0135] During the interactive actions performed by the robotic arm, a six-dimensional force / torque sensor installed at the end of the robotic arm synchronously collects force and torque data during the interaction process, and simultaneously records the pose or displacement information of the end of the robotic arm.

[0136] B-3: Based on force and attitude information, generate mechanical curves characterizing the mechanical properties of the interaction process (used to quantify the operational feel of the interaction mechanism). These mechanical curves are parameterized and used to correct the friction parameters, damping parameters, or contact model parameters of the interaction mechanism in the simulation environment.

[0137] In some embodiments of the present invention, the mechanical curves include force-time curves and / or force-displacement hysteresis curves, which are used to characterize the damping characteristics, frictional characteristics, or rebound characteristics of the interactive mechanism during operation. The frictional force is the frictional force between the hinge and the plug / socket. The damping force is the damping force of the latch or magnetic attraction.

[0138] C: Based on the difference between the simulation results and the actual measurement results, adjust the parameters of the flexible body simulation model (so that the simulation results are consistent with the actual scene).

[0139] In some embodiments of the present invention, the trajectory deviations of real and simulated motions under the same motion are compared to verify the credibility of the simulation.

[0140] Static state: By visually observing the static deformation of the flexible body under gravity in the static state, relevant stiffness and other parameters in the real and simulated states can be calculated for comparison (stiffness in the simulation can be directly specified from the simulation software).

[0141] Dynamic state: Design a fixed trajectory to clamp and swing a flexible body, with the following requirements: the same clamping point; the same swing trajectory and speed; the same flexible body size; and the same key observation points.

[0142] By recording the positional changes of key observation points (such as the maximum deflection point) under the same swing trajectory, as well as the oscillation performance when the robot arm is stationary after swinging (the positional changes of key observation points when the cable oscillates after the robot arm moves quickly and stops rapidly), the dynamic characteristics of flexible bodies are compared (the elastic performance of real and simulated flexible bodies under motion and inertia is consistent).

[0143] By employing the aforementioned technical solutions, this invention combines visual and mechanical analysis to easily measure the stiffness parameters of flexible bodies using a non-contact method, overcoming the limitation of existing technologies that cannot invert the physical properties of flexible bodies through visual data under non-contact conditions. Furthermore, for bifurcate flexible bodies, this invention uses a bifurcation point topology matching algorithm to effectively solve the problems of self-occlusion and 3D reconstruction fracture in complex wire harnesses. Moreover, this invention, through the reverse injection of real data, endows the simulation environment with realistic physical properties, improving the authenticity and reliability of the simulation data.

[0144] In addition, this disclosure also provides an electronic device, including:

[0145] Memory, used to store computer programs;

[0146] A processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, it implements the method for determining the stiffness parameters of a flexible body and / or the method for correcting a flexible body simulation model as described in any of the above embodiments of this disclosure.

[0147] Below, for reference Figure 8 To describe an electronic device according to embodiments of this disclosure. For example... Figure 8 As shown, the electronic device includes one or more processors and memory.

[0148] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0149] The memory can store one or more computer program products. The memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor can run the computer program products to implement the methods for determining the stiffness parameters of flexible bodies and / or the methods for correcting flexible body simulation models, as described in the various embodiments of this disclosure, and / or other desired functions.

[0150] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0151] In addition, the input device may also include, for example, a keyboard, a mouse, etc.

[0152] This output device can output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0153] Of course, for the sake of simplicity, Figure 8 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0154] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods for determining the stiffness parameters of a flexible body and / or the methods for modifying a flexible body simulation model according to various embodiments of this disclosure as described in the foregoing portions of this specification.

[0155] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0156] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform steps in the methods for determining the stiffness parameters of a flexible body and / or modifying a flexible body simulation model according to various embodiments of this disclosure as described in the foregoing portion of this specification.

[0157] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0158] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0160] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0161] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0162] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0163] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0164] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0165] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for determining the deformation stiffness of a flexible body, characterized in that, include: Based on non-collinear multi-view images of the flexible body in the gripping state of the robotic arm, the three-dimensional centerline of the flexible body is reconstructed. Obtain the geometric curvature and torsion of each point along the arc length of the three-dimensional centerline; Based on the sensors installed on the robotic arm, the force and torque information of the flexible body under the clamping state of the robotic arm is collected; Based on the preset model, the mechanical equilibrium relationship of the flexible body, the geometric curvature and torsion of each point, and the force and torque information are inverted to obtain the deformation stiffness information of the flexible body; The method of reconstructing the three-dimensional centerline of the flexible body based on multi-view images of the flexible body in the gripping state of the robotic arm includes: Semantic segmentation is performed on the multi-view images, and the skeleton information of the flexible body is determined based on the semantic segmentation results; Based on the skeleton information of the flexible body, a three-dimensional reconstruction is performed to obtain a three-dimensional reconstruction model of the flexible body. Based on the three-dimensional reconstruction model of the flexible body, the three-dimensional centerline of the flexible body is determined. The step of performing three-dimensional reconstruction based on the skeleton information of the flexible body to obtain a three-dimensional reconstruction model of the flexible body includes: Using a preset global cost function, a combination of positive and negative multi-view curves is generated based on the skeleton information of the flexible body; The combination with the smallest reprojection error is selected from the positive and negative combinations of the multi-view curves, and three-dimensional reconstruction is performed using spline interpolation to obtain the three-dimensional reconstruction model of the flexible body. Alternatively, the step of performing three-dimensional reconstruction based on the skeleton information of the flexible body to obtain a three-dimensional reconstruction model of the flexible body includes: In response to the fact that the flexible body is a bifurcated flexible body with a bifurcation structure, the endpoints and bifurcation points of the bifurcated flexible body are determined based on the skeleton information; Based on the endpoints and bifurcation points of the bifurcation flexible body, a multi-segment sub-curve of the bifurcation flexible body is generated; The multiple sub-curves are reconstructed and stitched together in three dimensions between the multiple perspectives corresponding to the multi-view images to obtain the three-dimensional reconstruction model of the bifurcated flexible body.

2. The method according to claim 1, characterized in that, The preset model is the Cosserat rod model.

3. The method according to claim 1, characterized in that, Also includes: During the movement of the flexible body held by the robotic arm, the maximum deflection point of the flexible body is located in real time, and a curve of the coordinates of the maximum deflection point changing over time is generated.

4. The method according to any one of claims 1-3, characterized in that, The multi-view images are acquired by a non-collinear multi-view visual acquisition device, which includes at least three cameras arranged non-collinearly in space. The optical axes of the at least three cameras form a non-collinear spatial distribution, and a unified visual coordinate system is established through joint extrinsic parameter calibration.

5. A method for correcting a flexible body simulation model, characterized in that, include: The method according to any one of claims 1-4 determines the deformation stiffness information of the flexible body; The deformation stiffness information of the flexible body is provided to the flexible body simulation model, and the actual measurement results and simulation results of the flexible body under preset motion conditions are obtained. Based on the difference between the simulation results and the actual measurement results, the parameters of the flexible body simulation model are adjusted.

6. The method according to claim 5, characterized in that, The process of obtaining the actual measurement results of the flexible body under preset motion conditions includes: Control the robotic arm to interact with the mechanism under test along a preset trajectory; During the interactive action performed by the robotic arm, force information during the interaction is collected by sensors installed on the robotic arm, and the posture information of the robotic arm at a preset position is obtained. Based on the force information and the attitude information, a mechanical curve characterizing the mechanical properties of the interaction process is generated.

7. The method according to claim 6, characterized in that, The mechanical curves include force-time curves and / or force-displacement hysteresis curves.

8. An electronic device, characterized in that, include: Memory, used to store computer program products; A processor for executing a computer program product stored in the memory, wherein when the computer program product is executed, it implements the method described in any one of claims 1-7.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-7.

10. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1-7.

Images