A monocular variable range multi-axis force sensor

Abstract

The application discloses a monocular variable-range multi-axis force sensor, and relates to the technical fields of force sensing and machine vision, comprising a force bearing system, a deformation measuring system and a force solving system, wherein the force bearing system comprises an upper mounting plate, an elastic body and a lower mounting plate, and is used for generating elastic deformation under the action of external force or torque; the deformation measuring system comprises a feature pattern layer arranged on one side of the elastic body and a monocular camera arranged opposite to the feature pattern layer, and is used for collecting images of the feature pattern before and after loading and obtaining two-dimensional displacement information; and the force solving system is used for directly solving multi-axis force and torque components from the two-dimensional image displacement field based on a pre-calibrated displacement-force mapping model. The application realizes multi-axis force measurement without three-dimensional reconstruction by acquiring two-dimensional displacement through monocular vision and establishing a mapping relationship between the two-dimensional displacement and six-dimensional load, and solves the problems of high cost, fixed range and difficult maintenance of traditional sensors.

Description

Technical Field

[0001] This invention relates to the fields of force sensing and machine vision technology, and in particular to a monocular variable range multi-axis force sensor. Background Technology

[0002] Multi-axis force sensors are core components for robots to achieve force perception and intelligent interaction with the environment, and are widely used in humanoid robots, industrial robotic arms, collaborative robots and precision assembly equipment.

[0003] However, their high manufacturing cost leads to a sharp increase in the total system cost in humanoid robots or complex automated production lines that require multi-joint deployment. Secondly, sensors are mostly integrated packages, making on-site repair difficult after damage, requiring complete replacement, resulting in long maintenance cycles and high costs. Most importantly, their measurement range is fixed, making it impossible to flexibly switch between different tasks requiring delicate operations (such as precision assembly) and heavy-load handling (such as handling large masses). Therefore, in fields such as service robots, smart industry, medical rehabilitation, and aerospace, where the requirements for cost, reliability, and adaptability are increasingly stringent, there is an urgent need for a new type of multi-axis force sensor solution that combines low cost, easy maintenance, and dynamic variable range capability. Summary of the Invention

[0004] The main objective of this invention is to provide a monocular variable range multi-axis force sensor.

[0005] Another objective of this invention is to propose a multi-axis force measurement method based on monocular vision.

[0006] The third objective of this invention is to provide an electronic device.

[0007] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0008] To achieve the above objectives, a first aspect of the present invention provides a monocular variable range multi-axis force sensor, comprising:

[0009] A force-bearing system is used to withstand externally applied forces or moments and generate elastic deformation. The force-bearing system includes an upper mounting plate, an upper transparent rigid support layer, a transparent elastomer, a lower transparent rigid support layer, and a lower mounting plate. The elastomer deforms under external load to reflect the force or moment information. A deformation measurement system is used to acquire image information corresponding to the deformation of the transparent elastomer. The deformation measurement system includes a feature pattern layer disposed on one side of the transparent elastomer and a monocular camera disposed opposite to the feature pattern layer. The monocular camera is used to acquire images of the feature pattern layer in an unloaded state and a loaded state to obtain two-dimensional displacement information of the feature pattern in the image plane. The force calculation system is communicatively connected to the monocular camera and is used to directly calculate the multi-axis force / torque components based on the two-dimensional image displacement information of the feature pattern through a pre-established displacement-force mapping model.

[0010] Optionally, the multi-axis force sensor has various structural forms, including: single-axis distribution form, optical path reflection form, dual-axis distribution form, and external measurement form.

[0011] Optionally, when the multi-axis force sensor is in a uniaxial distribution form, the force bearing system and the deformation measurement system are spatially aligned in a straight line, comprising the following layered structure from top to bottom: The upper mounting plate is a top-supporting component and has mounting holes for applying or transferring the load to be measured. A transparent rigid support layer is used to restrict non-target deformation, allowing elastic deformation to occur in the elastic body; The feature pattern layer has a pseudo-random speckle pattern, a regular matrix pattern, a QR code pattern, or a combination thereof. A reflective layer is also provided on the feature pattern layer to enhance the contrast of the feature pattern. A transparent elastomer, wherein the transparent elastomer material includes polymer or metal material, and the geometric parameters of the elastomer include thickness, area, and structural morphology, is used to adjust the overall stiffness of the sensor by changing the geometric parameters to achieve range switching; A transparent rigid support layer is used to restrict non-target deformation, allowing elastic deformation to occur in the elastic body; A monocular camera, facing the feature pattern layer, is used to continuously acquire feature pattern images; The lower mounting plate serves as the bottom support component.

[0012] Optionally, when the multi-axis force sensor is of the optical path reflection type, the rest of the structure remains the same as the single-axis distribution type, with only a reflector added to change the optical path direction and shorten the axial length of the sensor. When the multi-axis force sensor is in a biaxial distribution form, the force bearing system and the deformation measurement system are respectively located on two axes in space, and the rest of the structure is consistent with the single-axis distribution form; When the multi-axis force sensor is in the form of an external measurement, the monocular camera is located externally. The sensor is not spatially packaged as a whole, but the monocular camera is fixed to the lower transparent rigid support layer or its relative displacement is known. The rest of the structure is consistent with the single-axis distribution form.

[0013] To achieve the above objectives, a second aspect of the present invention provides a multi-axis force measurement method based on monocular vision, applied to the multi-axis force sensor described in any one of the first aspects, comprising: The multi-axis force sensor is fixed to the measurement system via upper and lower mounting plates. The image of the feature point under unloaded state is acquired as the reference image, and the initial image coordinates of each feature point are recorded. The force / torque to be measured is applied to a rigid mounting plate to cause deformation of the elastomer and displacement of feature points. The image of the feature points after deformation is acquired in real time by a monocular camera. The two-dimensional image plane displacement field of the deformed feature point image relative to the reference image is calculated using an image correlation algorithm. The two-dimensional image planar displacement field is input into a pre-calibrated displacement-force mapping model, and a real-time multidimensional force / torque vector is output. The mapping model does not perform three-dimensional spatial displacement reconstruction, but directly establishes a mapping relationship between the two-dimensional image displacement data and the multidimensional force / torque.

[0014] Optionally, the displacement-force mapping model is obtained in the following way: A series of known uniaxial forces / torques are applied to the sensor, and the corresponding displacement fields of feature points are recorded. A linear transformation matrix is ​​formed by fitting the data using the least squares method, and then a displacement-force mapping model is constructed.

[0015] Optionally, the displacement-force mapping model is obtained in the following way: A neural network model is trained based on calibration data, and a nonlinear mapping from displacement field to force / torque is established to obtain a displacement-force mapping model.

[0016] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory to implement the method described in the second aspect.

[0017] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the second aspect.

[0018] The embodiments of the present invention have the following beneficial effects: First, it exhibits excellent dynamic response performance. The embodiments of this invention acquire two-dimensional image displacement information based on monocular vision, eliminating the need for complex three-dimensional reconstruction calculations. This results in low data processing dimensionality, a simple computational path, and high algorithm implementation efficiency, enabling rapid extraction of the displacement field and real-time force / torque calculation, thus meeting the real-time requirements of dynamic load measurement and high-frequency response scenarios.

[0019] Secondly, it is low-cost and easy to maintain. The core sensing unit of this invention mainly consists of a common industrial camera and a transparent elastomer, eliminating the need for strain gauges, bridge circuits, and complex signal conditioning units. The overall manufacturing cost is significantly lower than that of traditional strain gauge-type multi-axis force sensors. Furthermore, the transparent elastomer, as a replaceable module, can be directly replaced modularly when damaged by extreme loads or long-term use, eliminating the need for complete factory repairs. This facilitates rapid on-site maintenance and low-cost operation and maintenance.

[0020] Third, it offers strong range adjustability and a wide range of applications. By replacing transparent elastomer modules with different materials or geometric parameters, the overall system stiffness can be flexibly adjusted, thereby changing the sensor's range and sensitivity, and enabling rapid switching between multiple range modes. Users can select a suitable elastomer structure according to specific application requirements, allowing a single system to adapt to various testing conditions, improving equipment utilization and application adaptability.

[0021] Fourth, the structure exhibits high robustness and strong overload resistance. The embodiments of this invention employ a purely optical measurement principle, eliminating the need to attach strain gauges to the surface of the elastomer or lay electrical leads. There are no electrical connection nodes prone to detachment or aging, resulting in high overall structural reliability. Furthermore, the transparent elastomer can withstand significant elastic deformation within a reasonable design range, is not prone to permanent damage under overload conditions, and automatically returns to its original state and continues to function once the load returns to normal, demonstrating excellent impact and overload resistance.

[0022] Fifth, it offers significant potential for accuracy improvement and strong technological scalability. The measurement accuracy of this invention primarily depends on image resolution, feature pattern design, and the accuracy of the solution algorithm. By employing higher-resolution imaging equipment, optimizing the feature point distribution, and improving image correlation algorithms and displacement-force mapping models, the system's measurement accuracy and stability can be continuously improved. This technical approach is not limited by the physical constraints of traditional strain gauge sensitivity coefficients and bridge structures, possessing substantial performance improvement potential and promising technological development prospects. Attached Figure Description

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram of a monocular variable range multi-axis force sensor provided in an embodiment of the present invention; Figure 2 Schematic diagrams of various structural forms of a monocular variable range multi-axis force sensor provided for embodiments of the present invention; Figure 3 A flowchart of a multi-axis force measurement method based on monocular vision provided in an embodiment of the present invention; Figure 4This is a schematic diagram of speckle patterns provided in a certain embodiment of the present invention; Figure 5 for Figure 4 A schematic diagram of the calibration results provided in the Chinese embodiment. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] The existing technology has the following main limitations: 1. High cost: Traditional sensors based on precision strain gauges have complex elastomer processing and patching processes, as well as sophisticated signal conditioning circuits, resulting in high manufacturing costs and making them difficult to deploy on a large scale in multi-joint robots.

[0027] 2. Fixed measuring range and poor adaptability: The measuring range of traditional sensors is determined by the stiffness of the elastic body structure, which cannot be changed once manufactured. It is impossible to flexibly switch between different tasks that require high-precision small force measurement (such as precision assembly) and high-range large force measurement (such as heavy object handling).

[0028] 3. Difficult to maintain and insufficient robustness: Strain gauge sensors are highly integrated, and damage usually requires factory repair or complete replacement. Furthermore, strain gauges and their leads are easily damaged under overload, impact, or harsh environments.

[0029] 4. Complex vision solutions: Binocular or structured light-based 3D vision solutions require complex system calibration, extensive image data processing, and 3D reconstruction calculations, resulting in high system costs, slow dynamic response, and difficulty in achieving real-time high-speed force measurement. This invention aims to overcome the aforementioned shortcomings and solve the problem of how to achieve multi-axis force / torque measurement with dynamic range adjustment and easy maintenance in a low-cost and highly robust manner. Specifically, the technical problem to be solved is: how to stably and accurately calculate the multi-dimensional spatial forces and torques applied to an elastic body using a simplified monocular vision system, relying solely on two-dimensional image displacement information.

[0030] The following description, with reference to the accompanying drawings, describes a monocular variable-range multi-axis force sensor and a multi-axis force measurement method based on monocular vision, according to embodiments of the present invention.

[0031] Example 1 This embodiment provides a variable-range multi-axis force sensor based on monocular vision. Its core idea is: when a force is applied to an elastic body, a monocular camera captures the displacement or deformation of a feature pattern image, which implicitly contains information about the elastic body's deformation. Then, a pre-established neural network or analytical solution algorithm, along with a force calculation and compensation algorithm, is used to achieve real-time calculation of three-dimensional force and torque.

[0032] Reference Figure 1 The variable range multi-axis force sensor comprises the following three systems: a force bearing system, a deformation measurement system, and a force calculation system.

[0033] The following is a detailed description of each system and extended examples.

[0034] (1) A force-bearing system for bearing externally applied forces or moments and generating elastic deformation, the force-bearing system comprising an upper mounting plate, an upper transparent rigid support layer, a transparent elastomer, a lower transparent rigid support layer, and a lower mounting plate. Each part plays a different but synergistic role in the overall force-bearing system to ensure that the force or moment can be accurately transmitted and generate the required elastic deformation.

[0035] Specifically, the upper mounting plate is the top load-bearing component of the force-bearing system. Its main function is to receive externally applied loads and transfer them to the transparent elastomer below. The upper mounting plate typically has multiple mounting holes for applying or transferring the load to be measured, allowing the force sensor to accept forces or torques of different directions or magnitudes.

[0036] The upper transparent rigid support layer, located between the upper mounting plate and the transparent elastomer, primarily functions to restrict non-target deformation. It ensures that the elastomer deforms only in a predetermined direction and within a specific region, accurately reflecting the applied external force or torque. By rationally designing the thickness and stiffness of this layer, it is possible to ensure that deformation occurs only within the elastomer, thereby improving measurement accuracy.

[0037] The transparent elastomer is the core component of this system. It deforms under external force, reflecting the force or torque through this deformation. Suitable materials for the transparent elastomer include high-molecular polymers such as PDMS (polydimethylsiloxane), TPU (thermoplastic polyurethane), and polyethylene, or even harder materials like metals. The elastomer can be blocky, cylindrical, or mesh-like, with the specific structure designed according to actual needs. The geometric parameters of the transparent elastomer, such as thickness, area, and structural morphology, can be adjusted to change the overall stiffness of the sensor, thereby enabling switching of force or torque measurement ranges. By adjusting these parameters, the sensor's sensitivity and measurement range can be effectively altered to adapt to different measurement requirements.

[0038] Similar to the upper transparent rigid support layer, the lower transparent rigid support layer also serves to restrict non-target deformation, ensuring that the deformation of the elastic body only occurs within the predetermined area. By rationally designing this support layer, the mechanical properties of the force-bearing system can be further optimized.

[0039] The lower mounting plate is the bottom load-bearing component of the force-bearing system, primarily responsible for bearing the deformation reaction force of the elastic body and providing it with support. The design of the lower mounting plate needs to ensure good cooperation with other components, enabling the entire force-bearing system to achieve a stable and precise working state.

[0040] Through the above structural design, the force-bearing system can effectively withstand externally applied forces or moments and generate controllable elastic deformation, ultimately achieving accurate measurement of external forces or moments. The advantages of this system lie in its clear structure and distinct functions, enabling range switching by adjusting geometric parameters and selecting different materials, thus providing a flexible mechanical response.

[0041] (2) Deformation measurement system, used to acquire deformation information of transparent elastomer under external force, indirectly reflecting the deformation of elastomer by capturing pixel displacement or deformation of feature pattern. The system includes feature pattern layer and monocular camera set opposite to it. They work together to acquire accurate image information and extract two-dimensional displacement information from it in order to calculate the deformation of transparent elastomer.

[0042] In this embodiment of the invention, a feature pattern layer is disposed on one side of the transparent elastomer. The selection and design of the feature pattern are crucial; it can be a regular speckle pattern, a speckle matrix, a QR code, or any other easily identifiable pattern, or even a combination of multiple patterns. The function of the feature pattern is to correlate with the deformation of the transparent elastomer; therefore, the deformation of the pattern directly reflects the deformation of the transparent elastomer. To ensure the stability and accuracy of the pattern, the feature pattern layer is typically fixedly connected to a rigid support, the position of which relative to the transparent elastomer is known, ensuring that the real-time positional changes of the pattern under load can be accurately tracked.

[0043] A monocular camera is fixed to another rigid support, and its position relative to the feature pattern layer is known in real time. The camera's main task is to image the feature pattern layer, recording images of the feature patterns under both unloaded and loaded conditions, and capturing pixel displacement information. This displacement information can be converted into deformation data of the elastic body. To ensure the system's accuracy, the camera's mounting position must ensure that the acquired images can clearly distinguish the deformation or displacement of the feature patterns under six different deformation modes of the elastic body. In this way, the deformation measurement system can provide accurate two-dimensional displacement data, thereby revealing the deformation characteristics of the transparent elastomer under different load conditions.

[0044] To improve the accuracy of image acquisition, a reflective layer is also provided on the feature pattern layer proposed in this embodiment of the invention. The function of this reflective layer is to enhance the contrast of the feature pattern, making the distinction between the pattern and the background more obvious during camera shooting, thereby improving image quality and ensuring accurate capture of displacement or deformation.

[0045] This deformation measurement system can accurately acquire deformation information of a transparent elastomer and convert it into two-dimensional displacement data that can be used for further calculations. This data provides the necessary input for the force calculation system, enabling the entire sensor to accurately reflect externally applied forces or torques with high measurement accuracy and sensitivity.

[0046] (3) Force calculation system, which is connected to the monocular camera, is used to directly calculate the multi-axis force / torque components based on the two-dimensional image displacement information of the feature pattern through a pre-established displacement-force mapping model.

[0047] In this embodiment of the invention, the communication connection between the force calculation system and the monocular camera enables it to directly calculate multi-axis force and torque components based on the two-dimensional image displacement information of the feature pattern using a pre-established displacement-force mapping model. The core task of this system is to deduce the externally applied force and torque from the pixel displacement or deformation of the feature pattern, thereby providing accurate measurement of multi-axis forces. In practical applications, the force calculation system first performs preliminary calculations on the displacement data based on the known positional relationship between the feature pattern, the camera, and the rigid support, using a predetermined displacement-force mapping model. This process typically employs a neural network or analytical solution algorithm based on calibration data. After the preliminary calculation, the system further corrects system errors using a force calculation compensation algorithm, thereby improving the accuracy and stability of the measurement.

[0048] To obtain high-precision force / torque calculation results, the force calculation system relies on a displacement-force mapping model. This model can convert displacement field data obtained from the deformation measurement system into specific force or torque components. This mapping model can be obtained in various ways, including but not limited to the following methods: Experimental Calibration and Linear Mapping: In this method, the sensor is calibrated by applying a series of known uniaxial forces or torques. Displacement field data corresponding to the characteristic patterns under these known forces / torques are recorded, and a least-squares method is used for fitting to obtain a linear transformation matrix. This matrix directly converts the displacement field data into force and torque components, suitable for preliminary system calculations.

[0049] Data-driven model: This method trains a neural network model based on a large amount of calibration data (whether obtained through experiments or simulations), thereby establishing a nonlinear mapping relationship from displacement field to force or torque. Unlike traditional linear mappings, data-driven models can handle more complex nonlinear mechanical behaviors, and are therefore suitable for more complex and sophisticated sensor systems. In this way, the neural network model can accurately deduce the corresponding force or torque components based on the input displacement information.

[0050] Hybrid Models (Combining Linear and Nonlinear Characteristics): In some cases, the deformation behavior of sensors may simultaneously exhibit linear and nonlinear features. In such situations, using a hybrid model to handle displacement-force mapping is an effective approach. This method combines the advantages of linear mapping and neural network models, handling simple deformation patterns through certain linear relationships while using neural networks to process complex deformation patterns. This approach can address more diverse deformation characteristics while maintaining a certain level of computational efficiency.

[0051] The displacement-force mapping model obtained through these methods, combined with the force calculation compensation algorithm, can effectively and accurately calculate the multiaxial force / torque components from the displacement information of the feature pattern, ensuring that the force calculation system can provide high-precision mechanical measurement results in various complex environments.

[0052] The multi-axis force sensor is characterized by having various structural forms, including: single-axis distribution, optical path reflection, dual-axis distribution, and external measurement.

[0053] Furthermore, in the embodiments of the present invention, the multi-axis force sensor has various structural forms depending on the scenario. Each structure is designed according to different application requirements and measurement requirements to ensure efficient mechanical performance and accurate measurement results.

[0054] Reference Figure 2 Specifically, the following four structural forms were considered and applied to this sensor system: (1) Uniaxial distribution form: In the uniaxial distribution form, the force bearing system and the deformation measurement system are arranged in a straight line in space. This type of structure includes the following layers from top to bottom: Upper mounting plate 1: As the top component of the force-bearing system, it is mainly used to bear externally applied loads and transfer them to the various components below.

[0055] Upper transparent rigid support layer 2: Used to limit non-target deformation and ensure that the elastomer undergoes mainly elastic deformation.

[0056] Feature pattern layer 3: Features feature patterns used to capture displacement information through a deformation measurement system.

[0057] Transparent elastomer 4: The core component, which deforms under external force and reflects the force or torque applied from the outside.

[0058] Lower transparent rigid support layer 5: Similar to the upper transparent rigid support layer, its main function is to limit non-target deformation and ensure that elastic deformation mainly occurs on the elastic body.

[0059] Monocular camera 6: Used to capture image information of feature patterns and reflect the deformation of the elastic body through image displacement data.

[0060] Lower mounting plate 7: Bottom load-bearing component, responsible for receiving the force transmitted from below and providing support.

[0061] The advantage of this structural form is its simple design, with the force-bearing system and deformation measurement system located on the same axis, which facilitates high-precision mechanical measurements.

[0062] (2) Optical path reflection: In the optical path reflection type, a reflector 8 is introduced to reduce the axial length of the compressive force bearing system. This design enables the camera to acquire the image of the feature pattern layer through the reflection path by refracting the light path. By introducing the reflector, the overall structure of the sensor can be made more compact and the space occupied can be effectively reduced. This design is particularly suitable for applications that require space saving.

[0063] (3) Dual-axis distribution: In a dual-axis distribution, the force sensor places the force-bearing system and the deformation measurement system on two separate axes in space. The advantage of this design is that it can simultaneously measure force and torque in two different directions, thereby improving the sensor's multi-axis measurement capability. With dual-axis distribution, the sensor can simultaneously process multiple forces or torques applied in different directions, which is very advantageous for some complex application scenarios, such as multi-directional load measurement.

[0064] (4) External Measurement Form: In the external measurement form, the external monocular camera 6 of the sensor is located externally, and the sensor structure is no longer a completely enclosed whole, but is partially exposed to the external environment. Although the sensor structure is not completely encapsulated as a whole in space, the relative position between the external monocular camera 6 and the lower transparent rigid support layer is known, and displacement information of the feature pattern is captured through this camera. This structural form is suitable for some specific application scenarios, especially when it is necessary to expose the sensor or integrate it into complex devices.

[0065] Through the different designs of these four structural forms, the sensor can provide flexible and accurate mechanical measurement solutions according to different needs and application environments. Each form has unique advantages and can perform optimally in different mechanical measurement tasks.

[0066] Example 2 This invention also relates to a multi-axis force measurement method based on monocular vision, applied to the multi-axis force sensor shown in Example 1, such as... Figure 3 As shown, the method includes the following steps: S1, Fix the multi-axis force sensor to the measurement system through the upper and lower mounting plates, collect the image of the feature point in the unloaded state as the reference image, and record the initial image coordinates of each feature point; S2, the force / torque to be measured is applied to the rigid mounting plate to cause deformation of the elastic body and displacement of the feature points, and the feature point images after deformation are acquired in real time by a monocular camera; S3, calculates the two-dimensional image plane displacement field of the deformed feature point image relative to the reference image using an image correlation algorithm; S4, input the two-dimensional image planar displacement field into the pre-calibrated displacement-force mapping model, and output a real-time multidimensional force / torque vector. The mapping model does not perform three-dimensional spatial displacement reconstruction, but directly establishes the mapping relationship between the two-dimensional image displacement data and the multidimensional force / torque.

[0067] In this embodiment of the invention, image correlation algorithms are used to calculate the displacement of objects or feature points in an image. In this application embodiment, the following image correlation algorithms may be used: cross-correlation algorithm, block matching-based algorithm, optical flow method, feature matching algorithm, and phase shift method.

[0068] Furthermore, the specific details of the displacement-force mapping model have been explained in Example 1, and will not be repeated here.

[0069] Furthermore, this embodiment of the invention provides a specific example in which the speckle layer of the monocular multi-axis force sensor used is as follows: Figure 4 As shown, the speckle patterns are arranged in an array matrix. The camera axis forms an angle of approximately 20 degrees with the normal to the speckle plane. The relationship between speckle displacement and force or bending moment is calculated through single-axis calibration.

[0070] During the calibration process, the following formulas are used to establish the relationship between force and bending moment and speckle displacement:

[0071]

[0072] in, For pixel displacement, For calibration force, To calibrate the bending moment, and These are the compliance coefficients for force and bending moment, respectively, expressed in pixels per Newton or pixels per Newton-meter. Compliance coefficient matrix. A 2n×6 matrix representing the responses of force and bending moment at different degrees of freedom:

[0073]

[0074] Next, an evaluation function is constructed through multiple random samplings or a genetic algorithm, and the optimal function is selected. Given the condition number, the matrix selects 27 degrees of freedom from the 2n degrees of freedom of the n speckle patterns. The initial solution for multiaxial forces is achieved using the following formula:

[0075] The calibration results show that the effects of force and bending moment have a certain coupling effect on the Fz term. To address this issue, a correction function is obtained by interpolating the calibration results:

[0076] This correction function can be used to further correct for coupling effects, ultimately yielding the true multiaxial force value. The relationship between the calibrated results and the applied load is as follows: Figure 5 As shown, the experimental results demonstrate that the relationship between force and bending moment exhibits good linearity.

[0077] In summary, compared to traditional binocular multi-axis force sensors, monocular multi-axis force sensors offer advantages such as low cost (a single camera costs more than 50% less than a binocular camera), low computational cost (no need for computationally expensive binocular image matching algorithms), and high dynamic range (monocular camera modules can achieve frame rates of up to 300fps, while binocular modules only reach a maximum of 120fps; monocular camera structures do not require temporal binocular synchronization). Technically, unlike binocular or multi-view systems that reconstruct speckle spatial displacement, monocular cameras cannot fully perceive three-dimensional spatial displacement, leading to entirely different algorithm design and camera-speckle relative position design compared to binocular vision.

[0078] The core innovation and key breakthrough of this application lies in the successful realization, under the hardware constraints of monocular vision, of a stable and accurate calculation of the three-dimensional spatial forces and moments (Fx, Fy, Fz, Mx, My, Mz) acting on an elastic body by relying solely on the two-dimensional image planar displacement field of feature points. This technical approach is fundamentally different from traditional visual force sensing schemes that rely on binocular or multi-view vision for three-dimensional spatial displacement reconstruction, and overcomes a series of technical challenges arising therefrom. This invention solves the inherent problem of "inverting three-dimensional effects from two-dimensional information": monocular cameras can only acquire the projection of the three-dimensional world onto a two-dimensional imaging plane, losing depth information. Traditional visual force measurement usually relies on binocular or multi-view systems to first reconstruct the three-dimensional spatial displacement of feature points, and then couple it with a mechanical model. This invention abandons this complex and computationally intensive intermediate step, and directly establishes a high-precision mapping relationship between the two-dimensional image displacement field and the six-dimensional force / moment vector through a unique elastic body structure design, feature point pattern optimization, and an algorithm model strongly correlated with mechanical response. This essentially transforms the complex problem of "three-dimensional reconstruction + mechanical modeling" into a more efficient black-box or gray-box model recognition problem of "two-dimensional observation to multi-dimensional solution," which has significant simplicity and practicality in engineering.

[0079] Overcoming the technical bottlenecks and design challenges of monocular vision: Compared with binocular solutions, the technical improvements and breakthroughs of monocular solutions are not simply about reducing a camera, but rather about triggering a series of entirely new design requirements: Feature pattern design: The distribution, density and shape of feature points (speckles) need to be carefully designed. It is not only necessary to ensure the efficiency and robustness of image correlation algorithms in the two-dimensional plane, but more importantly, to make the generated two-dimensional displacement patterns highly distinguishable and sensitive to different three-dimensional force / torque components, so as to solve the decoupling difficulties caused by the reduction of information dimensionality under monocular conditions.

[0080] Solution Model and Algorithm: The solution model must be able to fully exploit and learn the complex nonlinear modes implicit in the two-dimensional displacement field and coupled with forces / torques in various dimensions. This has driven the development from traditional linear mapping to data-driven (such as neural networks) intelligent nonlinear mapping to compensate for the lack of explicit three-dimensional geometric information in monocular systems.

[0081] System Structure and Calibration: The relative positions of the camera and the elastic body (e.g., facing each other and maintaining a certain distance) need to be optimized to ensure that the two-dimensional displacement field can reflect the overall deformation characteristics of the elastic body to the greatest extent. The calibration process should also be performed directly on the end-to-end relationship of "two-dimensional image displacement - six-dimensional load", rather than calibrating the camera's intrinsic and extrinsic parameters first.

[0082] Therefore, the technological improvement of this invention goes far beyond the hardware simplification and cost reduction brought about by "processing only two-dimensional image information". It is also reflected in a brand-new technological paradigm: that is, accepting the information constraints of monocular vision and transforming this constraint into a comprehensive advantage of the system in terms of cost, speed and reliability through collaborative innovation at the three levels of mechanical structure, visual target and intelligent algorithm. This achieves the non-obvious technical effect of completing dynamic variable range multi-axis force measurement on a low-cost and easy-to-maintain platform.

[0083] Example 3 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0084] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0086] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A monocular variable-range multi-axis force sensor, characterized in that, include: A force-bearing system is used to withstand externally applied forces or moments and generate elastic deformation. The force-bearing system includes an upper mounting plate, an upper transparent rigid support layer, a transparent elastomer, a lower transparent rigid support layer, and a lower mounting plate. The elastomer deforms under external load to reflect the force or moment information. A deformation measurement system is used to acquire image information corresponding to the deformation of the transparent elastomer. The deformation measurement system includes a feature pattern layer disposed on one side of the transparent elastomer and a monocular camera disposed opposite to the feature pattern layer. The monocular camera is used to acquire images of the feature pattern layer in an unloaded state and a loaded state to obtain two-dimensional displacement information of the feature pattern in the image plane. The force calculation system is communicatively connected to the monocular camera and is used to directly calculate the multi-axis force / torque components based on the two-dimensional image displacement information of the feature pattern through a pre-established displacement-force mapping model.

2. The monocular variable range multi-axis force sensor according to claim 1, characterized in that, The multi-axis force sensor has various structural forms, including: single-axis distribution, optical path reflection, dual-axis distribution, and external measurement.

3. The monocular variable range multi-axis force sensor according to claim 2, characterized in that, When the multi-axis force sensor is in a uniaxial distribution form, the force bearing system and the deformation measurement system are in a straight line in space, and from top to bottom, it includes the following layered structure: The upper mounting plate is a top-supporting component and has mounting holes for applying or transferring the load to be measured. A transparent rigid support layer is used to restrict non-target deformation, allowing elastic deformation to occur in the elastic body; The feature pattern layer has a pseudo-random speckle pattern, a regular matrix pattern, a QR code pattern, or a combination thereof. A reflective layer is also provided on the feature pattern layer to enhance the contrast of the feature pattern. A transparent elastomer, wherein the transparent elastomer material includes polymer or metal material, and the geometric parameters of the elastomer include thickness, area, and structural morphology, is used to adjust the overall stiffness of the sensor by changing the geometric parameters to achieve range switching; A transparent rigid support layer is used to restrict non-target deformation, allowing elastic deformation to occur in the elastic body; A monocular camera, facing the feature pattern layer, is used to continuously acquire feature pattern images; The lower mounting plate serves as the bottom support component.

4. The monocular variable range multi-axis force sensor according to claim 3, characterized in that, When the multi-axis force sensor is in the form of optical path reflection, the rest of the structure remains the same as the single-axis distribution form, only a reflector is added to change the direction of the optical path to shorten the axial length of the sensor. When the multi-axis force sensor is in a biaxial distribution form, the force bearing system and the deformation measurement system are respectively located on two axes in space, and the rest of the structure is consistent with the single-axis distribution form; When the multi-axis force sensor is in the form of an external measurement, the monocular camera is located externally. The sensor is not spatially packaged as a whole, but the monocular camera is fixed to the lower transparent rigid support layer or its relative displacement is known. The rest of the structure is consistent with the single-axis distribution form.

5. A multi-axis force measurement method based on monocular vision, applied to the multi-axis force sensor according to any one of claims 1-4, characterized in that, include: The multi-axis force sensor is fixed to the measurement system via upper and lower mounting plates. The image of the feature point under unloaded state is acquired as the reference image, and the initial image coordinates of each feature point are recorded. The force / torque to be measured is applied to a rigid mounting plate to cause deformation of the elastomer and displacement of feature points. The image of the feature points after deformation is acquired in real time by a monocular camera. The two-dimensional image plane displacement field of the deformed feature point image relative to the reference image is calculated using an image correlation algorithm. The two-dimensional image planar displacement field is input into a pre-calibrated displacement-force mapping model, and a real-time multidimensional force / torque vector is output. The mapping model does not perform three-dimensional spatial displacement reconstruction, but directly establishes a mapping relationship between the two-dimensional image displacement data and the multidimensional force / torque.

6. The method according to claim 5, characterized in that, The displacement-force mapping model is obtained in the following way: A series of known uniaxial forces / torques are applied to the sensor, and the corresponding displacement fields of feature points are recorded. A linear transformation matrix is ​​formed by fitting the data using the least squares method, and then a displacement-force mapping model is constructed.

7. The method according to claim 5, characterized in that, The displacement-force mapping model is obtained in the following way: A neural network model is trained based on calibration data, and a nonlinear mapping from displacement field to force / torque is established to obtain a displacement-force mapping model.

8. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 5-7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 5-7.

Images