Methods for force inference, methods for training feedforward neural networks, force inference modules, and sensor configurations.

Abstract

Improve sensor configurations for robotics. The invention relates to a method for force inference of a sensor configuration using image data, a corresponding method for training a feed-back neural network, a corresponding force inference module, and a corresponding sensor configuration.

Description

Technical Field

[0001] The present invention relates to a method for inferring force in the configuration of a sensor using image data, a method for training a corresponding feedforward neural network, a corresponding force inference module, and a corresponding sensor configuration.

Background Art

[0002] When developing applications such as robots, sensing the forces applied to the hands, legs, or other parts of the robot such as operating devices is important for enhancing the functions of the robot to move around and manipulate objects. Known implementations of sensor configurations that can be used in robotic applications to obtain feedback regarding the applied forces are very expensive and do not have sufficient resolution.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The problems of the present invention are, therefore, to provide a method for inferring force that is an alternative to or an improvement compared with existing methods. A further problem of the present invention is to provide a method for training a feedforward neural network corresponding to the above-described method for inferring force. A further problem of the present invention is to provide a force inference module and a sensor configuration corresponding to the above-described method for inferring force.

Means for Solving the Problems

[0004] The problems are achieved by the subject matter of the main claims. Preferred embodiments can be derived, for example, from the dependent claims. The content of the claims becomes the content of the description by explicit reference.

[0005] The present invention relates to a method for inferring force in the configuration of a sensor for detecting force.

[0006] The sensor configuration is, for example, a sensor configuration to which the present method can be applied, and particularly includes at least the following. An elastically deformable wall, the wall having an outer measuring surface and an inner reflective surface, where the reflective surface partially divides the internal space, the wall and, A light source configuration comprising multiple light sources arranged to emit light toward the interior space, Image sensor installed inside the internal space That is the case.

[0007] For details regarding the sensor configuration, refer to the descriptions elsewhere in this specification. All embodiments and modifications may be applied.

[0008] The method for inferring force involves the following steps: The steps include reading image data from an image sensor, A step of calculating a force map on a measurement surface based on image data, preferably using a feedforward neural network, wherein the force map comprises multiple force vectors. That is the case.

[0009] Such methods provide highly accurate force inference based on image detection. Using a feedforward neural network eliminates the need to implement analytical force assessment. The neural network can be trained as disclosed herein. The training has been shown to enable the detection of multiple indenters, as well as the identification of the indenter's position, force direction, and shape, leading to highly accurate and detailed force inference.

[0010] The force map may be a map defined on an actual measurement surface, where the force map may comprise multiple map points. At each map point, some information, such as force vectors, can be determined, as will be further described below. The force map typically provides information about forces applied to the measurement surface. For example, such forces may originate from one or more indenters pushing against the measurement surface, or from an object currently being manipulated by the sensor configuration (e.g., if the sensor configuration is a robot's fingertip).

[0011] Feedforward neural networks can be artificial neural networks in particular. They take image data as input and output a force map. In principle, feedforward neural networks are artificial neural networks in which the connections between nodes do not form cycles.

[0012] The following describes the training methods for the network. The training procedures mentioned in this section are to be considered as steps performed before the actual force measurement and force inference are carried out. Therefore, the force inference method can be considered as a combination of the training steps performed before force inference and force inference using the trained network. The force inference method can be considered as force inference itself, using the network trained accordingly. Further, another training method is described below. This training method may be performed independently of any force inference. Typically, force inference, in which an image sensor is read and a force map is generated, is considered an act performed in a particular use case, i.e., when the sensor configuration is used to measure or evaluate the force acting on the measurement surface. When the sensor configuration is used to measure or evaluate the force acting on the measurement surface, for example, it is because the sensor configuration is currently manipulating an object, or otherwise in contact with an object that is applying pressure to the measurement surface.

[0013] According to a preferred implementation, the feedforward neural network was trained in the following steps, which are performed before force inference: - A step of performing multiple force tests on the sensor configuration, wherein each force test is performed The configuration of the sensor involves applying force to a position on the measurement surface using a single indenter, and simultaneously measuring the force applied by the indenter. A step of performing multiple force tests, which includes simultaneously reading image data from an image sensor, - For each force test, the step is to perform a corresponding simulated test using one model of sensor configuration, Each simulation test includes applying simulated forces to the simulated measurement surface of the model, thereby calculating a map of the simulated forces on the simulated measurement surface. The simulated force map contains multiple simulated force vectors, The steps include performing a simulated test, wherein the simulated force corresponds to the measured force and is applied to a position on the simulated measurement surface that corresponds to the position on the measurement surface, - Steps to train a feedforward neural network using image data and corresponding computed simulated power maps. That is the case.

[0014] Such training steps can provide proper training for a feedforward neural network. The feedforward neural network can learn real-world forces and corresponding force maps, the former obtained from measurements and the latter from simulations.

[0015] Force measurement and image data retrieval are typically performed while the force is actually being applied, preferably in a stationary state. The force is then used in a simulated test.

[0016] It should be noted that all terms expressed as "simulated" (tested) typically relate to simulated testing. For example, a simulated measurement surface is a measurement surface that exists only in a simulated test. The model can calculate a force map in a deterministic manner from the applied simulated forces. For example, a simple spatial distribution of forces around a point using Hertzian contact theory may be used. As an alternative, a finite element model can be used.

[0017] A simulated force may be the same as a measured force. This may mean that the simulated force has the same components in three dimensions, or that it has the same direction and absolute value. However, the simulated force may also correspond to a measured force according to a predefined relationship.

[0018] Preferably, the force test for training a feedforward neural network is performed using multiple indenters, each having a different indenter shape. In particular, the indenter shapes may differ. Thus, the feedforward neural network can be trained to distinguish between different indenter shapes, i.e., to generate different force maps when different indenters are applied.

[0019] For example, the indenter shape is selected from the group comprising at least a tip, a circle, a triangular cross-section, a square cross-section, a hemisphere, a cube, and a cylinder. All of the above indenter shapes may be used in the training process, or only a subset of them may be used. Other indenter shapes may also be used.

[0020] Preferably, the simulation test is performed with simulated forces based on simulated indenters, each having a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test. Multiple such simulated indenters may each have a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test. Therefore, the simulated force applied to the simulated measurement surface will more closely correspond to the actual force because the indenter shapes are similar. This improves the training of the feedforward neural network.

[0021] Preferably, the feedforward neural network was trained using multiple different indenter shapes. This allows the feedforward neural network to be trained to distinguish between the forces generated by different indenter shapes.

[0022] In one implementation, a feedforward neural network was trained using multiple indenters of different sizes. This allowed the feedforward neural network to be trained to distinguish between the forces generated by different indenter sizes.

[0023] Preferably, the feedforward neural network was trained with the indenter applied for each shear force for at least a portion of the force tests used to train the feedforward neural network. This allows the feedforward neural network to be trained to distinguish between different shear forces applied to the measurement surface. For example, the force map may also include the simulated shear forces. In particular, different shear forces may result in different force maps.

[0024] Preferably, the measured force comprises a normal force component, a first shear force component, and a second shear force component, respectively. This defines the strength and direction of the force in the coordinate system. In particular, a global coordinate system may be used. However, other force representations may also be used.

[0025] Preferably, in the measured force, the first shear force component corresponds to the first shear force, and the second shear force component corresponds to the second shear force. In particular, the first shear force is perpendicular to the second shear force.

[0026] Each measured force may have three components in a reference coordinate system. The reference coordinate system may be a global coordinate system. Different representations requiring only standard mathematical transformations are considered equivalent.

[0027] Preferably, the feedforward neural network was trained using multiple forces with different shear force components. This allows the feedforward neural network to be specifically trained to distinguish between different shear forces. In particular, different shear forces in the actual applied forces may result in different force maps.

[0028] Preferably, the feedforward neural network was trained using multiple forces with different normal force components. This allows the feedforward neural network to be trained to distinguish between various normal forces. The normal force component may be a component of the force locally perpendicular to the surface.

[0029] The force may be measured using a force sensor placed inside or next to the indenter. This allows for direct measurement of the force. In particular, the force sensor can measure not only the absolute value of the force but also its corresponding direction. The shear force can be derived from the orientation.

[0030] Preferably, each simulated force vector comprises a normal force component, a first shear force component, and a second shear force component. Thus, the simulated force vectors can be similar to the shear force components on a simulated force map that can be used to train a feedforward neural network.

[0031] Preferably, in the simulated force vector, the first shear force component corresponds to the first shear force, and the second shear force component corresponds to the second shear force. In particular, the first shear force is perpendicular to the second shear force.

[0032] Preferably, each simulated force vector has three components in the reference coordinate system. The reference coordinate system may be used to represent all forces in the method; however, other representations may also be used.

[0033] Preferably, the image data on which the calculated force map is based includes several invariant images, preferably three, in addition to the image data read from the image sensor. The image data read from the image sensor may be referred to as variable images. The invariant images can be set as follows. This has shown improvement in the results of feedforward neural networks. The invariant images may be used unchanged in all training and force inference steps.

[0034] Preferably, the invariant image is at least one of a grayscale gradient image, a structural image, and a reference light pattern. Such images have proven suitable for typical force inference applications.

[0035] Preferably, the variable image, as part of the image data, was captured immediately before calculating the force map. Thus, the force map corresponds to the actual state of the sensor configuration.

[0036] In a preferred implementation, the force map is in mm 2 (1×10 -6 m 2 ) with at least 0.25 force vector per mm 2 with at least 0.5 force vector per mm 2 with at least 0.75 force vector per mm 2 with at least 1 force vector per mm 2 with at least 1.5 force vector per mm, or 2 with at least 2 force vectors per mm. In a preferred implementation, the force map is in mm 2 with a maximum of 0.25 force vector per mm 2 with a maximum of 0.5 force vector per mm 2 with a maximum of 0.75 force vector per mm 2 with a maximum of 1 force vector per mm 2 with a maximum of 1.5 force vector per mm, or 2 with a maximum of 2 force vectors per mm. To provide appropriate spacing, relatively low values may be combined with relatively high values respectively. However, other values can also be used.

[0037] Preferably, each force vector comprises a normal force component, a first shear force component and a second shear force component. Thus, the force vector can provide not only the normal force but also the shear force components.

[0038] Preferably, the first shear force component corresponds to the first shear force and the second shear force component corresponds to the second shear force. The first shear force may be particularly perpendicular to the second shear force.

[0039] Preferably, each force vector comprises three components in a reference coordinate system. This can also indicate the direction of the force, and thus can indicate the shear force. Note that a force vector not perpendicular to the local surface typically comprises a shear force.

[0040] Preferably, a feedforward neural network is trained or a force map is calculated using additional images of the reflective surface of the sensor configuration without external impact as part of the image data. This can improve detection accuracy.

[0041] Preferably, an image of the wall structure of the sensor configuration is used as part of the image data to train a feedforward neural network or to calculate a force map. This can also improve detection accuracy.

[0042] Preferably, a grayscale gradient image for location encoding is used as part of the image data to train a feedforward neural network or to compute a force map. This can also improve detection accuracy.

[0043] Such additional images, such as images of reflective surfaces without external impact, images of the structure, and grayscale gradient images, can provide at least one of improved detection accuracy and better training. These images may be immutable images and may be used as part of the image data in addition to variable images that may be read from the image sensor. Images of reflective surfaces without external impact may, in particular, be images of the reflective surface taken when no force is applied to the measurement surface.

[0044] The following describes an alternative method for training feedforward neural networks. This method is not part of the force inference method, but is performed separately for network training. To avoid repetition, the previously described methods for network training and force inference will be referenced for their respective features.

[0045] This invention relates to a method for training a feedforward neural network. The feedforward neural network preferably calculates a force map on the measurement surface of the sensor configuration based on image data from the image sensor, and the force map comprises a plurality of force vectors. Here, the feedforward neural network is trained in the following steps. - A step of performing multiple force tests on the configuration of the sensor, The test of multiple forces comprises the steps of performing a test of each force, which involves applying a force with one indenter to a position on the measurement surface of the sensor configuration, simultaneously measuring the force applied by the indenter, and simultaneously reading image data from the image sensor. - For each force test, the step is to perform a corresponding simulated test using one model of the sensor configuration, Each simulation test includes applying simulated forces to the simulated measurement surface of the model, thereby calculating a map of the simulated forces on the simulated measurement surface. The simulated force map contains multiple simulated force vectors, The simulated force corresponds to the measured force and is applied to the position on the simulated measurement surface that corresponds to the position on the measurement surface. Steps to take a mock exam, - Steps to train a feedforward neural network using image data and corresponding simulated power maps. That is the case.

[0046] In one implementation, the force test for training a feedforward neural network is performed using multiple indenters, each with its own indenter shape.

[0047] In one implementation, the indenter shape is selected from the group comprising at least a tip, a circular shape, a triangular cross-section, a square cross-section, a hemisphere, a cube, and a cylinder.

[0048] In one implementation, the simulated test is performed by applying or by applying multiple simulated forces based on the simulated indenters, each preferably having a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test.

[0049] In one implementation, the simulation test is performed with multiple simulated forces based on multiple simulated indenters, each having a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test.

[0050] In one implementation, a feedforward neural network is trained using multiple indenters of different sizes.

[0051] In one implementation, a feedforward neural network is trained with multiple indenters, each with a different shear force, for at least part of the force tests used to train the feedforward neural network.

[0052] In one implementation, each measured force comprises a normal force component, a first shear force component, and a second shear force component.

[0053] In one implementation, among the multiple measured forces, the first shear force component may correspond to the first shear force, the second shear force component may correspond to the second shear force, and the first shear force may be perpendicular to the second shear force.

[0054] In one implementation, each measured force has three components within the reference coordinate system.

[0055] In one implementation, the feedforward neural network is trained using multiple forces with different shear force components.

[0056] In one implementation, a feedforward neural network is trained using multiple forces with different normal force components.

[0057] In one implementation, the force is measured using a force sensor placed inside or next to the indenter.

[0058] In one implementation, each vector of the simulated force has a normal force component, a first shear force component, and a second shear force component.

[0059] In one implementation, among the multiple vectors of simulated force, the first shear force component may correspond to the first shear force, the second shear force component may correspond to the second shear force, and the first shear force may be perpendicular to the second shear force in particular.

[0060] In one implementation, each simulated force vector has three components within the reference coordinate system.

[0061] 1. According to the implementation, a feedforward neural network is used in the force inference method described herein. All embodiments and modifications of the force inference method may be applied.

[0062] In each implementation, the force map is mm 2 A force vector of at least 0.25 per mm 2 A force vector of at least 0.5 mm per unit area. 2 A force vector of at least 0.75 per unit, mm 2 A force vector of at least 1 per unit, mm 2 A force vector of at least 1.5 per unit, or mm 2 It may have at least 2 force vectors.

[0063] In each implementation, the force map is mm 2 Maximum force vector of 0.25 per unit, mm 2 Maximum force vector of 0.5 per unit, mm 2 Maximum force vector of 0.75 per unit, mm 2 Maximum force vector of 1 per unit, mm 2 A force vector of up to 1.5 per unit, or mm 2 It may have a force vector of up to 2 per unit.

[0064] In one implementation, each force vector comprises a normal force component, a first shear force component, and a second shear force component.

[0065] In one implementation, the first shear force component may correspond to the first shear force, and the second shear force component may correspond to the second shear force. The first shear force may be perpendicular to the second shear force in particular.

[0066] In one implementation, each force vector has three components within the reference coordinate system.

[0067] In one implementation, a feedforward neural network is trained using image data, as well as additional images of the reflective surface of the sensor configuration without external impact.

[0068] In one implementation, a feedforward neural network is trained using images of the wall structure of the sensor configuration as part of the image data.

[0069] In one implementation, a feedforward neural network is trained on grayscale gradient images for encoding position as part of the image data.

[0070] In one implementation, a feedforward neural network is trained on one or more of the following: grayscale gradient images, images of the structure, and reference light patterns.

[0071] 1. In this implementation, the sensor configuration is that of a force-sensing sensor. The sensor configuration may particularly include one or more of the following: An elastically deformable wall comprising an outer measuring surface and an inner reflective surface, wherein the reflective surface partially divides the internal space, and the wall, A light source configuration that has multiple light sources and is configured to emit light toward the interior space, Image sensor installed inside the internal space That is the case.

[0072] See further descriptions of sensor configurations elsewhere in this specification. All embodiments and modifications described may apply.

[0073] 1. In this implementation, the sensor configuration is that of a force-sensing sensor. The sensor configuration may particularly consist of one or more of the following: - Base and, - A top having an elastically deformable wall, the top being attached to the base such that the top and base define an internal space, the wall having an outer measuring surface and an inner reflective surface, where the reflective surface partially demarcates the internal space, the top and - A light source configuration that includes multiple light sources attached to the base and is configured to emit light toward the internal space, - An image sensor mounted on the base within the internal space and That is the case.

[0074] For further possible aspects of the sensor configuration, refer to the descriptions elsewhere in this specification. All embodiments and modifications are applicable.

[0075] The sensor configuration may be at least one of the following: the robot's end effector and the robot's operating element. This allows for the integration of the function into the robot. However, other implementations and applications are also possible.

[0076] The present invention further relates to a force inference module for force inference of a force-sensing sensor configuration, wherein the force inference module is configured to perform the force inference method described herein. All implementations and modifications of this method can be applied.

[0077] The force inference module may be implemented, for example, as a microcontroller, microprocessor, field-programmable gate array, application-specific integrated circuit, or computer. In particular, it may include processing means and storage means, where the storage means stores program code that causes the processing means to perform the method disclosed herein.

[0078] The present invention further relates to a configuration of a force-sensing sensor, wherein the sensor configuration comprises one or more of the following: - Base and, - A top having an elastically deformable wall, the top being attached to the base such that the top and base define an internal space, the wall having an outer measuring surface and an inner reflective surface, where the reflective surface partially demarcates the internal space, the top and - A light source configuration that includes multiple light sources attached to the base and is configured to emit light toward the internal space, - An image sensor mounted on the base within the internal space, - The force inference module described here and That is the case.

[0079] With respect to the force inference module, all embodiments and variations described herein may be applied.

[0080] The present invention further relates to a computer program product that performs the method disclosed herein. The present invention further relates to program code that performs the method disclosed herein. The present invention further relates to a non-volatile computer-readable storage medium in which the program code is stored, wherein the program code, when executed by a processor, causes the processor to perform the method disclosed herein. With respect to the method, all embodiments and variations disclosed herein may be applied.

[0081] Further inventive aspects are described below. These aspects can be considered alone, for example, in combination with other features disclosed herein. They can also be considered as separate inventive aspects and may be the subject of claims.

[0082] The present invention relates to a configuration of a force-sensing sensor. The sensor configuration comprises a base. The sensor configuration comprises a top having an elastically deformable wall. The top is attached to the base such that the top and base define an internal space. The wall has an outer measuring surface and an inner reflective surface, where the reflective surface partially divides the internal space.

[0083] The sensor configuration includes a light source configuration comprising multiple light sources mounted on a base and arranged to emit light toward a reflective surface. The sensor configuration also includes an image sensor with a detection surface for observing at least a portion of the reflective surface.

[0084] Using this sensor configuration, the force applied to the measurement surface can be detected by optical image recognition. The light reflected from the reflective surface generates a pattern on the image sensor that depends in a way that is highly sensitive to the force applied to the measurement surface. Therefore, such a light pattern can be used to perform highly sensitive force inference.

[0085] In particular, a light source can generate structured light within the internal space.

[0086] The base is typically the component that defines the foundation of the sensor's configuration. For example, it can be fixed to some means of holding, particularly a component of a robot. The top is typically the component that comes into contact with the object to which the force is applied, or the component that applies the force to the sensor's configuration.

[0087] When an element is attached to a base, it may also mean that it is attached to another element on the base, or that it is part of the base. This may be true for elements such as light source components, collimators, and image sensors. For example, an element attached to a base may be attachable to a support structure, which would provide stability and connectivity to other elements.

[0088] A wall is typically a component that comes into contact with an external object and deforms when the object applies a force to it or when a force is applied to it, and vice versa. Possible implementations of walls are described further below.

[0089] The internal space is typically completely enclosed and defined by a top and a base. The internal space may be hollow; however, it may be filled with material. Typically, light emanating from a light source propagates through the internal space toward a reflective surface, is reflected by the reflective surface, and then propagates to an image sensor.

[0090] The term "sensor configuration" should be understood particularly in the context of force detection, as a configuration (device) of multiple components that perform the function of a sensor.

[0091] In particular, each light source may have its own color. Preferably, the light source configuration includes light sources with at least two or three different colors. This results in patterns of light of different colors on the detection surface, and thus allows for identification of the light as originating from one or at least some of the light sources, potentially leading to a better evaluation. However, the same color may also be used for the light sources.

[0092] The light source and the detection surface are preferably arranged such that light emitted from the light source and reflected by the reflective surface generates a pattern of light on the detection surface. This pattern of light can be used for evaluation purposes, particularly using a neural network. The pattern of light has been shown to be highly sensitive to the force applied to the measurement surface. This is also related to the direction and shape of the force applied by the indenter. Therefore, such a pattern of light has been found to constitute a powerful indicator of the applied force.

[0093] In particular, the light pattern on the detection surface changes with deformation of the measurement surface. Such changes can be evaluated for inferring force or force maps.

[0094] In particular, the color distribution of light reflected from the reflective surface to the detection surface changes with deformation of the measurement surface. Such color distributions can change especially when different colors are used for the light source. Changes in color distribution have been shown to be highly sensitive to applied forces.

[0095] In one implementation, the reflective surface is diffuse. This specifically means that light striking the reflective surface is reflected not as if it were a mirror, but rather as if it were at least a slightly rough surface. Light may be slightly attenuated by reflection at a diffuse reflective surface. In an alternative implementation, a mirror-like reflective surface may also be used.

[0096] In particular, the image sensor can be mounted in at least one of the base and / or internal space. This is a simple and reliable implementation and reduces the distance to the reflective surface. However, the image sensor may be mounted in other locations.

[0097] One, several, or all light sources may have adjustable colors. This is particularly useful for setting colors during the setup procedure. However, fixed colors can also be used.

[0098] One, several, or all of the light sources may have adjustable brightness. This is particularly useful for setting the brightness during the setup procedure. However, fixed brightness is also available.

[0099] Preferably, the light source is positioned around the image sensor. This can lead to a favorable distribution of light within the internal space.

[0100] Preferably, the light source is positioned to emit light such that it produces a distribution of reflected light on the detection surface that does not result in an intensity exceeding the saturation density. This prevents oversaturation, which can lead to errors when estimating forces. For example, variables that can be adjusted to prevent oversaturation are the brightness of the light source, the position and size of the collimator, the camera exposure time, and the lens. However, other variables can also be used for this purpose.

[0101] According to a preferred embodiment, the light source is a light-emitting diode. Such a light-emitting diode is a reliable light source. However, other types can also be used.

[0102] Preferably, the sensor configuration includes multiple collimators, each assigned to a single light source, and defining at least one of the illumination angle and the cone of the emitted light. Using such collimators, the propagation of light rays to the reflective surface can be controlled with great precision.

[0103] One, some, or all collimators may be positioned ascentrically relative to the assigned light source. This provides suitable light distribution for many applications. However, a centered arrangement may also be used.

[0104] A collimator can be embodied, in particular, as a hole within a collimator ring. This demonstrates the ease of realizing a collimator. The collimator ring may block light other than that passing through the hole.

[0105] At least one of the light source and the collimator can be positioned to illuminate at least 80%, at least 85%, at least 90%, or 100% of the reflective surface, or the intended measurement area, or the measurement surface or an area inside the intended measurement surface. Thus, a large portion of such a surface or area can be used. The intended measurement area may, in particular, be a certain area, i.e., a subpart of the measurement surface. The area inside the measurement surface may be the area on the wall opposite to the measurement surface. The intended measurement surface may be a subpart of the measurement surface, where the measurement surface can be defined as a surface that can, in principle, be used for force inference.

[0106] At least one of the light source and collimator can be positioned such that at least 60%, at least 70%, at least 80%, at least 90%, or 100% of the reflective surface is directly illuminated by at least one of the light sources (up to four) and at least two. This creates a pattern of light on the detection surface, which has been proven to be quite effective for force inference. Directly illuminated components are those in which light rays propagate directly from the light source to the component without being reflected by other components on the reflective surface.

[0107] According to one implementation, the collimator has a collimator hole diameter of at least 0.8 mm. According to one implementation, the collimator has a collimator hole diameter of up to 4 mm. Such diameters have proven suitable for typical applications. However, diameters and shapes other than round holes can also be used.

[0108] The light sources may be arranged in a circular pattern at the base. Thus, the light sources can form or define a circle. This results in a predictable light pattern and is particularly suitable for typical implementations where the top is circularly symmetrical.

[0109] The detection surface may be configured and positioned to view at least 70%, at least 80%, at least 90%, or the entire reflective surface. This may, in particular, mean that the detection surface can detect light reflected from such portions of the reflective surface. This results in high measurement accuracy.

[0110] In particular, the image sensor may be a color camera sensor or color-sensitive. This enhances the measurement capabilities compared to monochrome or grayscale sensors, and in principle, the image sensor can be used.

[0111] In particular, an image sensor may have multiple pixels, each configured to detect light individually. Therefore, the number of pixels can determine the resolution of the photodetector.

[0112] The detection surface may face the reflective surface. Therefore, light reflected by the reflective surface can propagate directly to the detection surface, and ultimately may pass through some optical means, such as a wide-angle lens as described herein. However, other implementations are also possible.

[0113] The detection surface may be parallel to the inner surface of the base. This allows for simple implementation. However, other orientations are also possible.

[0114] The detection surface may be configured to detect at least one of a light pattern and / or an image. Such a light pattern or image may be used for force inference, as described herein.

[0115] The image sensor may be configured to detect at least one of a light pattern and / or an image at a frame rate. Thus, frames are typically detected sequentially with a constant time interval between them.

[0116] In particular, the frame rate can be at least 10fps (frames per second), at least 20fps, at least 30fps, at least 50fps, or at least 100fps. Specifically, the frame rate can be up to 30fps, up to 50fps, up to 100fps, or up to 200fps. Such values ​​have proven suitable for general use. Each lower value can be combined with a higher value to form an appropriate interval. However, other values ​​may also be used.

[0117] The image sensor may include a wide-angle lens or a fisheye lens optically positioned between the reflective surface and the detection surface. This may improve image detection. For example, the image detector may ensure that a specific portion of the reflective surface is observed.

[0118] In particular, the internal space may be a hollow space. This may mean that the space is filled with air. However, it can be filled with other materials, especially optically transparent materials such as fluids, solids, glass, and elastomers.

[0119] At least one of the light source and the collimator may be configured to emit light into each cone. Such cones may be defined by the angles of each cone that divide the outer dimensions of the cone. The cones of light may have a cross-sectional area that increases at a constant rate with increasing distance from the light source. The collimator may also be used to determine such a shape after the light has passed through it.

[0120] In particular, one, several, or all of the cones may have a conical axis that is inclined outward by more than 0° with respect to at least one of the common axis and the axis perpendicular to the base. This may help prevent supersaturation at the tip of the apex. This may improve the measurability of forces near the tip. The base may extend particularly along a plane, with its axis perpendicular, i.e., perpendicular.

[0121] In particular, one, several, or all cones may have conical axes that are tilted outward by up to 10° with respect to at least one of the common axis and the axis perpendicular to the base. This has proven suitable for general applications. However, larger angles can also be used.

[0122] In particular, one, some, or all of the cones may have at least one outer cone angle between at least 35° and up to 80°. The outer cone angle may be the angle of the outer boundary of the cone with respect to at least one of the cone axis and the direction of central propagation. The cone axis may define at least one of the center of the cone and the direction of light propagation.

[0123] In particular, the cones may partially overlap in a plane perpendicular to the common axis, where the overlap depends on the distance between the plane and the base. This would omit the portion of the unlit reflective surface, making it unusable for force inference. Specifically, the light source and image sensor may be arranged such that the light emitted by the light source and reflected by the reflective surface is detected by the image sensor.

[0124] According to one implementation, the reflective surface is covered with at least one of a pattern and / or multiple traceable objects. This would improve the accuracy of force inference. According to an alternative implementation, the reflective surface is a smooth surface.

[0125] According to one embodiment, the sensor configuration includes a common axis. In particular, the top may be perfectly circular around the common axis. This corresponds to a simple implementation. Alternatively, the top may be partially circular around the common axis. This would allow for certain structures, such as thin areas as described herein.

[0126] The detection surface may, in particular, be perpendicular to the common axis.

[0127] The light source may be positioned to emit light parallel to the common axis. This would result in a geometrically simple implementation. This emission is, in particular, the emission before possible deflection or occlusion by a collimator.

[0128] In particular, different parts of the reflective surface may be illuminated by light sources of different colors from different directions. This improves the ability to infer forces.

[0129] According to one embodiment, the top can be tapered with an outer diameter that decreases as the distance from the base increases. This will result in an outer shape suitable for typical use cases where the pointed object is used for manipulating purposes or for measuring specific forces.

[0130] In particular, at least one of the top and the wall may be cone-shaped.

[0131] In particular, the wall may be configured to relay deformation from the measurement surface to the reflective surface. This specifically means that a force applied to the measurement surface induces deformation of the measurement surface, and this deformation is relayed through the inner portion of the wall to the reflective surface, which is also deformed, resulting in different light reflections at specific points or regions. These different reflections will result in different light patterns on the detection surface, which may be measured and evaluated.

[0132] Preferably, the top is detachably attached to the base. This allows the base to be reused with a different top, for example, if the top is damaged or loses characteristics relevant to the measurement purpose. For example, the base may remain fixed to the robot or part of another component, and the top may be replaced as needed.

[0133] Removable mounting may, in particular, mean that specific means are provided for releasing the top from the base without damaging the components.

[0134] In particular, the top portion may be attached to the base portion in a replaceable manner.

[0135] The top may be releasably attached to the base by at least one of a bayonet mount and a screw connection, in addition or alternatively by a pair of tongues and corresponding notches. Such connections have proven to provide easy replacement and secure connection. However, other connection methods may also be used.

[0136] In one implementation, the top section comprises only a wall. In this case, the wall typically provides sufficient stability on its own. In particular, the wall may consist only of homogeneous wall material, especially if it is not accompanied by a supporting structure made of another material.

[0137] 1. According to the implementation, the top section comprises a structural element located inside the wall. Such a structural element will provide additional rigidity. It may be made of a different material than the wall. In particular, the structural element may comprise multiple wires or wire-like elements.

[0138] Preferably, the structure is lattice-like. This may mean that the structure is made of relatively thin wires or other elements that leave spaces between them. These spaces are typically filled with wall material. Furthermore, the structure is typically enclosed, either completely or partially, by wall material.

[0139] The frame may preferably be made of steel, stainless steel, or aluminum. Such materials provide sufficient stability. However, other materials can also be used. In particular, any material can be used that can withstand the desired maximum force and that can be manufactured into the required shape (e.g., by three-dimensional fabrication).

[0140] For example, aluminum or stainless steel can be used for the frame. Other materials such as copper, bronze, brass, and carbon fiber are also possible.

[0141] In particular, the structural frame is rigid or semi-rigid. This ensures sufficient rigidity.

[0142] In particular, walls may be equipped with wall materials. This can be suitable for transmitting deformation from the measurement surface to the reflective surface.

[0143] Preferably, the wall material contains an elastomer. For example, Smooth-On's Ecoflex 00-30, Ecoflex 00-35, Ecoflex 00-50, etc., can be used. Particularly, a soft elastomer with a high elongation rate can be used. For example, the elongation rate of the wall material may be at least 800% and at least one of 1,000% or 900%.

[0144] Preferably, the wall material contains at least one of aluminum powder and aluminum flakes. This indicates that it gives a desirable reflectivity to the reflective surface. For example, at least one of the powder and flakes is present in creating a suitable type of reflective surface. Changes in deformation or changes in the angle to the light source may result in changes in shading. In particular, aluminum powder can be used to divert ambient light, and aluminum flakes can be used to increase reflectivity. The powder typically has a smaller diameter than the flakes.

[0145] The structural frame is preferably enclosed by wall material. This improves the stability of the wall material.

[0146] In particular, the structural frame may be completely overmolded by the wall material. Alternatively, the structural frame may be partially overmolded by the wall material.

[0147] The structural frame is particularly reusable. This means, for example, that the wall material surrounding the frame can be dissolved in some solvent, leaving the frame without the surrounding wall material. Then, new wall material can be overmolded.

[0148] In particular, the wall thickness may be at least 0.8 mm or at least 1.2 mm.

[0149] In particular, the wall thickness is a maximum of 4 mm or 5 mm.

[0150] While these values ​​are suitable for general use, other values ​​can also be used. The given values ​​may relate particularly to the outer wall portion of a thin area.

[0151] According to one implementation, the wall has thin sections that are thinner than the rest of the wall. Such thin sections will give particularly high sensitivity in this local area.

[0152] Thin areas may be located, in particular, on the opposite side of the base. High sensitivity would be desirable in such locations.

[0153] The thin area may be shaped to conform to the fingernail (area). For example, the sensor configuration may be shaped like a thumb or another finger. The thin area may be placed at the location of the fingernail (area). The thin area may be completely or at least substantially flat, or only slightly curved. In general, the thin area may have any shape.

[0154] In particular, the thin area may be less than one-quarter of the measurement surface. This provides appropriate, increased measurement capability without compromising stability.

[0155] Preferably, the thin area has a thickness of at most 0.8 mm or a maximum of 1.2 mm, and at least one of the following: a thickness of at least 30% or a maximum of 50% of the thickness of the wall outside the thin area. This provides a suitable improvement in measurement sensitivity. However, other values ​​may also be used. The wall may have a uniform thickness outside the thin area. However, if the wall thickness is not uniform, the criterion for defining the relative thickness of the thin area may be an average value.

[0156] The sensor configuration could be at least one of the following: a fingertip or a robotic control element. Therefore, the sensor configuration can have two functions: manipulating the element and measuring the applied force. However, other implementations are also possible.

[0157] The present invention further relates to a method for fabricating a top for the configuration of a sensor, the method comprising the following steps: - The process of providing a structural frame made of structural materials. The structural frame encloses the interior space. - A step of covering the structure with wall material such that the wall material forms an elastically deformable wall that defines an outer measuring surface and an inner reflective surface. The reflective surface divides the interior space.

[0158] This allows for the easy and reliable manufacture of the top section. Details of the elements are described elsewhere in this specification. All descriptions given regarding the method are, in principle, applicable to the structural aspects, and vice versa.

[0159] In particular, the structural material may be stronger than the wall material. Therefore, the structure provides stability. The structure may also be more rigid than the wall material.

[0160] The process of providing the structural frame may include the three-dimensional modeling of the frame. In other words, the frame may be three-dimensionally modeled. This improves the flexibility of the frame design. However, other processes can also be used.

[0161] The structure may consist of multiple wires (wiring) with openings formed between them. Such openings may be filled with wall material.

[0162] Preferably, the structure may be in the shape of a dome or a cone.

[0163] Covering can be achieved, in particular, by overmolding. This is a reliable method of placing a material such as elastomer around the structure. However, other methods can also be used.

[0164] The internal space may be hollow at the top. However, it may be filled with, for example, an optically transparent material.

[0165] This method may further include the step of covering the reflective surface with at least one of a pattern and several traceable objects. Alternatively, the reflective surface may be left smooth.

[0166] In particular, structured casts may be applied to generate patterns. Such structured casts may correspond to patterns.

[0167] The structure may have a common axis. The structure may be partially or completely circular around the common axis.

[0168] The frame may have a taper, where the outer diameter decreases towards the tip. This can correspond to a desirable shape at the top.

[0169] The wall material may be configured to relay deformation from the measurement surface to the reflective surface.

[0170] The structure may be in the form of a grid.

[0171] The frame may be made of steel, stainless steel, or aluminum. Further alternatives are described elsewhere in this specification.

[0172] The structural frame can be rigid or semi-rigid.

[0173] The wall material may contain elastomer.

[0174] The wall material may contain at least one of aluminum powder and aluminum flakes.

[0175] The structural frame may be covered with wall material such that the wall material surrounds the structural frame.

[0176] The wall material is formed to be at least 0.8 mm or at least 1.2 mm thick, and at least one of the following: it is formed to be up to 4 mm or up to 5 mm thick.

[0177] The wall material may be formed to include thin sections that are thinner than the rest of the wall material (the parts other than the thin sections). This can be used particularly for light touch detection and shape recognition.

[0178] Thin areas may be placed near the front of the structure.

[0179] Thin areas can be shaped according to the shape of the fingernail (area).

[0180] The thin area may include less than one-quarter of the measurement surface.

[0181] The thickness of the thin area may be at least one of the following: a maximum of 0.8 mm or a maximum of 1.2 mm, or a maximum thickness of 30% or a maximum of 50%.

[0182] The building structure may be covered by wall materials that can be detachably attached to the structure.

[0183] The present invention further relates to a method for fabricating a sensor configuration, the method comprising the following steps. Steps include providing the base and A step to create a top with an internal space, The steps include: attaching a light source configuration with multiple light sources to a base, The steps include attaching an image sensor to the base, The light source is positioned to emit light towards the internal space, and the image sensor is positioned in the internal space, with the base covered by the top. That is the case.

[0184] This method provides reliable manufacturing of sensor components.

[0185] In particular, the top portion may be fabricated as disclosed herein. All implementations and modifications are applicable.

[0186] The light source may be mounted so as to surround the image sensor.

[0187] This method may further include the step of arranging a plurality of collimators across the light source, each defining at least one of the irradiation angles of the emitted light and the cone of the emitted light. For example, such collimators may be formed within a collimator ring, thereby effectively arranging a collimator ring.

[0188] The configuration of the fabricated sensor may be particularly embodied as described herein. All embodiments and modifications are applicable.

[0189] The present invention further relates to a sensor configuration as disclosed herein, or a sensor configuration manufactured as disclosed herein. The present invention further comprises an electronically controlled module configured to perform a method of force inference for the sensor configuration.

[0190] Therefore, the sensor configuration may include its own control module. For example, the control module may be an electronic entity located inside the base or near other parts of the base. Alternatively, the control module may be located away from the base and top, and may be, for example, a computer.

[0191] The control module may be configured to perform a force inference method that provides a force map of the measurement surface and a force map comprising multiple force vectors. The force inference method is described elsewhere in this specification. All embodiments and modifications are applicable.

[0192] In particular, the power map is mm 2 (1×10 -6 m 2 ) A force vector of at least 0.25 per mm 2 A force vector of at least 0.5 mm per unit area. 2 A force vector of at least 0.75 per unit, mm 2 A force vector of at least 1 per unit, mm 2 A force vector of at least 1.5 per unit, or mm 2 It must have at least 2 force vectors, mm 2 Maximum force vector of 0.25 per unit, mm 2 Maximum force vector of 0.5 per unit, mm 2 Maximum force vector of 0.75 per unit, mm 2 Maximum force vector of 1 per unit, mm 2 A force vector of up to 1.5 per unit, for example, mm 2 It has a force vector of up to 2 per unit and It may have at least one of the following values. Other values ​​may also be used.

[0193] In particular, each force vector comprises a normal force component, a first shear force component, and a second shear force component.

[0194] In particular, the first shear force component may correspond to the first shear force, and the second shear force component may correspond to the second shear force. The first shear force is perpendicular to the second shear force.

[0195] In particular, the sensor configuration disclosed herein may be finger-shaped. The sensor configuration is at least one of being a soft sensor and having the ability to sense force around its entire circumference. The sensing function may be enabled by machine learning. The sensor placement is accurate, highly sensitive, durable, and affordable.

[0196] There are two main techniques for obtaining three-dimensional (3D) information from a single camera that can be suitably used with the disclosed sensor configuration. Photometric stereo (or illumination stereo, PS) technology uses multiple images of the same scene with different, dispersed light sources to infer 3D shapes from shading information. Structured Light (SL) technology is a single-shot 3D surface reconstruction technique that utilizes unique light patterns and the fact that they are projected onto the 3D surface in different ways.

[0197] Generally, SL is typically used for global reconstruction, while PS excels at capturing local details. Insight combines PS and SL to perform deformation reconstruction of a complete 3D dome-shaped surface in a single-camera, single-image setup. Next to the camera or image detector are several light sources that generate cones of light. When a region of the measurement surface is in contact and deformed, the contact region moves, and the visible color changes due to two effects: color differences due to shading and movement between color regions with different light intensities due to the cones of light.

[0198] At least one of the top and base may be designed to prevent ambient light from entering the internal space. This prevents distortion of measurements due to such ambient light.

[0199] Further embodiments and advantages will be apparent to those skilled in the art from the following description of the enclosed drawings. These drawings illustrate the following: [Brief explanation of the drawing]

[0200] [Figure 1]Figure 1 shows an exploded view of the sensor configuration. [Figure 2] Figure 2 shows a cross-sectional view of the sensor configuration. [Figure 3] Figure 3 shows the top. [Figure 4] Figure 4 shows the building structure. [Figure 5] Figure 5 shows a different structural frame. [Figure 6] Figure 6 shows a detection surface with a light pattern. [Figure 7] Figure 7 shows a detection surface with a different light pattern. [Figure 8] Figure 8 shows the detection surface with intensity lines. [Figure 9] Figure 9 shows a detection surface with lines of different intensity. [Figure 10] Figure 10 shows multiple light sources equipped with collimators. [Figure 11] Figure 11 shows the frame and mold. [Figure 12] Figure 12 shows a further mold. [Figure 13] Figure 13 shows one of several different types of casts. [Figure 14] Figure 14 shows one of several different types of casts. [Figure 15] Figure 15 shows one of several different types of casts. [Figure 16] Figure 16 shows one of several different types of casts. [Figure 17] Figure 17 shows one of several different types of casts. [Figure 18] Figure 18 shows the outline of the cast. [Figure 19] Figure 19 shows the generation of a force map. [Figure 20] Figure 20 shows how to create a force map. [Figure 21] Figure 21 shows the configuration for force testing. [Figure 22] Figure 22 shows what can be used as an indenter. [Figure 23] Figure 23 shows a force map. [Figure 24] Figure 24 shows a map of simulated forces. [Modes for carrying out the invention]

[0201] Figure 1 shows the configuration 10 of a force-sensing sensor (sensor device 10). The sensor configuration 10 comprises a base 100 and a top 200.

[0202] The base 100 includes a support structure 110. This support structure 110 can be used in particular to mount the sensor configuration 10 to another entity, such as a robot. The base 100 further includes a printed circuit board 120 on which electronic components that control the sensor configuration 10 are mounted.

[0203] The base 100 further comprises an image sensor 130 located directly above the printed circuit board 120. The image sensor 130 is embodied as a color camera capable of detecting light and generating image data in response, where typically the image data generated by the image sensor 130 will be further processed by electronic components located on the printed circuit board 120.

[0204] The base 100 further includes a wide-angle lens 140 located directly above the image sensor 130. The wide-angle lens 140 is positioned such that all light striking the image sensor 130 passes through it. This allows the wide-angle lens 140 to define the field of view of the image sensor 130. The wide-angle lens 140 may be considered as part of the image sensor 130.

[0205] The base 100 further includes a mounting structure 150. The mounting structure 150 is directly attached to the support structure 110 using a number of screws 152 used to fix the top 200 to the base 100, so that the top 200 can be removed from the base 100. How the top 200 and the base 100 are connected is further described below.

[0206] The base 100 further comprises a light source configuration 160. The light source configuration 160 includes a support ring 162 attached to the mounting structure 150. The light source configuration 160 comprises a plurality of light sources 164. These light sources 164 are embodied as light-emitting diodes in this embodiment. The light sources 164 are arranged to emit light toward the internal space of the top portion 200, which will be further described below with reference to Figure 2.

[0207] The light source 164 has different colors. For example, it can be red, blue, or green. The following explains further how the light emitted from the light source can be used to detect the force applied to the sensor configuration 10.

[0208] A collimator ring 175 is located directly above the light source configuration 160. The collimator ring 175 comprises multiple collimators 170, which are embodied as holes projecting vertically through the collimator ring 175. Each collimator 170 is located directly above one light source 164. This ensures that only light that has passed through a collimator 170 can reach the internal space of the top 200. The collimator 170 may thus define multiple cones of light within the internal space. In particular, each cone of light has its own cone axis and outer cone angle.

[0209] The top portion 200 includes an elastically deformable wall 210. The elastically deformable wall 210 presents a measuring surface 220 to the outside of the sensor configuration 10. The measuring surface 220 is the surface to which a force is applied. Here, the force should be measured to provide a force map that depends, for example, on the force actually applied. The actual reasoning of the force will be explained further below.

[0210] The top 200 is comprised of a structural frame 240. The structural frame 240 is surrounded by walls 210. In Figure 1, the structural frame 240 and walls 210 are shown separately from each other.

[0211] The frame 240 comprises a base ring 242 and a grid 244 located above the base ring 242. The detailed structure of the frame 240 will be described further below. The base ring 242 comprises a plurality of projections 246 arranged radially outward and oriented perpendicularly to the base 100. Each of these projections 246 has a screw hole for a plurality of screws 152 extending vertically. Each screw 152 can be applied from below through these screw holes. Here, the screws 152 are fixed into their respective holes in the mounting structure 150 of the base 100. This allows the top 200 to be detachably mounted on the base 100.

[0212] Figure 2 shows a cross-sectional view of the sensor configuration 10 in its installed state. The frame 240 is surrounded by the wall 210, but is visible inside the wall 210. This is for illustrative purposes only. In reality, the frame 240, surrounded by the wall 210, would not be visible, or at least not clearly visible.

[0213] Figure 2 shows all components of the sensor configuration 10 in their final positions. As an exception, the collimator ring 175 is not shown in Figure 2 for the sake of clarity of the light source configuration 160.

[0214] As shown in Figure 2, the top 200 and the base 100 define the internal space 12. The internal space 12 is surrounded by a wall 210. The wall 210 defines the inner reflective surface 230 that surrounds the internal space 12.

[0215] As shown in Figure 2, the light source 164 is positioned on the base 100 so as to emit light towards the internal space 12.

[0216] Light emitted from the light source 164 first passes through the collimator 170, which is not shown in Figure 2. The collimator 170 defines the further propagation of the light (in particular, the individual propagation directions and outer cone angles of the cones of light). Then the light travels through the internal space 12 and reaches the reflective surface 230. Because the reflective surface 230 is diffuse, the incident light is reflected in all directions with a specific angle-dependent intensity, and the reflection is not like that of a mirror.

[0217] As shown in Figure 2, the reflective surface 230 is separated from the measurement surface 220 only by the wall 210. The wall 210 is made of an elastic material that relays deformation from the measurement surface 220 to the reflective surface 230. This means that any force applied to the measurement surface 220 deforms not only the wall 210 on the outer measurement surface 220, but also the inner reflective surface 230. Such deformation of the reflective surface 230 locally distorts the reflection of light rays. Thus, the measurement of light rays inside the sensor configuration 10 can be used to estimate the force applied to the measurement surface 220.

[0218] The image sensor 130 is also located inside the sensor configuration 10. The image sensor 130 is surrounded by the light source 164 so that light emanating from the light source 164 and reflected from the reflective surface 230 propagates to the image sensor 130. On the detection surface of the image sensor 130, the light generates a pattern of light indicating the applied force. Such a pattern of light has been shown to be an indicator of the location, amplitude, and direction of the force, as well as the shape and size of the applied indenter. Multiple forces can also be evaluated.

[0219] The grid 244 comprises a main section 248 and a fingernail section 249. The fingernail section 249 holds a thin section, which will be further described below. The main section 248 provides increased stability to the wall 210 so that it can withstand greater forces and does not deform substantially under the influence of external forces and gravity. However, despite the structure 240, the ability to slightly deform and relay the deformation from the measuring surface 220 to the reflective surface 230 remains effective.

[0220] Figure 3 shows the wall surface 210 separately. The wall 210 has a conical shape as shown in the figure. The tip of the wall 210 is rounded. Note that this shape is one of the representative shapes that has been proven to be suitable for multiple applications. However, it is not the only possible shape. Rather, all suitable shapes can be used. The measurement surface 220 is defined outside the wall 210. If a force is applied to the measurement surface 220, the wall 210 will deform.

[0221] Figure 4 shows the structural frame 240 separated from the others. For the components already described, please refer to the explanation in Figure 1. In particular, it can be seen that the grid 244 is formed from multiple wires with relatively large spaces between them. When these spaces are filled with the material of the wall 210, the wall 210 surrounds the structural frame 240. In this way, the structural frame 240 provides appropriate stability to the wall 210.

[0222] Figure 5 shows a wall 210 according to the second embodiment. In contrast to the embodiment shown in Figure 3, the wall 210 shown in Figure 5 has a thin area 250. The thin area 250 is thinner than the rest of the wall 210. The thin area 250 is surrounded by a border 255.

[0223] Looking at Figures 4 and 5 together, it becomes clear that the fingernail area 249 of the structural frame 240 supports the thin area 250 of the wall 210. The support is located almost along the edge 255. This measure greatly increases the stability of the thin area 250. However, it should be noted that the same structural frame design can also be used for the top 200 where there is no thin area 250.

[0224] The thin area 250 results in a localized increase in sensitivity, particularly with respect to force detection. For example, a force applied to the thin area 250 leads to a greater deformation of the wall 210, and therefore to a greater deformation of the reflective surface 230. Consequently, a force applied to the thin area 250 also leads to a larger change in the light pattern detected by the image sensor 130.

[0225] Figure 6 shows the pattern of light 132 on the detection surface 131 of the image sensor 130. The detection surface 131 typically comprises multiple pixels, which are not shown in Figure 6.

[0226] As shown in FIG. 6, an exemplary light pattern 132 includes eight light spots indicated by reference numerals 133, 134, and 135. The light spots 133, 134, 135 are at least approximately elliptical. The three light spots 133 originate from a light source having a first color, for example, blue. The three light spots 134 originate from a light source having a second color, for example, red. The two light spots 135 originate from a light source having a third color, for example, green. Note that the number of the eight light spots 133, 134, 135 having three different colors is for illustrative purposes only here, and any other number of light spots and colors can be used. In particular, the light emitted from the light source 164 propagates conically through the internal space 12, is reflected by the reflection surface 230, and further propagates to the detection surface 131, and each light spot 133, 134, 135 may correspond to one light source 164.

[0227] FIG. 6 shows a typical light pattern 132 in an undeformed state. That is, such a light pattern 132 may be viewed on the detection surface 131 if no force is applied to the measurement surface 220. For example, any means of force inference that may be a neural network can be trained to detect that no force is applied when detecting the light pattern 132 by the image sensor 130 in FIG. 6.

[0228] FIG. 7 shows a further light pattern 132 on the detection surface 131. The light pattern 132 in FIG. 7 corresponds to a state where a force is applied to the measurement surface 220. As can be seen from the figure, the two light spots 133, 135 have a deformed portion 136 with a local color change due to different reflection characteristics inside the reflection surface 230 with respect to the applied force. The situation in FIG. 7 enables an inference that a force has been applied. In particular, the change in the light pattern 132 is characteristic not only of the strength of the force but also of its position, direction, and the shape and size of the indenter applying the force. This is also true for two or more forces that can be applied simultaneously.

[0229] FIG. 8 schematically shows a line 138 of the intensity of the intensity pattern 137 on the detection surface 131. Each of the intensity lines 138 corresponds to a line of a certain light intensity on the detection surface 131. FIG. 8 shows a non-deformed state corresponding to the state shown in FIG. 6. FIG. 9 shows a deformed state having intensity lines 138 with different intensities due to locally different reflection characteristics of the reflection surface 230. This is also characteristic of the applied force and can thus be used for force inference.

[0230] FIG. 10 purely schematically shows an image sensor 130 having a detection surface 131, and the image sensor 130 is positioned between a light source configuration 160 with a support ring 162 and a light source 164 with a collimator 170 and a collimator ring 175.

[0231] As shown in the figure, the light emitted from the light source 164 passes through the collimator 170 and further propagates in the internal space 12 in each cone 166. These cones 166 are defined by a central propagation direction 167 which can also be regarded as a cone axis, and an outer cone angle 168 which defines the maximum range of the horizontal light with respect to the central propagation direction 167. As shown in the figure, the collimator 170 is arranged slightly outside the point where the light exits from the light source 164 such that the central propagation direction 167 is not perpendicular but slightly outward. Thereby, prevention of oversaturation at the tip of the top 200 is ensured. With this implementation, an optical structure defined inside the internal space 12 can be obtained. The reflected light from the reflection surface 230 not shown in FIG. 8 may propagate to the detection surface 131 and the light may be detected for force inference.

[0232] FIG. 11 shows a mold 600 for fabricating the top for the sensor configuration ⑩. The mold 600 includes a mold body 620 in which an opening 610 is formed. FIG. 12 shows a further mold 605 which also has a mold body 620 with an opening 610 formed therein. When the mold 600 and the further mold 605 are joined together, only one opening 610 remains. The opening 610 can be used for fabricating the top 200 by overmolding the housing 240.

[0233] Figure 11 shows one state of the manufacturing process for the top section 200. The structural frame 240 is inserted into the opening 610. A further mold 605, shown in Figure 12, will be used to form a single opening 610. The structural frame 240 is overmolded with wall material to form the wall 210 surrounding the structural frame 240.

[0234] The outside of the formed wall 210 is defined by the opening 610, which will later become the measurement surface 220. In other words, the measurement surface 220 conforms to the shape of the opening 610.

[0235] Cast 700 is used to define the internal shape which will later become the reflective surface 230. Possible embodiments of cast 700 are shown in Figures 13 to 17.

[0236] Figure 13 shows a cast 700 according to the first embodiment. The cast 700 includes a support ring 720. From the support ring 720, the main portion 710 has a rounded cross-section that becomes smaller towards the tip. In the embodiment of Figure 13, the main portion 710 has a flat outer surface, so when the cast 700 is used to define the inner reflective surface 230 of the wall 210 inside the structure 240 shown in Figure 11, a flat reflective surface 230 is formed.

[0237] Figure 14 shows a cast 700 according to a second embodiment. In contrast to the embodiment shown in Figure 13, the cast 700 of Figure 14 has a plurality of grooves 716 that can be used to provide a specific complementary structure on the reflective surface 230. The grooves 716 form complementary protrusions within the reflective surface 230, which can more significantly increase the variation in the light pattern 132. As shown in Figure 14, the grooves 716 are located near the tip of the main portion 710. For example, the grooves 716 may be applied to a thin area 250 of the wall 210, as shown in Figure 5.

[0238] Figure 15 shows a cast 700 according to a third embodiment. In the embodiment of Figure 15, the cast 700 has an outer grid structure 712. This outer grid structure 712 leads to a complementary structure on the reflective surface 230. This improves the force detection function in many situations.

[0239] Figure 16 shows a cast 700 according to the fourth embodiment, which is materialized similarly to the embodiment in Figure 15, but this cast 700 has a finer lattice structure 712.

[0240] Figure 17 shows a cast 700 according to the sixth embodiment. In addition to the outer lattice structure 712 or Figure 16, the embodiment shown in Figure 17 has a flat portion 714 that does not have a lattice structure. The flat portion 714 is also located near the tip. It leads to a locally flat reflective surface 230. Such a locally flat reflective surface 230 can be applied, for example, in a thin area 250.

[0241] Figure 18 shows a typical cross-sectional shape (outer shape, profile) of the cast 700, for example, as shown in Figure 16. The cross-sectional shape comprises an outer lattice structure 712 with (multiple) protrusions 713. Between the (multiple) protrusions 713 are flat areas that connect to flat areas on the reflective surface 230. In the illustrated embodiment, the protrusions 713 have a semicircular outer shape. For example, the distance between the protrusions 713 may be at least 0.1 mm, at least 0.5 mm, at least 1 mm, or at least 2 mm. The distance between the protrusions 713 may be up to 0.5 mm, up to 1 mm, up to 2 mm, or up to 5 mm. Thus, it can range from sub-millimeters to several millimeters. The same values ​​or ranges may apply to the radius of the protrusions 713. However, other values ​​may also be used.

[0242] Figure 19 shows a schematic diagram of the force inference.

[0243] As already mentioned above, the image sensor 130 includes a detection surface 131. The detection surface 131 has multiple pixels P. Each pixel is denoted as P1, P2, ..., Px. Each pixel P can individually colorimetrically detect the light incident on the pixel. This allows the image sensor 130 to detect the light pattern 132.

[0244] The output data from the image sensor 130 is supplied to a feedforward neural network (FFNN). This is an artificial neural network that maps the image data from the image sensor 130 onto a force map (FM). The force map FM comprises multiple force vectors F1, F2, ..., Fx. The force vectors F of the force map FM will be explained further below.

[0245] The feedforward neural network (FFNN) may be trained by training method T. Appropriate training methods are further described below with reference to Figure 20.

[0246] In principle, a feedforward neural network (FFNN) can detect force from image data received from the image sensor 130. This can be enhanced by machine learning techniques. A properly trained feedforward neural network (FFNN) has been proven to be able to infer the position, amplitude, and direction of a force, as well as the shape and size of the indenter. Such inference is possible even when multiple forces are applied simultaneously.

[0247] To enhance force inference, at least one of the force inference and training steps should be performed by inputting additional image data in each case, including observation of the undeformed illuminated reflective surface, a gradient image in grayscale, and an image of the structure.

[0248] Figure 20 shows method T for training a feedforward neural network (FFNN).

[0249] In the first step T_1, multiple force tests are performed using the sensor configuration 10, as further explained with reference to Figure 21. In each force test, the applied force is measured, and the position of the force on the measurement surface 220 is also measured. Furthermore, image data corresponding to the light pattern 132 is read from the image sensor 130.

[0250] In the second step T_2, a plurality of simulation tests are performed such that each simulation test corresponds to one force test. Each simulation test is performed using a model of the configuration 10 of the sensor that simulates the behavior of the components, particularly the wall 210, when a force is applied. In this way, a simulated force map FM’ is calculated in each simulation test.

[0251] In the third step T_3, the feed-forward neural network FFNN is trained with the image data of the force test and the simulated force map FM’ calculated in the corresponding simulation test. Specifically, each sub-step of the training may include training the feed-forward neural network with the light pattern 132 and the corresponding simulated force map FM’. Thereby, the feed-forward neural network FFNN learns how to map the light pattern 132 to the force map FM.

[0252] Note that the simulated force map FM’ only exists in the simulation test based on the model of the sensor configuration 10. The force map FM exists on the actual measurement surface 220.

[0253] FIG. 21 shows an arrangement 500 for a force test that performs a force test. The arrangement 500 for a force test includes a base 510. On the base 510, a first arm 520 is arranged, which is connected by a joint 530. At the joint 530, a second arm 540 is positioned. The first arm 520 may be rotated on the base 510 and the second arm 540 may be swung around the joint 530 by electric driving means not shown.

[0254] The above-described sensor configuration 10 is attached to the second arm 540. It may be rotated about the axis of the second arm 540. The sensor configuration 10 can thus be positioned by the arrangement 500 for a force test.

[0255] There is also a top 550, to which a force sensor 560 is attached. The force sensor 560 is connected by an indenter 800. The indenter 800 remains in a substantially unchanging position. In the force testing configuration 500, the sensor configuration 10, in particular its measuring surface 220, can be brought into contact with the indenter 800, so that force can be applied. This force can be measured by the force sensor 560.

[0256] In a preferred implementation, arms 520 and 540 are used to select the intended position on the measuring surface 220 where the indenter 800 should contact the measuring surface 220. The indenter 800 is then moved by moving its top 550 in three dimensions, and thus a force can be applied to the measuring surface 220, which may have both normal force and shear force components. The sensor configuration 10 may remain in place during the application of its force. However, other implementations of force testing are also possible, particularly with respect to the movement of parts. For example, the top 550 may be moved simultaneously with arms 520 and 540. Alternatively, only arms 520 and 540 may be used for force application.

[0257] The position at which the indenter 800 contacts the measurement surface 220 can be calculated using mechanical variables or a kinematic model. However, observation with a camera may also be performed.

[0258] Figure 22 schematically shows four different shapes of indenters 800 that can be used as physical indenters 800 for use in force testing arrangement 500, or as simulated indenters 800' in simulated tests, as will be further described below with respect to Figure 24.

[0259] Figure 22a shows an indenter 800 with a flat contact portion for its contact with the measurement surface 220. Figure 22b shows an indenter 800 with a pointed contact portion. Figure 22c shows an indenter 800 with a hemispherical contact portion. Figure 22d shows an indenter 800 that is identical in shape to the indenter 800 shown in Figure 1, but with a smaller contact portion. Using such different indenters 800 allows for the optimization of the training of the feedforward neural network FFNN with respect to these different shapes, meaning that the ability of a feedforward neural network FFNN trained with such different indenters 800 increases with respect to the reconstructive forces applied by indenters 800 with different indenter shapes. To put it another way, the force map FM reconstructed after applying a flat-shaped indenter 800 will be different from the force map FM reconstructed after applying a hemispherical-shaped indenter 800.

[0260] Figure 23 shows a sensor configuration 10 with a schematic diagram of the force map FM. The force map FM comprises multiple force vectors F located around the entire perimeter of the measurement surface 220. Two force vectors F are shown in Figure 23, but a typical implementation can use many more force vectors F. For example, mm 2 One force vector F can be used per example implementation.

[0261] The vector F of each force has a normal force component F N , First shear force component F S1 and the second shear force component F S2 It has a vertical force component F. n This gives the value of the applied force, i.e., the normal force component perpendicular to the local orientation of the measurement surface 220. Shear force component F S1 F S2 This gives the value of the shear force applied to the measurement surface 220 at each point. The shear force is typically parallel to the local orientation of the measurement surface 220, typically perpendicular to each other, and perpendicular to the normal force. This may particularly relate to the undeformed orientation of the measurement surface 220, which can define the direction of the force vector F, especially its normal component.

[0262] Therefore, each force vector F gives the strength and direction of the force applied to a specific point on the measurement surface 220. Such a force may arise, for example, from an indenter 800, one such indenter 800 of which is shown as an example in Figure 23. When a force is applied, the measurement surface 220 deforms slightly.

[0263] Note that other definitions of the force vector F can also be used; for example, only the normal force component may be considered, or the shear force may have an alternative definition.

[0264] Figure 24 shows a corresponding case using a simulated force map FM'. Reference numerals are denoted by an apostrophe ('). In the case of a simulated force map FM', the simulated force vector F' of such a simulated force map FM' on the simulated measurement surface 220' is represented by the simulated components, e.g., the vertical force component. F’N , the first shear force component F' S1 and the second shear force component F' S2 Such a simulated power map FM' is calculated, in particular, in a simulated test performed on the model as described above.

[0265] There is also a simulated indenter 800' shown in Figure 24. The simulated indenter 800' is applied to the simulated test in the corresponding force test with the same size, direction and position as the actual indenter 800. Using the model, a map of simulated forces FM' is calculated and used to train a feedforward neural network FFNN.

[0266] The steps (processes) mentioned in the method of the present invention can be performed in a given order. However, they can be performed in a different order, as long as it is technically reasonable. In embodiments, the method of the present invention can be carried out, for example, using a specific combination of steps, so that no further steps are performed. However, other steps, including steps not mentioned, can also be performed.

[0267] While the various features (of this invention) may be used or implemented independently of each other, it should be noted that, for clarity, for example, the features are described in combination within the claims and specification. Those skilled in the art will realize that such features can be combined with other features, or that there may be combinations of features that are independent of each other.

[0268] References within dependent claims may indicate preferred combinations of features, but do not preclude other combinations of features. This application provides, for example, the following perspectives. (Perspective 1) A method for inferring force from a force measuring sensor configuration (10), The configuration of the sensor (10) is, An elastically deformable wall (210), the wall (210) comprising an outer measuring surface (220) and an inner reflective surface (230), wherein the reflective surface (230) partially demarcates the internal space (12), the wall (210) and, A light source configuration (160) comprising multiple light sources (164) configured to emit light toward the interior space, An image sensor (130) is installed in the internal space (12) and The method for inferring the force is provided, The steps include reading image data from the image sensor (130), A step of calculating a force map (FM) on the measurement surface (220) based on the image data using a feedforward neural network (FFNN), wherein the force map (FM) comprises a plurality of force vectors (F); A method for inferring the force, comprising the following: (Perspective 2) The aforementioned feedforward neural network (FFNN) is trained in the following steps, which are performed before force inference: - A step (T_1) in which multiple force tests are performed on the sensor configuration (10), wherein each force test is performed The configuration of the sensor (10) involves applying force to a position on the measurement surface (220) by one indenter (800), and simultaneously measuring the force applied by the indenter (800). Step (T_1) of performing multiple force tests, which includes simultaneously reading image data from the image sensor (130), - Step (T_2) of performing a corresponding simulated test with one model (10') of the sensor configuration (10) for each force test, Each simulated test comprises applying simulated forces to the simulated measurement surface (220') of the model (10'), thereby calculating a map (FM') of the simulated forces on the simulated measurement surface (220'), The aforementioned simulated force map (FM') comprises multiple simulated force vectors (F'), Step (T_2) of performing a simulated test, wherein the simulated force corresponds to the measured force and is applied to a position on the simulated measurement surface (220') corresponding to a position on the measurement surface (220), - Step (T_3) of training a feedforward neural network (FFNN) using the aforementioned image data and the corresponding computed simulated force map (FM') and The method according to viewpoint 1, comprising: (Perspective 3) The method according to viewpoint 2, wherein the force test for training the aforementioned feedforward neural network (FFNN) is performed using a plurality of indenters (800) each having an indenter shape. (Perspective 4) The method according to viewpoint 3, wherein the indenter shape is selected from the group comprising at least a tip, a circular, triangular, square, hemisphere, cube, and cylinder. (Perspective 5) The method according to viewpoint 3 or 4, wherein the simulated test is performed with a simulated force applied by a simulated indenter (800) having a simulated indenter shape corresponding to a plurality of actual indenter shapes used in the corresponding force test. (Perspective 6) The method according to any one of viewpoints 2 to 5, wherein the feedforward neural network (FFNN) is trained using a plurality of different indenter shapes. (Perspective 7) The method according to any one of viewpoints 2 to 6, wherein the feedforward neural network (FFNN) is trained using multiple indenters (800) of multiple different sizes. (Perspective 8) The method according to any one of views 2 to 7, wherein the feedforward neural network is trained with the indenter (800) applied with each shear force for at least a portion of the force tests used to train the feedforward neural network. (Perspective 9) The method according to any one of views 2 to 8, wherein each of the measured forces comprises a normal force component, a first shear force component, and a second shear force component. (Perspective 10) Of the measured forces, the first shear force component corresponds to the first shear force, the second shear force component corresponds to the second shear force, and The method according to viewpoint 9, wherein the first shear force is perpendicular to the second shear force. (Perspective 11) The method according to any one of viewpoints 2 to 10, wherein each of the measured forces comprises three components in the reference coordinate system. (Perspective 12) The aforementioned feedforward neural network (FFNN) is trained using multiple forces having different shear force components, according to any one of viewpoints 2 to 11. (Perspective 13) The aforementioned feedforward neural network (FFNN) is trained using multiple forces with different normal force components, according to one of viewpoints 2 to 12. (Perspective 14) The force is measured using a force sensor (560) located inside or next to the indenter (800). The method described in any one of viewpoints 2 through 13. (Perspective 15) Each of the above simulated force vectors (F') has a normal force component (F' N ), first shear force component (F' S1 ) and the second shear force component (F' S2 A method according to any one of viewpoints 2 to 14, comprising ) (Perspective 16) The method according to any one of viewpoints 2 to 14, wherein each of the aforementioned simulated force vectors (F') has three components in the reference coordinate system. (Perspective 17) The method according to any one of views 1 to 16, wherein the image data on which the calculated force map (FM) is based comprises, in addition to the image data read from the image sensor, several invariant images, or three invariant images. (Perspective 18) The method according to viewpoint 17, wherein the invariant image is at least one of a grayscale gradient image, an image of the structure, and a reference light pattern. (Perspective 19) The variable image as part of the image data is captured immediately before calculating the force map (FM) as described in view 17 or 18. (Perspective 20) The aforementioned force map (FM) is mm 2 (1×10 -6 m 2 ) A force vector (F) of at least 0.25 per mm 2 A force vector (F) of at least 0.5 per unit, mm2 A force vector (F) of at least 0.75 per unit, mm 2 A force vector (F) of at least 1 per unit, mm 2 A force vector (F) of at least 1.5 per unit, or mm 2 It must have at least 2 force vectors, The aforementioned force map (FM) is mm 2 Maximum force vector of 0.25 per unit, mm 2 Maximum force vector of 0.5 per unit, mm 2 Maximum force vector of 0.75 per unit, mm 2 Maximum force vector of 1 per unit, mm 2 A maximum force vector of 1.5 per unit, or mm 2 It has a force vector of up to 2 at the point of contact. A method according to any one of viewpoints 1 to 19, which is at least one of the above. (Perspective 21) Each of the aforementioned force vectors (F) has a normal force component (F N ) and the first shear force component (F S1 ) and the second shear force component (F S2 A method according to any one of viewpoints 1 to 20, comprising ) and . (Perspective 22) The aforementioned first shear force component (F S1 ) corresponds to the first shear force, and the second shear force component (F S2 ) responds to the second shear force, and, The method according to viewpoint 21, wherein the first shear force is perpendicular to the second shear force. (Perspective 23) The method according to any one of viewpoints 1 to 20, wherein each of the force vectors (F) has three components in the reference coordinate system. (Perspective 24) As part of the aforementioned image data, an additional image of the reflective surface (230) of the sensor configuration (10) is used without external impact, The method according to any one of viewpoints 1 to 23, wherein the feedforward neural network (FFNN) is trained, or the calculation of the force map (FM) is performed. (Perspective 25) The method according to any one of viewpoints 1 to 24, wherein the feedforward neural network (FFNN) is trained or the force map (FM) is calculated using an image of the structure (240) of the wall (210) of the sensor configuration (10) as part of the image data. (Perspective 26) The method according to any one of viewpoints 1 to 25, wherein the feedforward neural network (FFNN) is trained or the force map (FM) is calculated using a grayscale gradient image for location coding as part of the image data. (Perspective 27) A method (T) for training a feedforward neural network (FFNN), The feedforward neural network calculates a force map (FM) on the measurement surface (220) of the sensor configuration (10) based on image data from the image sensor (130), and the force map (FM) comprises a plurality of force vectors (F). The aforementioned feedforward neural network (FFNN) proceeds to the next step, - A step (T_1) in which multiple force tests are performed on the sensor configuration (10), Each force test includes applying a force with a single indenter (800) at a position on the measurement surface (220) of the sensor configuration (10), simultaneously measuring the force applied by the indenter (800), and simultaneously reading image data from the image sensor (130). Step (T_1) involves performing tests on the aforementioned multiple forces, - For each of the aforementioned force tests, step (T_2) is to perform a corresponding simulated test using one model (10') of the sensor configuration (10), Each simulated test comprises applying simulated forces to the simulated measurement surface (220') of the model (10'), thereby calculating a map (FM') of the simulated forces on the simulated measurement surface (220'), The aforementioned simulated force map (FM') comprises multiple simulated force vectors (F'), The simulated force corresponds to the measured force and is applied to a position on the simulated measurement surface (220') that corresponds to a position on the measurement surface (220). Step (T_2) of conducting the aforementioned mock exam, - Step (T_3) of training the feedforward neural network (FFNN) using the image data and the corresponding simulated force map (FM') A method for training a feedforward neural network (FFNN), which is trained using [this method]. (Perspective 28) The method according to viewpoint 27, wherein the force test for training the feedforward neural network (FFNN) is performed using a plurality of indenters (800), each indenter having its own indenter shape. (Perspective 29) The method according to viewpoint 28, wherein the indenter shape is selected from the group comprising at least a tip, a circular, triangular, square, hemisphere, cube, and cylinder. (Perspective 30) The method according to view 28 or 29, wherein the simulated test is performed with a simulated force applied by a plurality of simulated indenters (800'), each having a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test. (Perspective 31) The method described above, wherein the feedforward neural network (FFNN) is trained using multiple different indenter shapes, according to any one of viewpoints 27 to 30. (Perspective 32) The method according to any one of views 27 to 31, wherein the feedforward neural network (FFNN) is trained using a plurality of indenters (800) of different sizes. (Perspective 33) The method according to any one of views 27 to 32, wherein the feedforward neural network (FFNN) is trained with a plurality of indenters (800) each having a shear force for at least a portion of the force tests used to train the feedforward neural network (FFNN). (Perspective 34) The method according to any one of views 27 to 33, wherein each measured force comprises a normal force component, a first shear force component, and a second shear force component. (Perspective 35) Among the multiple measured forces, the first shear force component corresponds to the first shear force, the second shear force component corresponds to the second shear force, and The method according to viewpoint 34, wherein the first shear force is perpendicular to the second shear force. (Perspective 36) The method according to any one of viewpoints 27 to 33, wherein each measured force comprises three components in a reference coordinate system. (Perspective 37) The method according to any one of viewpoints 27 to 36, wherein the feedforward neural network (FFNN) is trained using multiple forces having different shear force components. (Perspective 38) The method according to any one of viewpoints 27 to 37, wherein the feedforward neural network (FFNN) is trained using multiple forces having different normal force components. (Perspective 39) The method according to any one of views 27 to 38, wherein the force is measured using a force sensor (560) located inside or next to the indenter (800). (Perspective 40) Each of the vectors (F') of the simulated force has a normal force component (F' N ) and the first shear force component (F' S1 ) and the second shear force component (F' S2 A method according to any one of viewpoints 27 to 39, having the following characteristics. (Perspective 41) Among the multiple vectors (F') of the simulated force, the first shear force component (F'S1 ) corresponds to the first shear force, and the second shear force component (F' S2 ) responds to the second shear force, and The method according to viewpoint 40, wherein the first shear force is perpendicular to the second shear force. (Perspective 42) The method according to any one of viewpoints 27 to 39, wherein each of the simulated force vectors (F') has three components in the reference coordinate system. (Perspective 43) The method according to any one of views 27 to 42, wherein the feedforward neural network (FFNN) is used in the method according to view 1 or the method according to one view subordinate to view 1. (Perspective 44) The aforementioned force map (FM) is mm 2 A force vector (F) of at least 0.25 per unit, mm 2 A force vector (F) of at least 0.5 per unit, mm 2 A force vector (F) of at least 0.75 per unit, mm 2 A force vector (F) of at least 1 per unit, mm 2 A force vector (F) of at least 1.5 per unit, or mm 2 It has at least 2 force vectors (F) per unit, The aforementioned force map (FM) is mm 2 Maximum force vector (F) of 0.25 per unit, mm 2 Maximum force vector (F) of 0.5 per unit, mm 2 Maximum force vector (F) of 0.75 per unit, mm 2 The maximum force vector (F) per unit area is 1, mm 2 A maximum force vector (F) of 1.5 per unit, or mm 2 It has a force vector (F) of a maximum of 2. A method according to any one of viewpoints 27 to 43, which is at least one of the above. (Perspective 45) Each of the aforementioned force vectors (F) has a normal force component (F N ) and the first shear force component (F S1 ) and the second shear force component (F S2 A method according to any one of viewpoints 27 to 44, comprising the above. (Perspective 46) The aforementioned first shear force component (F S1 ) corresponds to the first shear force, and the second shear force component (F S2 ) responds to the second shear force, and The method according to viewpoint 45, wherein the first shear force is perpendicular to the second shear force. (Perspective 47) The method according to any one of viewpoints 27 to 44, wherein each of the force vectors (F) has three components in the reference coordinate system. (Perspective 48) The method according to any one of viewpoints 27 to 47, wherein the feedforward neural network (FFNN) is trained using additional images of the reflective surface (230) of the sensor configuration (10) without external impact as part of the image data. (Perspective 49) The method according to any one of viewpoints 27 to 48, wherein the feedforward neural network (FFNN) is trained using images of the structure of the wall (210) of the sensor configuration (10) as part of the image data. (Perspective 50) The method according to any one of viewpoints 27 to 49, wherein the feedforward neural network (FFNN) is trained on a grayscale gradient image for encoding position as part of the image data. (Perspective 51) The method according to any one of viewpoints 27 to 50, wherein the feedforward neural network (FFNN) is trained with one or more of the following: a grayscale gradient image, an image of the structure, and a reference light pattern. (Perspective 52) The sensor configuration (10) is a sensor configuration (10) for detecting force, The configuration of the sensor (10) is, An elastically deformable wall (210), the wall (210) comprising an outer measuring surface (220) and an inner reflective surface (230), wherein the reflective surface (230) partially demarcates the internal space (12), the wall (210) and A configuration (160) of a light source having multiple light sources (164) and configured to emit light toward the internal space (12), An image sensor (130) is installed in the internal space (12) and A method according to any one of viewpoints 27 to 51, comprising: (Perspective 53) The sensor configuration (10) is a sensor configuration (10) for detecting force, The configuration of the sensor (10) is, - Base (100), - A top (200) comprising an elastically deformable wall (210), wherein the top (200) is attached to the base (100) such that the top (200) and the base (100) define an internal space (12), and the wall (210) comprises an outer measuring surface (220) and an inner reflective surface (230), where the reflective surface (230) partially demarcates the internal space (12), the top (200) and, - A configuration (160) of a light source having a plurality of light sources (164) attached to the base (100) and configured to emit light toward the internal space (12), - An image sensor (130) attached to the base (100) within the internal space (12) and A method according to any one of viewpoints 1 to 52, comprising: (Perspective 54) The method according to any one of viewpoints 1 to 53, wherein the sensor configuration (10) is at least one of the robot's tip and the robot's operating element. (Perspective 55) A force inference module for force inference of a force-sensing sensor configuration (10), the force inference module configured to perform the method described in any one of views 1 to 54. (Perspective 56) A configuration (10) of a force-sensing sensor, wherein the configuration (10) of the sensor is - Base (100), - A top (200) comprising an elastically deformable wall (210), wherein the top (200) is attached to the base (100) such that the top (200) and the base (100) define an internal space (12), and the wall (210) comprises an outer measuring surface (220) and an inner reflective surface (230), where the reflective surface (230) partially demarcates the internal space (12), the top (200) and, - A configuration (160) of a light source having a plurality of light sources (164) attached to the base (100) and configured to emit light toward the internal space (12), - An image sensor (130) attached to the base (100) within the internal space (12), The force inference module described in perspective 55 and A sensor configuration (10) comprising the following: [Explanation of symbols]

[0269] 10 Sensor Configuration 12 Interior space 100 base 110 Support structure 120 Printed circuit boards 130 Image Sensor 131 Detected surface 132 Light Patterns 133 Spots of Light 134 Spots of Light 135 Spots of Light 136 Deformed part 137 Light intensity patterns 138 Intensity lines 139 Deformation range 140 wide-angle lens 150 Mounting structure 152 screws 160 Light source configurations 162 Support ring 164 light source 166 Cone 167 Central propagation direction 168 Outer cone angle 170 Collimator 175 Collimator ring 200 Top 210 Wall 220 Measurement surface 230 Reflective Surface 240 skeleton 242 Bottom ring 244 Lattice 246 Protrusions (multiple) 248 Main part 249 Fingernail area 250 Thin area 255 yen 500 force test setup 510 base 520 First Arm 530 joints 540 Second Arm 550 Top 560 force sensor 600 molds 605 Further molding 610 Aperture 620 Mold Body 700 (Casting) 710 Main part 712 Lattice structure 713 Protrusions (multiple) 714 Flat part 716 grooves (multiple) 720 Support ring 800 indenter P pixels FFNN (Feedback Neural Network) FM Power Map T Training method F is the force vector. F N Normal force component F S Shear force component Apostrophe "'" - Elements of a mock exam

Claims

1. A method for inferring force from a force measuring sensor configuration (10), The configuration of the sensor (10) is, An elastically deformable wall (210), the wall (210) comprising an outer measuring surface (220) and an inner reflective surface (230), wherein the reflective surface (230) partially partitions the internal space (12), the wall (210) and, A light source configuration (160) comprising multiple light sources (164) configured to emit light toward the interior space, An image sensor (130) is installed in the internal space (12) and The method for inferring the force is provided, The steps include reading image data from the image sensor (130), A step of calculating a force map (FM) on the measurement surface (220) based on the image data using a feedforward neural network (FFNN), wherein the force map (FM) comprises a plurality of force vectors (F); Equipped with, The aforementioned feedforward neural network (FFNN) is trained by the following steps, which are performed before force inference: - A step (T_1) in which multiple force tests are performed on the sensor configuration (10), wherein each force test is performed The method involves applying force to a position on the measurement surface (220) of the sensor configuration (10) using one indenter (800), and simultaneously measuring the force applied by the indenter (800). Step (T_1) of performing multiple force tests, which includes simultaneously reading image data from the image sensor (130), - Step (T_2) of performing a corresponding simulated test with one model (10') of the sensor configuration (10) for each force test, Each simulated test comprises applying simulated forces to the simulated measurement surface (220') of the model (10'), thereby calculating a map (FM') of the simulated forces on the simulated measurement surface (220'), The aforementioned simulated force map (FM') comprises multiple simulated force vectors (F'), Step (T_2) of performing a simulated test, wherein the simulated force corresponds to the measured force and is applied to a position on the simulated measurement surface (220') corresponding to a position on the measurement surface (220), - Step (T_3) of training a feedforward neural network (FFNN) using the aforementioned image data and the corresponding computed simulated force map (FM') and A method for inferring force from a force measuring sensor configuration (10) that includes the following.

2. A method (T) for training a feedforward neural network (FFNN), The feedforward neural network calculates a force map (FM) on the measurement surface (220) of the sensor configuration (10) based on image data from the image sensor (130), and the force map (FM) comprises a plurality of force vectors (F). The aforementioned feedforward neural network (FFNN) proceeds to the next step, - A step (T_1) in which multiple force tests are performed on the sensor configuration (10), Each force test includes applying a force using a single indenter (800) at a position on the measurement surface (220) of the sensor configuration (10), simultaneously measuring the force applied by the indenter (800), and simultaneously reading image data from the image sensor (130). Step (T_1) of performing the aforementioned multiple force tests, - Step (T_2) of performing a corresponding simulated test with one model (10') of the sensor configuration (10) for each of the aforementioned force tests, Each simulated test comprises applying simulated forces to the simulated measurement surface (220') of the model (10'), thereby calculating a map (FM') of the simulated forces on the simulated measurement surface (220'), The aforementioned simulated force map (FM') comprises multiple simulated force vectors (F'), The simulated force corresponds to the measured force and is applied to a position on the simulated measurement surface (220') that corresponds to a position on the measurement surface (220). Step (T_2) of conducting the aforementioned mock exam, - Step (T_3) of training the feedforward neural network (FFNN) using the image data and the corresponding simulated force map (FM') and A method for training a feedforward neural network (FFNN), which is trained using [this method].

3. The method according to claim 1 or 2, wherein the force test for training the feedforward neural network (FFNN) is performed using a plurality of indenters (800), each indenter having its own indenter shape.

4. The method according to any one of claims 1 to 3, wherein the indenter shape is selected from the group comprising at least a tip, a circular shape, a triangular cross-section, a square cross-section, a hemisphere, a cube, and a cylinder.

5. The method according to claim 3 or 4, wherein the simulated test is performed with a simulated force applied by a plurality of simulated indenters (800'), each having a simulated indenter shape corresponding to the actual indenter shape used in the corresponding force test.

6. The aforementioned feedforward neural network (FFNN) is trained using multiple different indenter shapes, The feedforward neural network (FFNN) is trained using a plurality of indenters (800) of different sizes. The method according to any one of claims 1 to 5, wherein at least one of the above is performed.

7. The method according to any one of claims 1 to 6, wherein the feedforward neural network (FFNN) is trained with a plurality of indenters (800) each having a shear force for at least a portion of the force tests used to train the feedforward neural network (FFNN).

8. The method according to any one of claims 1 to 7, wherein each measured force comprises a normal force component, a first shear force component, and a second shear force component.

9. Among the multiple measured forces, the first shear force component corresponds to the first shear force, the second shear force component corresponds to the second shear force, and The method according to claim 8, wherein the first shear force is perpendicular to the second shear force.

10. The method according to any one of claims 1 to 9, wherein each measured force comprises three components in a reference coordinate system.

11. The aforementioned feedforward neural network (FFNN) is trained using multiple forces with different shear force components, The aforementioned feedforward neural network (FFNN) is trained using multiple forces with different normal force components. The method according to any one of claims 1 to 10, wherein at least one of the following is performed.

12. The method according to any one of claims 1 to 11, wherein the force is measured using a force sensor (560) located inside or next to the indenter (800).

13. Each of the aforementioned force vectors (F) has a normal force component (F Ｎ ) and the first shear force component (F Ｓ１ ) and the second shear force component (F Ｓ２ The method according to any one of claims 1 to 12, comprising:

14. The aforementioned feedforward neural network (FFNN) is trained or will be trained using additional images of the reflective surface (230) of the sensor configuration (10) without external impact as part of the image data, The feedforward neural network (FFNN) is either trained or will be trained using an image of the body of the sensor configuration (10) as part of the image data. The method according to any one of claims 1 to 13, wherein at least one of the above.

15. The aforementioned feedforward neural network (FFNN) is trained or will be trained with a grayscale gradient image for encoding position as part of the image data, The aforementioned feedforward neural network (FFNN) is trained or will be trained using one or more of the following: a grayscale gradient image, an image of the structure, and a reference light pattern. The method according to any one of claims 1 to 14, wherein at least one of the above.

16. The sensor configuration (10) is a sensor configuration (10) for detecting force, The configuration of the sensor (10) is, An elastically deformable wall (210), the wall (210) comprising an outer measuring surface (220) and an inner reflective surface (230), wherein the reflective surface (230) partially divides the internal space (12), the wall (210) and A configuration (160) of a light source having multiple light sources (164) and configured to emit light toward the internal space (12), An image sensor (130) is installed in the internal space (12) and The method according to any one of claims 2 to 15, comprising:

17. The sensor configuration (10) is a sensor configuration (10) for detecting force, The configuration of the sensor (10) is, - Base (100) and, - A top portion (200) comprising an elastically deformable wall (210), wherein the top portion (200) is attached to the base portion (100) such that the top portion (200) and the base portion (100) define an internal space (12), and the wall (210) comprises an outer measuring surface (220) and an inner reflective surface (230), where the reflective surface (230) partially divides the internal space (12), the top portion (200) and, - A configuration (160) of a light source having a plurality of light sources (164) attached to the base (100) and configured to emit light toward the internal space (12), - An image sensor (130) attached to the base (100) within the internal space (12) and The method according to any one of claims 1 to 16, comprising:

18. The method according to any one of claims 1 to 17, wherein the sensor configuration (10) is either the fingertip of the robot or the operating element of the robot, or both.

19. A force inference module for force inference of a force sensing sensor configuration (10), the force inference module configured to perform the method described in any one of claims 1 to 18.

20. A configuration (10) of a force-sensing sensor, wherein the configuration (10) of the sensor is - Base (100) and, - A top portion (200) comprising an elastically deformable wall (210), wherein the top portion (200) is attached to the base portion (100) such that the top portion (200) and the base portion (100) define an internal space (12), and the wall (210) comprises an outer measuring surface (220) and an inner reflective surface (230), where the reflective surface (230) partially divides the internal space (12), the top portion (200) and, - A configuration (160) of a light source having a plurality of light sources (164) attached to the base (100) and configured to emit light toward the internal space (12), - An image sensor (130) attached to the base (100) within the internal space (12), The force inference module according to claim 19 and A sensor configuration (10) comprising the above.

Images