A visual-tactile sensor and a reflection curved surface adjusting method thereof and a computer device

Abstract

The application relates to a visual-tactile sensor and a reflection curved surface adjusting method thereof and a computer device, relates to the technical field of sensors, and the visual-tactile sensor comprises a camera module, a transparent elastomer provided with a pattern layer, and a transparent support arranged in the light path between the camera module and the transparent elastomer. The transparent support comprises an inner surface, an outer surface and a side surface extending between the inner surface and the outer surface. The transparent elastomer is coupled to the inner surface. The outer surface is integrated with a first reflection curved surface. The first reflection curved surface is configured to reflect light from the pattern layer to the camera module, so that the field of view of the camera module covers the pattern layer. Compared with the prior art, the technical scheme can reduce the overall size of the visual-tactile sensor, improve the space utilization, expand the field of view of the camera module to cover the pattern layer, and realize the demand of large sensing area on the premise of small size.

Description

Technical Field

[0001] This application relates to the technical field of sensors, and more particularly to a visual-tactile sensor and a method for adjusting its reflective surface, as well as a computer device. Background Technology

[0002] With their advantages of high resolution and multimodal sensing, visual-tactile sensors have been widely used in fields such as intelligent robots. Their core is to capture the deformation information of the patterned layer on a transparent elastomer through a camera module to realize the perception of external contact. As application scenarios continue to increase the requirements for miniaturization and integration of equipment, the structural design of traditional visual-tactile sensors has gradually revealed obvious limitations.

[0003] The core components of existing visual-tactile sensors, such as camera modules, reflective components, and transparent supports, are mostly set up independently. They require complex installation and adjustment to ensure optical performance, which not only increases the size and weight of the system but also reduces space utilization efficiency, making it difficult to adapt to the installation requirements of miniaturized devices. At the same time, the field of view of the camera module is limited by the physical structure of the lens, and the reflection range of the planar reflective component is limited. As a result, the sensor cannot achieve a large area of ​​sensing coverage while maintaining a small size, making it difficult to balance the core requirements of miniaturization and large sensing area.

[0004] To address the aforementioned issues, some solutions attempt to expand the sensing area by adding a planar mirror inside the visual-touch sensor to extend or fold the optical path. However, such designs are still limited by the size of the rigid planar mirror and fail to overcome the spatial constraints imposed by the independent setting of components. Consequently, they still suffer from low space utilization and limited sensing range expansion. Summary of the Invention

[0005] This application provides a visual-tactile sensor and its reflective surface adjustment method and computer device, which can reduce the overall size of the visual-tactile sensor and improve space utilization, and expand the field of view of the camera module to cover the pattern layer, thus achieving the requirement of a large sensing area while maintaining a small size.

[0006] In a first aspect, this application provides a visual-tactile sensor, comprising: a camera module; a transparent elastomer having a patterned layer; and a transparent support disposed in an optical path between the camera module and the transparent elastomer; the transparent support includes an inner surface, an outer surface, and a side surface extending between the inner and outer surfaces; the transparent elastomer is coupled to the inner surface; the outer surface integrates a first reflective surface; the first reflective surface is configured to reflect light from the patterned layer to the camera module, such that the field of view of the camera module covers the patterned layer.

[0007] In one possible implementation, the outer surface is a curved surface, and a reflective coating is provided on the outer surface; the first reflective curved surface is composed of the curved surface and the reflective coating.

[0008] In one possible implementation, the side surface is provided with a support structure, the support structure including at least one curved surface, on which a second reflective surface is integrated; the first reflective surface receives incident light from the pattern layer and reflects the incident light to the second reflective surface, the second reflective surface reflects the incident light to the camera module, so that the field of view of the camera module covers the pattern layer.

[0009] In one possible implementation, the supporting structure further includes at least one reflective surface; the first reflective surface receives incident light from the pattern layer and reflects the incident light to the second reflective surface, the second reflective surface reflects the incident light to the at least one reflective surface, and the at least one reflective surface sequentially reflects the incident light to the camera module so that the field of view of the camera module covers the pattern layer.

[0010] Secondly, this application provides a method for adjusting a reflective surface, the method being applied to a visual-tactile sensor as described in any of the preceding claims; the method includes: determining whether the field of view of the camera module covers the pattern layer; if the field of view of the camera module does not cover the pattern layer, adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer, until the field of view of the camera module covers the pattern layer.

[0011] In one possible implementation, adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer includes: determining multiple boundary rays corresponding to the field of view boundary of the camera module; calculating the first projected coordinates of the intersection point of the target boundary ray and the pattern layer on a first plane; and calculating the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane; wherein the target boundary ray is any one of the multiple boundary rays; adjusting the first included angle corresponding to the target boundary ray according to the coordinate deviation between the first projected coordinates and the second projected coordinates to adjust the normal vector of the first reflective surface; wherein the first included angle is the angle between the normal vector of the incident ray of the target boundary ray on the first reflective surface and the reflected ray.

[0012] In one possible implementation, calculating the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane includes: determining the target boundary point of the pattern layer closest to the intersection point; and calculating the second projected coordinates of the target boundary point on the first plane.

[0013] In one possible implementation, the first projection coordinates include a first coordinate value in a first direction and a second coordinate value in a second direction; the second projection coordinates include a third coordinate value in the first direction and a fourth coordinate value in the second direction; adjusting the first included angle corresponding to the target boundary ray based on the coordinate deviation between the first projection coordinates and the second projection coordinates specifically includes: adjusting the angle component of the first included angle in the first direction based on the deviation between the first coordinate value and the third coordinate value; and adjusting the angle component of the first included angle in the second direction based on the deviation between the second coordinate value and the fourth coordinate value.

[0014] In one possible implementation, adjusting the angle component of the first included angle in the first direction based on the deviation between the first coordinate value and the third coordinate value specifically includes: if the deviation is positive, increasing the angle component of the corresponding first included angle in the first direction; if the deviation is negative, decreasing the angle component of the corresponding first included angle in the first direction.

[0015] In one possible implementation, if the field of view of the camera module does not cover the pattern layer, the normal vector of the first reflective surface is adjusted based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer. This includes: if the field of view of the camera module does not cover the pattern layer, determining a target pattern sub-region in the pattern layer that is not covered by the field of view of the camera module; adjusting the normal vector of the target first reflective surface sub-region on the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer; the target first reflective surface sub-region is the reflective surface sub-region on the first reflective surface corresponding to the target pattern sub-region; the reflected light from each first reflective surface sub-region corresponds to a pattern sub-region that covers the pattern layer.

[0016] In one possible implementation, adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer until the field of view of the camera module covers the pattern layer includes: determining the first projected coordinates of the intersection point of the field of view boundary ray of the camera module and the pattern layer on a first plane; and calculating the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane; wherein the target boundary ray is any one of the plurality of boundary rays; calculating the coordinate deviation between the first projected coordinates and the second projected coordinates; adjusting the normal vector of the first reflective surface in the first direction based on the first coordinate deviation component of the coordinate deviation in the first direction on the first plane until the field of view of the camera module in the first direction covers the pattern layer; and adjusting the normal vector of the second reflective surface in the second direction based on the second coordinate deviation component of the coordinate deviation in the second direction on the first plane until the field of view of the camera module in the second direction covers the pattern layer.

[0017] In one possible implementation, after the field of view of the camera module covers the pattern layer, the method further includes: determining incident light rays from the pattern layer onto a target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region being any second reflective surface sub-region on the first reflective surface; determining a second angle between the normal vector of each intersection point of the incident light rays and the pattern layer and the corresponding incident light ray; if the angle difference between the second angle and a preset angle exceeds a preset angle threshold, adjusting the direction of the normal vector of the target second reflective surface sub-region until the angle difference does not exceed the preset angle threshold.

[0018] In one possible implementation, after the field of view of the camera module covers the pattern layer, the method further includes: determining incident light rays from the pattern layer onto a target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region being any second reflective surface sub-region on the first reflective surface; calculating the distance between any two adjacent intersection points of each incident light ray in the pattern layer; if the difference between any two distances exceeds a preset distance threshold, adjusting the normal vector direction of the target second reflective surface sub-region until the distance difference does not exceed the preset distance threshold.

[0019] Secondly, this application also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any of the above.

[0020] This application provides a visual-tactile sensor, a method for adjusting its reflective surface, and a computer device, which have the following advantages compared with the prior art:

[0021] This visual-tactile sensor includes a camera module; a transparent elastomer with a patterned layer; and a transparent support body disposed in the optical path between the camera module and the transparent elastomer. The transparent support body includes an inner surface, an outer surface, and a side surface extending between the inner and outer surfaces. The transparent elastomer is coupled to the inner surface. The outer surface integrates a first reflective surface. The first reflective surface is configured to reflect light from the patterned layer back to the camera module, so that the field of view of the camera module covers the patterned layer. Compared with the prior art, the present application's technical solution places the transparent support body in the optical path between the camera module and the patterned transparent elastomer, and the first reflective surface is integrated on the outer surface of the transparent support body. This design enables the transparent support to function as both a support carrier and a light reflector, eliminating the need for a separate reflective component. This simplifies the overall sensor assembly, significantly reduces the sensor's size, and achieves a compact design. It cleverly solves the core problems of traditional visual and tactile sensors, such as scattered components and low space utilization. Meanwhile, the first reflective surface is specifically configured to reflect the light from the pattern layer to the camera module. Compared to traditional planar reflective components, it can more precisely control the light reflection path, effectively expanding the camera module's field of view and ensuring that the camera's field of view completely covers the pattern layer. This achieves the requirement of a large sensing area while maintaining a small size, and ensures optical path stability without additional complex adjustment components, thus balancing integration, sensing performance, and ease of use. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0024] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0025] Figure 1 An exploded view of a visual-tactile sensor provided in an embodiment of this application;

[0026] Figure 2 This is a schematic flowchart of a reflection curve adjustment method provided in an embodiment of this application;

[0027] Figure 3 This is another exploded view of a visual-tactile sensor provided in the embodiments of this application;

[0028] Figure 4 This is a schematic flowchart of a reflection curve adjustment method provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram showing the coverage of the pattern layer by the camera field of view of the camera module provided in this application embodiment;

[0030] Figure 6 This is a schematic diagram of the first included angle provided in an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of the second included angle provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram showing that adjacent intersection points are equidistant from each other, as provided in the embodiments of this application.

[0033] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0034] Explanation of icon numbers:

[0035] Camera module 10, transparent elastomer 20, patterned layer 201, transparent support 30, inner surface 301, outer surface 302, side surface 303, outer shell 40, protective layer 50, camera fixing component 60, connector 70, first reflective surface 304, second reflective surface 305, and load-bearing structure 80. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0038] Example 1, see Figure 1 This application provides a visual-touch sensor, including a camera module 10; a transparent elastomer 20 with a patterned layer 201 on its surface; and a transparent support 30 disposed in the optical path between the camera module 10 and the transparent elastomer 20. The transparent support 30 includes an inner surface 301, an outer surface 302, and a side surface 303 extending between the inner surface 301 and the outer surface 302. The transparent elastomer 20 is coupled to the inner surface 301. The outer surface 302 integrates a first reflective surface 304. The first reflective surface 304 is configured to reflect light from the patterned layer 201 to the camera module 10, so that the field of view of the camera module 10 covers the patterned layer 201.

[0039] In one embodiment, the camera module 10 includes a camera disposed inside the visual-touch sensor for capturing deformation images of the patterned layer 201 on the transparent elastomer 20, providing a visual basis for subsequent contact information calculation. In some embodiments, the camera module 10 further includes a camera mounting bracket for fixing the camera.

[0040] In one embodiment, a transparent elastomer 20 is disposed on a transparent support 30. Specifically, the transparent elastomer 20 is coupled to the inner surface 301 of the transparent support 30. For example, in some embodiments, the transparent elastomer 20 is coupled to the inner surface 301 of the transparent support 30 by means of optical adhesive, achieving a tight fit between the two and avoiding refraction and scattering loss of light due to component gaps during transmission. The transparent elastomer 20 can be made of various materials, including but not limited to silicone, rubber, thermoplastic elastomers, or mixtures thereof. The transparent support 30 is made of a rigid transparent material and provides support for the transparent elastomer 20. A pattern layer 201 is disposed on the surface of the transparent elastomer 20 away from the transparent support 30, and one or more specific patterns are disposed on the pattern layer 201. When the transparent elastomer 20 is subjected to pressure applied by an external object, it deforms, thereby causing one or more specific patterns to produce corresponding shape changes. The camera module 10 identifies the contact information of the external object based on the deformation of these patterns. It should be understood that the embodiments of this application do not limit the specific form of one or more specific patterns. For example, one or more specific patterns can be dot matrix patterns, color block patterns, etc.

[0041] In some embodiments, a first reflective surface 304 is integrated on the outer surface 302 of the transparent support 30. The first reflective surface 304 is a curved surface with light reflection function. The first reflective surface 304 and the transparent support 30 can be two independent parts, with the first reflective surface 304 coupled to the transparent support 30. In some embodiments, the first reflective surface 304 and the transparent support 30 are an integral structure. In this embodiment, the outer surface 302 of the transparent support 30 is a curved surface, and a reflective coating is provided on the outer surface 302. The first reflective surface 304 is composed of this curved surface and the reflective coating. The reflective coating is made of a high-reflectivity optical reflective material, forming an integrated reflective structure with the curved surface of the outer surface 302, eliminating the need for an additional independent reflector and simplifying the sensor's component configuration.

[0042] The first reflective surface 304 is disposed in the optical path between the camera module 10 and the pattern layer 201, and is used to reflect the incident light received from the pattern layer 201 back to the camera module 10. Through the first reflective surface 304, the field of view of the camera module 10 can cover the pattern layer 201, thereby capturing an image that includes the entire effective area of ​​the pattern layer 201.

[0043] Based on the configuration of the first reflective surface 304, the camera module 10 does not need to be directly facing the pattern layer 201. Instead, it can indirectly observe the pattern layer 201 through the reflection of the first reflective surface 304. This fundamentally overcomes the difficulty that the camera module 10 cannot directly and comprehensively observe a large area of ​​the pattern layer 201 due to limitations in physical size and lens field of view. Since the optical path can be folded within the limited space inside the sensor, the visual-tactile sensor can be arranged more compactly in three-dimensional space, thereby significantly reducing the overall volume. Furthermore, the first reflective surface 304 can extend the optical path, ensuring that the field of view of the camera module 10 can cover the entire effective area of ​​the pattern layer 201, achieving effective support for a large sensing area within a limited space.

[0044] In one embodiment, the visual-tactile sensor provided in this application further includes a housing 40. Specifically, a transparent support 30 is fixed to the housing 40. The housing 40 has a receiving cavity for accommodating a camera module 10. In some embodiments, openings are provided on both sides of the receiving cavity, one side opening corresponding to the side surface 303 of the transparent support 30, and the other side opening for inserting the camera module 10 into the receiving cavity. After the camera module 10 is inserted into the receiving cavity, it is fixed by a camera fixing member 60. In some embodiments, the camera fixing member 60 is a fixing screw. In this embodiment, the outer surface of the receiving cavity has screw holes, and the camera module 10 has threaded holes, and the camera module 10 is fixed to the housing 40 by fixing screws.

[0045] In some embodiments, the visual-tactile sensor further includes a light source disposed on the housing 40 for illuminating the transparent elastomer 20 to ensure that the pattern of the pattern layer 201 can be clearly imaged.

[0046] In some embodiments, the visual-tactile sensor further includes a connector 70 for connecting the visual-tactile sensor to a finger joint. The connector 70 can be fixed to the camera module 10, or it can be fixed to the housing 40. Taking the connector 70 being fixed to the camera module 10 as an example, the connector 70 and the camera module 10 are connected in a detachable manner, for example, by screws or clips. The outer side of the connector 70 has a standardized interface for quick docking with the finger joints of a dexterous hand. This interface can be a snap-fit, magnetic, threaded connection, or quick-change flange, supporting quick installation and disassembly by hand or with tool assistance.

[0047] In one embodiment, the visual-tactile sensor further includes a protective layer 50. The protective layer 50 is coupled to the outer surface of the patterned layer 201 of the transparent elastomer 20, providing protection for both the transparent elastomer 20 and the inner patterned layer 201. The protective layer 50 is fixed by coating, bonding, or encapsulation with a support structure, sealing the patterned layer 201 within it, thereby achieving reliable protection of the patterned layer 201 and preventing direct scratching or abrasion of the patterned layer 201 by external objects during tactile sensing, thus ensuring the long-term clarity and integrity of the pattern. In some embodiments, the protective layer 50 has high abrasion resistance, capable of withstanding high-frequency repeated contact and pressing, improving the service life of the visual-tactile sensor.

[0048] The working principle of the visual-tactile sensor provided in this application is explained in detail as follows: When an external contact force is applied to the transparent elastomer 20 through the protective layer 50, the transparent elastomer 20 undergoes different degrees of elastic deformation depending on the contact position and pressure. The deformation of the transparent elastomer 20 causes the pattern layer 201 to produce corresponding morphological changes, encoding the mechanical contact information into optical image information. Furthermore, light from the pattern layer 201, after being transmitted through the transparent elastomer 20, enters the interior of the transparent support 30 through a gapless coupling interface, and then is transmitted to the first reflective surface 304 on its outer surface 302. The first reflective surface 304 directionally reflects the light to the camera module 10, ensuring that the reflected light allows the field of view of the camera module 10 to completely cover the entire pattern layer 201. The above-mentioned visual-tactile sensor structure, through the first reflective surface 304, expands the sensing surface of the visual-tactile sensor, enabling the camera module 10 to cover the entire pattern layer 201 without any blind spots, avoiding the loss of tactile information and ensuring the integrity and accuracy of tactile information recognition. Furthermore, by integrating the first reflective surface 304 into the transparent support 30, the internal optical path of the visual-touch sensor is folded, breaking through the field of view limitation of the camera module 10, improving the space utilization of the visual-touch sensor, and at the same time realizing the design requirements of small size and high sensing area.

[0049] Example 2, see Figure 1-3 This application proposes a visual-tactile sensor. Compared to Embodiment 1, this embodiment optimizes the structure of the transparent support 30 by adding an integrated bearing structure 80 and integrating a second reflective surface 305 to its side surface 303, thus constructing a secondary reflection optical path. Specifically, the first reflective surface 304 receives incident light from the pattern layer 201 and reflects it to the second reflective surface 305. The second reflective surface 305 then reflects the incident light to the camera module 10, so that the field of view of the camera module 10 covers the pattern layer 201. Based on this visual-tactile sensor structure, the first reflective surface 304 and the second reflective surface 305 form a cooperative secondary reflection structure. Through the folding design of the secondary reflection optical path, the field of view is expanded without increasing the sensor volume, ultimately achieving complete coverage of the pattern layer 201 by the field of view of the camera module 10. By coordinating the first reflective surface 304 and the second reflective surface 305, the shooting distance of the camera module 10 is further increased and the field of view of the camera module 10 is expanded; at the same time, the overall compact layout of the sensor is maintained to ensure that the field of view of the camera module 10 can still completely cover the pattern layer 201.

[0050] In one embodiment, the supporting structure 80 and the transparent support 30 are integrally molded. They are formed in one piece using the same high-transmittance optical material as the transparent support 30 through precision injection molding and thermoforming processes, without any splicing gaps or optical steps, thus avoiding light refraction and scattering losses at the connection surface. Furthermore, the overall dimensions of the supporting structure 80 are adapted to the sensor's volume limitations and optical path design requirements, ensuring that all reflected light from the first reflective surface 304 can be incident on the second reflective surface 305 integrated on the supporting structure 80. Optionally, the supporting structure 80 can be configured as a protrusion or a groove, on which the second reflective surface 305 is integrated. The second reflective surface 305 is a continuous smooth surface, and its curvature is determined through co-optimization with the first reflective surface 304. To adapt to the optical path requirements of secondary reflection, the effective reflection area of ​​the second reflective surface 305 matches the light emission range of the first reflective surface 304, preventing light overflow and optical path loss.

[0051] In some embodiments, one surface of the supporting structure 80 is a curved surface, and a reflective coating is disposed on the curved surface. The curved surface and the reflective coating together constitute the second reflective surface 305. By integrating the second reflective surface 305 with the transparent support 30, there is no need to install additional independent reflective components, the component complexity of the sensor is not increased, and the space utilization of the visual-tactile sensor is improved.

[0052] In this embodiment, the lens of the camera module 10 is precisely aligned with the light emission direction of the second reflective surface 305. The second accommodating cavity of the outer shell 40 is adaptively adjusted according to the design of the supporting structure 80, reserving space for light emission and reflection for the supporting structure 80, while still providing protection and fixation for the transparent support 30, the supporting structure 80, and the camera module 10. The protective layer 50 is still coupled to the outside of the pattern layer 201 of the transparent elastomer 20, providing physical protection for the pattern layer 201. The connection method and function of each auxiliary component are completely consistent with those in Embodiment 1, ensuring the stability of the overall visual and tactile structure.

[0053] The overall working logic of the visual-tactile sensor in this embodiment is the same as that in Embodiment 1. The core difference is that the light from the pattern layer 201 is transmitted to the camera module 10 after two directional reflections. The specific optical path transmission process is as follows:

[0054] The reflected light from the pattern layer 201 is transmitted through the transparent elastomer 20 and enters the interior of the transparent support 30 through the gapless coupling interface. The reflected light is then directionally reflected along a preset direction by the first reflective surface 304 of the transparent support 30 to the second reflective surface 305 of the transparent support 30. During this process, the light emission range of the first reflective surface 304 matches the effective reflection area of ​​the second reflective surface 305. After receiving the incident light from the first reflective surface 304, the second reflective surface 305 again directionally reflects it along a preset direction to the camera module 10, achieving a double reflection effect. Further folding of the optical path breaks through the field of view limitation of a single reflective surface, expanding the coverage of light without increasing the size of the sensor. The light reflected by the second reflective surface 305 accurately enters the lens of the camera module 10. Through the coordinated optimization of the first and second reflective surfaces 305, the field of view of the camera module 10 can completely cover the entire pattern layer 201. After the camera module 10 captures an image containing all the deformation information of the pattern layer 201, it transmits it to the supporting processing system for algorithm calculation, which can restore the tactile information such as the position, pressure, and texture of the external contact.

[0055] Example 3: The visual-tactile sensor proposed in this application, compared with Example 2 above, is based on the visual-tactile sensor including the first reflective surface 304 and the second reflective surface 305 in Example 2. The supporting structure 80 of the transparent support 30 is further optimized. By adding at least one reflective surface on the supporting structure 80, a light path system with multiple continuous reflections is constructed to realize the multi-level folding of the light path and the maximum expansion of the field of view. It can adapt to the sensing requirements of the ultra-large area pattern layer 201, while maintaining the overall compact layout of the sensor and ensuring that the field of view of the camera module 10 completely covers the pattern layer 201.

[0056] In one embodiment, the supporting structure 80 further includes at least one reflective surface. A first reflective surface 304 receives incident light from the pattern layer 201 and reflects the incident light to a second reflective surface 305. The second reflective surface 305 reflects the incident light to the at least one reflective surface. The at least one reflective surface sequentially reflects the incident light to the camera module 10, so that the field of view of the camera module 10 covers the pattern layer 201. Through the optical path design of multiple consecutive reflections, the field of view limitation of the camera module is overcome without increasing the physical volume of the sensor, achieving complete coverage of the ultra-large area pattern layer 201.

[0057] At least one of the aforementioned reflective surfaces can be curved or planar. The number of reflective surfaces can be flexibly set to one or more depending on the sensing area of ​​the pattern layer 201 and the volume limitations of the sensor. The effective reflective area of ​​all reflective surfaces is precisely matched with the light emission range of the previous reflective structure, ensuring that the reflected light from the previous reflective structure can all fall within the effective area of ​​the subsequent reflective surface.

[0058] Specifically, each reflective surface is coated with a reflective coating, and the coating thickness and preparation process are consistent with the previous reflective structure. When the reflective surface is an optical plane, all reflective surfaces are pre-set with a fixed tilt angle according to the actual needs during injection molding, based on the optical path design requirements, which greatly reduces the difficulty of processing and optimization.

[0059] Example 4, see Figure 4 This application provides a method for adjusting a reflective surface. This method is applied to a visual-tactile sensor as described in any of the above embodiments. Each visual-tactile sensor includes a first reflective surface. This method is used to adjust the first reflective surface of the visual-tactile sensor in any of the above embodiments. The specific structure of the visual-tactile sensor can be found in the detailed description of the visual-tactile sensor described above, and will not be repeated here. The method includes steps 410-420, as follows:

[0060] Step 410: Determine whether the camera module's field of view covers the pattern layer.

[0061] Step 420: If the field of view of the camera module does not cover the pattern layer, adjust the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer until the field of view of the camera module covers the pattern layer.

[0062] In the above steps, the camera module acquires an image of the pattern layer. Based on a comparison between the acquired image and the complete image of the pattern layer, it is determined whether the camera module's field of view covers the pattern layer. If the camera module's field of view covers the pattern layer, it means the camera module can observe the complete pattern layer, and there is no missing tactile information. If the camera module's field of view does not cover the pattern layer, tactile information in some sensing areas is missing, requiring adjustment of the optical path of the visual-tactile sensor. Based on the deviation between the camera module's field of view boundary and the pattern layer boundary, the portion of the pattern layer not captured by the camera module can be identified; this portion represents the missing tactile information. The normal vector of the first reflective surface of the visual-tactile sensor is adjusted to expand the sensor's sensing surface, thereby ensuring that the camera module's field of view covers the complete pattern layer.

[0063] In one embodiment, the method determines whether the field of view of the camera module covers the pattern layer based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer. Specifically, this includes the following steps: determining multiple boundary rays corresponding to the field of view boundary of the camera module; calculating the first projected coordinates of the intersection point of the target boundary ray and the pattern layer on a first plane; and calculating the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane; wherein the target boundary ray is any one of the multiple boundary rays corresponding to the field of view boundary of the camera module; and determining whether the field of view of the camera module covers the pattern layer based on the coordinate deviation between the first and second projected coordinates.

[0064] Specifically, a spatial coordinate system is established based on the structural characteristics of the visual-tactile sensor, with the geometric center of the pattern layer as the origin. The Z-axis extends along the normal direction of the tangent plane of the pattern layer at the origin, pointing towards the first reflective surface of the transparent support 30. The X-axis and Y-axis extend orthogonally within the tangent plane of the pattern layer at the origin, forming a pairwise orthogonal spatial coordinate system. The first plane can be any plane within this spatial coordinate system, serving as the unified projection reference plane for all subsequent points. For example, in one specific embodiment, the XOY plane is selected as the first plane.

[0065] Specifically, when determining the multiple boundary rays corresponding to the field of view boundary of the camera module, based on the reversible characteristic of light, a ray tracing method is used to determine the field of view boundary of the camera module according to the reverse light path. That is, the actual light propagation path is sequentially: pattern layer 201, transparent elastomer 20, transparent support 30, first reflective surface 304, and camera module 10. According to the reverse light path, simulated light is reflected from the camera module 10 and the first reflective surface 304 on the transparent support 30 to the pattern layer 201, and all boundary rays of the field of view contour edge of the camera module 10 are extracted. Among them, the intersection point of the boundary rays and the pattern layer directly determines the actual coverage range of the camera's field of view.

[0066] Iterate through all boundary rays of camera module 10, treating each boundary ray as a target boundary ray in turn, and calculate the first projected coordinates of the intersection point of the target boundary ray and the pattern layer on the first plane. For any selected target boundary ray, determine the intersection point of the target boundary ray with the pattern layer after passing through the camera module to the pattern layer. Then, following a unified projection rule, project this intersection point vertically onto the first plane, and calculate the two-dimensional first projected coordinates of this intersection point in the first plane. Integrate all the first projected coordinates to form the set of projected coordinates of the intersection points of the field of view boundary rays, which is the actual coverage position of the field of view of camera module 10 on pattern layer 201.

[0067] In some embodiments, when calculating the second projection of the boundary point of the pattern layer corresponding to the intersection point on the first plane, the actual boundary points of the pattern layer are calibrated, and each boundary point of the pattern layer is vertically projected onto the first plane. The two-dimensional second projection coordinates of each boundary point of the pattern layer in the first plane are calculated. All second projection coordinates are integrated to form a set of pattern layer boundary projection coordinates, which represents the target coordinate range to be covered by the camera module's field of view, thus defining the boundary reference for the field of view coverage. The second projection coordinates of the pattern layer boundary point corresponding to the intersection point on the first plane are determined based on the second projection coordinates of each actual boundary point of the pattern layer. Specifically, in the set of pattern layer boundary projection coordinates, the second projection coordinates of the target boundary point of the pattern layer corresponding to the intersection point are matched on the first plane according to the principle of closest proximity. In other embodiments, the actual boundary point of the pattern layer corresponding to the intersection point can be determined first, and then the boundary point can be projected onto the first plane to obtain the second projection coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane. Specifically, the boundary point of the pattern layer corresponding to the intersection point is the target boundary point of the pattern layer closest to the intersection point; the second projection coordinates of the target boundary point on the first plane are calculated to obtain the second projection coordinates of the pattern layer boundary point corresponding to the intersection point on the first plane. It should be understood that the above-described nearest-distance matching principle is merely an example and does not constitute a limitation on the embodiments of this application.

[0068] Taking the first plane as the XOY plane as an example, the coordinate deviation between the first projected coordinates and the corresponding second projected coordinates includes the difference in the X-axis coordinates and the difference in the Y-axis coordinates. In one embodiment, the X-axis coordinate deviation value and Y-axis coordinate deviation value corresponding to all boundary rays are traversed. If the X-axis coordinate deviation value or Y-axis coordinate deviation value corresponding to any boundary ray is less than 0, it indicates that the actual field of view edge of the camera module 10 is within the pattern layer 201, and it is determined that the field of view of the camera module 10 does not cover the pattern layer. If the X-axis coordinate deviation value or Y-axis coordinate deviation value corresponding to any boundary ray is less than 0, it indicates that the actual field of view edge of the camera module 10 is outside the pattern layer 201 and includes the complete pattern layer 201. In this case, it is determined that the field of view of the camera module 10 covers the pattern layer 201. See also Figure 5 , Figure 5 In this diagram, T1 represents the field of view observable by the camera module. This field of view includes both the pattern layer portion used to determine tactile information and the background pattern, such as partial shell imaging. Within T1, T2 represents the pattern layer observable in the camera module's field of view after reflection from the reflective surface. T3 represents the simulated pattern layer's projection mapping (the entire area of ​​the pattern layer) within the camera module's field of view. Figure 5 In the camera's field of view shown in the left figure, the pattern layer T2 that can be observed after reflection by the reflective surface does not cover the entire area T3 of the pattern layer, meaning that there is a lack of tactile information. Figure 5The right-hand image shows that after reflection by the reflective surface, the pattern layer T2 covers the entire area T3 of the pattern layer, and there is no loss of tactile information.

[0069] In one embodiment, adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer includes: determining multiple boundary rays corresponding to the field of view boundary of the camera module; calculating the first projected coordinates of the intersection point of the target boundary ray and the pattern layer on the first plane; and calculating the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane; wherein, the target boundary ray is any one of the multiple boundary rays; adjusting the first included angle corresponding to the target boundary ray according to the coordinate deviation between the first and second projected coordinates to adjust the normal vector of the first reflective surface; wherein, the first included angle θ is the angle between the normal vector of the incident ray of the target boundary ray on the first reflective surface and the reflected ray; see also Figure 6 , Figure 6 This is a schematic diagram of the first included angle provided in the embodiments of this application.

[0070] Specifically, the determination of the first and second projected coordinates and the implementation of their coordinate deviation are as described in step 410 above. Please refer to the detailed explanation of step 410 above. For the sake of brevity, the explanation will not be repeated here.

[0071] Specifically, if the camera module's field of view does not cover the pattern layer, the first angle corresponding to the target boundary ray is adjusted based on the coordinate deviation between the first and second projected coordinates obtained in the above steps, thereby adjusting the normal vector of the first reflective surface. The target boundary ray is a boundary ray ultimately received by the camera module 10, and this target boundary ray forms a complete optical path between the pattern layer 201 and the camera module 10. Part of this optical path involves the ray incident on the first reflective surface 304 and reflected by the first reflective surface 304 to the camera module 10. Specifically, as follows... Figure 6 As shown. The normal vector of the incident ray incident on the first reflecting surface 304 at the incident point of the first reflecting surface 304 is the normal vector of the incident ray on the first reflecting surface, and the angle between the normal vector and the ray reflected to the camera module 10 is defined as the first angle θ.

[0072] In one embodiment, the first projected coordinates include a first coordinate value in a first direction and a second coordinate value in a second direction; the second projected coordinates include a third coordinate value in the first direction and a fourth coordinate value in the second direction. Taking the first plane as the XOY plane, the first direction as the X-axis direction, and the second direction as the Y-axis direction as an example, the first coordinate value and the second coordinate value are respectively the X-axis coordinate value and the Y-axis coordinate value of the first projected coordinates; the third coordinate value and the fourth coordinate value are respectively the X-axis coordinate value and the Y-axis coordinate value of the second projected coordinates.

[0073] In one embodiment, adjusting the first included angle corresponding to the target boundary ray based on the coordinate deviation between the first and second projected coordinates specifically includes: adjusting the angular component of the first included angle in the first direction based on the deviation between the first and third coordinate values; and adjusting the angular component of the first included angle in the second direction based on the deviation between the second and fourth coordinate values. The first included angle determines the reflection direction of the target boundary ray, thereby affecting the position of the camera's field of view, and the change in the angle of the first included angle directly causes a change in the direction of the normal vector at the corresponding position of the first reflection surface. The coordinate difference obtained by subtracting the third coordinate value from the first coordinate value is used as the first direction coordinate deviation, and the coordinate difference obtained by subtracting the fourth coordinate value from the second coordinate value is used as the second direction coordinate deviation. The angular component of the first included angle in the first direction is adjusted based on the first direction deviation, and the angular component of the first included angle in the second direction is adjusted based on the second direction deviation, thereby achieving independent adjustment of the normal vector in both directions.

[0074] In some embodiments, adjusting the angle component of the first included angle in the first direction based on the deviation between the first coordinate value and the third coordinate value specifically includes: if the deviation is positive, increasing the angle component of the corresponding first included angle in the first direction; if the deviation is negative, decreasing the angle component of the corresponding first included angle in the first direction.

[0075] Specifically, based on the deviation between the second and fourth coordinate values, the angular component of the first included angle in the second direction is adjusted. This includes: if the deviation is positive, increasing the corresponding angular component of the first included angle in the second direction; if the deviation is negative, decreasing the corresponding angular component of the first included angle in the second direction. When adjusting the first included angle, the adjusted angular component can be set to a preset angle or set based on the coordinate deviation. For example, in some embodiments, the first included angle is adjusted by a preset angle each time, regardless of the absolute value of the deviation. In other embodiments, the adjustment range of the first included angle is proportional to the absolute value of the coordinate deviation. That is, the larger the absolute value of the coordinate deviation, the larger the adjustment range of the first included angle.

[0076] In some embodiments, based on the coordinate deviation between the first and second projected coordinates corresponding to all boundary rays, the angular component of the first included angle corresponding to each boundary ray in the first direction can be adjusted first, and then the angular component of the first included angle corresponding to each boundary ray in the second direction can be adjusted. In other embodiments, all boundary rays can be traversed, and the angular component of the first included angle corresponding to one boundary ray in the first direction and the angular component of the corresponding first included angle in the second direction can be adjusted first. After the adjustment of the current boundary ray is completed, the subsequent boundary rays are adjusted.

[0077] In one embodiment, the first included angle corresponding to the target boundary light is adjusted by iterative adjustment in order to adjust the normal vector of the first reflective surface; after a single adjustment, step 410 is re-executed to determine whether the field of view of the camera module covers the pattern layer, thereby achieving the coverage of the pattern layer by the field of view of the camera module through iterative adjustment.

[0078] Example 5 optimizes the adjustment step of the normal vector of the first reflective surface in step 420 based on Example 4. It replaces the overall surface adjustment with sub-region iterative adjustment, significantly reducing the computational load and improving iteration efficiency while ensuring complete coverage of the pattern layer by the camera module's field of view. Specifically, in this example, if the field of view of the camera module 10 does not cover the pattern layer, a target pattern sub-region not covered by the field of view of the camera module 10 is identified in the pattern layer. Based on the deviation between the field of view boundary of the camera module 10 and the boundary of the pattern layer, the normal vector of the target first reflective surface sub-region on the first reflective surface 304 is adjusted. The target first reflective surface sub-region is the reflective surface sub-region on the first reflective surface corresponding to the target pattern sub-region. The reflected light from each first reflective surface sub-region corresponds to a pattern sub-region covering the pattern layer. In this example, the first reflective surface 304 is pre-divided into multiple first reflective surface sub-regions, and the pattern layer 201 is pre-divided into multiple pattern sub-regions. The first reflective surface sub-regions and pattern sub-regions correspond one-to-one. That is, the light path of all rays passing through one of the first reflective surface sub-regions passes through the corresponding pattern sub-region and does not pass through other pattern sub-regions. Specifically, by using ray tracing, the coverage area of ​​the reflected light rays from each first reflective surface sub-region on the pattern layer after reflection is determined, establishing a one-to-one coverage relationship between the first reflective surface sub-regions and the pattern layer sub-regions. The sub-region division method of the first reflective surface 304 and the pattern sub-region division method of the pattern layer 201 can be customized by those skilled in the art when implementing the embodiments of this application. For example, with the first plane as a reference, the entire pattern layer can be cut along the minor axis and / or major axis direction of the visual tactile sensor, dividing it into several continuous strip-shaped sub-regions within the first plane. The division method of the first reflective surface 304 is adapted to the division method of the pattern sub-regions. It should be understood that the area and size of each pattern sub-region can be the same or different, and adjacent sub-regions have no overlap or gaps. The area and size of each first reflective surface sub-region can be the same or different, and adjacent first reflective surface sub-regions have no overlap or gaps.

[0079] Based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer, when adjusting the normal vector of the target first reflective surface sub-region on the first reflective surface, only the boundary light corresponding to the target first reflective surface sub-region is extracted, and the first projection coordinate of the intersection point of the boundary light corresponding to the target first reflective surface sub-region and the pattern layer on the first plane is calculated. In addition, the second projection coordinate of the boundary point of the pattern layer corresponding to the intersection point is calculated on the first plane. According to the coordinate deviation between the first projection coordinate and the second projection coordinate, the first included angle corresponding to the boundary light corresponding to the target first reflective surface sub-region is adjusted to adjust the normal vector of the target first reflective surface sub-region. The specific adjustment method is as described in step 420 above, and will not be repeated here.

[0080] In some embodiments, after adjusting the normal vector of the target first reflective surface sub-region, the overall contour of the first reflective surface is smoothed. The smoothing standard is that the normal vector change angle between adjacent first reflective surface sub-regions is the same, so as to avoid abrupt changes in the surface caused by the adjustment of the local first reflective surface sub-region and prevent optical path loss caused by light scattering.

[0081] Example 6: This example is based on the method for adjusting the reflective surface in Example 5 that only includes the first reflective surface. It is a method for adjusting the directional reflective surface of the dual reflective surface proposed in Example 2 for a visual-tactile sensor that includes both the first and second reflective surfaces.

[0082] In one embodiment, based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer, the normal vector of the first reflective surface is adjusted until the field of view of the camera module covers the pattern layer. Then, the first projection coordinates of the intersection point of the field of view boundary ray of the camera module and the pattern layer on the first plane are determined; and the second projection coordinates of the boundary point of the pattern layer corresponding to the intersection point are calculated on the first plane. Herein, the target boundary ray is any one of multiple boundary rays. The coordinate deviation between the first projection coordinates and the second projection coordinates is calculated. Based on the first coordinate deviation component of the coordinate deviation in the first direction of the first plane, the normal vector of the first reflective surface in the first direction is adjusted until the field of view of the camera module in the first direction covers the pattern layer. At this time, based on the second coordinate deviation component of the coordinate deviation in the second direction of the first plane, the normal vector of the second reflective surface in the second direction is also adjusted until the field of view of the camera module in the second direction covers the pattern layer.

[0083] Specifically, when determining the first projected coordinates of the intersection point of the camera module's field-of-view boundary ray and the pattern layer on the first plane, a ray tracing method is used to extract the boundary rays from all rays between the camera module and the pattern layer. These boundary rays are all rays included within the field-of-view boundary of the camera module. Any boundary ray is defined as a target boundary ray. For each target boundary ray, the first projected coordinates of its intersection point with the pattern layer on the first plane and the second projected coordinates of the corresponding pattern layer boundary point on the first plane are calculated. The first projected coordinates include a first coordinate value and a second coordinate value; the second projected coordinates include a third coordinate value and a fourth coordinate value. The first coordinate deviation component in the first direction of the first plane is the coordinate difference between the first and third coordinate values, and the second coordinate deviation component in the second direction of the first plane is the coordinate difference between the second and fourth coordinate values. In some embodiments, the first plane is an XOY plane, where the first direction within the first plane is the X-axis direction and the second direction is the Y-axis direction, or the first direction within the first plane is the Y-axis direction and the second direction is the X-axis direction.

[0084] In one embodiment, based on the principle of independent single-axis adjustment, the normal vector of the first reflective surface is adjusted only for the deviation in the first direction, and the normal vector of the second reflective surface is adjusted only for the deviation in the second direction. The adjustments of the two surfaces do not interfere with each other, ensuring the accuracy of optical path folding. That is, when adjusting the normal vector of the first reflective surface, the normal vector of the second reflective surface is kept fixed. The normal vector of the first reflective surface in the first direction is adjusted only for the deviation component in the first direction until the field of view of the camera module in the first direction completely covers the pattern layer. If the normal vector of the first reflective surface has been adjusted, the normal vector of the first reflective surface is locked unchanged. The normal vector of the second reflective surface in the second direction is adjusted only for the deviation component in the second direction until the field of view of the camera module in the second direction completely covers the pattern layer. Based on this, the adjustment of the first and second reflective surfaces is completed.

[0085] Specifically, when adjusting the normal vector of the first reflective surface in the first direction, the deviation components of the first direction corresponding to all target boundary rays are traversed. If the deviation component of the first direction corresponding to the current target boundary ray is not less than zero, it means that the field of view of the camera module extends to the outside of the pattern layer in the first direction and the field of view of the camera module covers the pattern layer in the first direction. In this case, there is no need to adjust the normal vector of the first reflective surface corresponding to the current target boundary ray in the first direction. If the deviation component of the first direction corresponding to the current target boundary ray is less than zero, it means that the field of view of the camera module in the first direction falls inside the pattern layer and does not extend to the boundary of the pattern layer. That is, the field of view of the camera module does not cover the pattern layer in the first direction. In this case, it is necessary to adjust the normal vector of the first reflective surface corresponding to the current target boundary ray in the first direction.

[0086] If the field of view of the camera module does not cover the pattern layer in the first direction, then the area of ​​the pattern layer not covered in the first direction in the first plane is determined. The first reflective surface can be divided into sub-regions according to the sub-region division rules of Embodiment 5. For the target boundary light rays that cause the field of view of the camera module to not cover the pattern layer in the first direction, a target adjustment sub-region corresponding to the area of ​​the first direction not covered is matched on the first reflective surface. For the target adjustment sub-region, its normal vector in the first direction is adjusted. For example, based on the sign of the first direction deviation component, the angle component of the normal vector of the target boundary light rays on the first reflective surface in the first direction is changed. If the first direction deviation component is positive, the angle component of the normal vector of the target boundary light rays on the first reflective surface in the first direction is increased; if the first direction deviation component is negative, the angle component of the normal vector of the target boundary light rays on the first reflective surface in the first direction is decreased. When adjusting the angle component in the first direction, the adjusted angle component can be set to a preset angle or set based on the coordinate deviation. For example, in some embodiments, regardless of the absolute value of the deviation, the angle component in the first direction is adjusted by a preset angle each time. In other embodiments, the adjustment range of the angle component in the first direction is proportional to the absolute value of the coordinate deviation. That is, the larger the absolute value of the coordinate deviation, the greater the adjustment range of the angular component in the first direction.

[0087] Specifically, when adjusting the normal vector of the second reflective surface in the second direction, the deviation components of the second direction corresponding to all target boundary rays are traversed. If the deviation component of the second direction corresponding to the current target boundary ray is not less than zero, it means that the field of view of the camera module extends to the outside of the pattern layer in the second direction and covers the pattern layer in the second direction. In this case, there is no need to adjust the normal vector of the second reflective surface corresponding to the current target boundary ray in the second direction. If the deviation component of the second direction corresponding to the current target boundary ray is less than zero, it means that the field of view of the camera module in the second direction falls inside the pattern layer and does not extend to the boundary of the pattern layer. That is, the field of view of the camera module does not cover the pattern layer in the second direction. In this case, it is necessary to adjust the normal vector of the second reflective surface corresponding to the current target boundary ray in the second direction.

[0088] If the field of view of the camera module does not cover the pattern layer in the second direction, then the area of ​​the pattern layer in the first plane that is not covered by the field of view of the camera module in the second direction is determined. In some embodiments, the second reflective surface can be divided into sub-regions according to the sub-region division rules of Embodiment 5. For the target boundary light rays of the camera module whose field of view does not cover the pattern layer in the second direction, the target adjustment sub-regions corresponding to the uncovered pattern layer in the second direction are matched on the second reflective surface. For the target adjustment sub-regions, their normal vectors in the second direction are adjusted, such as changing the angular component of the normal vector of the target boundary light rays in the second reflective surface in the second direction based on the sign of the second direction deviation component. If the second direction deviation component is positive, the angular component of the normal vector of the target boundary light rays in the second reflective surface in the second direction is increased; if the second direction deviation component is negative, the angular component of the normal vector of the target boundary light rays in the second reflective surface in the second direction is decreased. The change in the angular component in the second direction is positively correlated with the absolute value of the second direction deviation component. Optionally, a unit angle is adjusted in each iteration.

[0089] Optionally, in this embodiment, the adjustment order of the first reflective surface and the second reflective surface can be flexibly switched. The second reflective surface can be adjusted first, and then the first reflective surface can be adjusted. The specific adjustment order can be flexibly selected according to the actual optical path design of the sensor and the extension direction of the pattern layer, with the ultimate goal of achieving bidirectional full coverage of the field of view of the camera module.

[0090] Example 7 is based on Example 6 and is applicable to the aforementioned single-reflection surface and double-reflection surface visual-tactile sensors. After achieving complete coverage of the pattern layer by the camera module's field of view using the methods of the aforementioned examples, this example finely adjusts the normal vector of the first reflective surface from two dimensions: the incident angle of light and the uniformity of the pattern layer intersection distribution. This eliminates optical distortions such as stretching, compression, and deflection during the imaging process, ensuring the clarity and uniformity of the pattern layer deformation image. This provides a high-quality optical imaging foundation for the accurate calculation of subsequent tactile information such as contact position and pressure.

[0091] In one embodiment, after the field of view of the visual-touch sensor camera module covers the pattern layer, the reflective surface adjustment method further includes: determining the incident light rays from the pattern layer onto a target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region is any second reflective surface sub-region on the first reflective surface; determining a second angle between the normal vector of the intersection point of each incident light ray and the pattern layer and the corresponding incident light ray; if the angle difference between the second angle and the preset angle exceeds a preset angle threshold, adjusting the direction of the normal vector of the target second reflective surface sub-region until the angle difference does not exceed the preset angle threshold. See also Figure 7 , Figure 7 This is a schematic diagram of the second included angle φ provided in an embodiment of this application. Figure 7 In this diagram, based on the principle of reversibility of light paths, the direction of light propagation is marked according to the reverse light path. That is, the actual light path is: pattern layer → first reflective surface → camera module. Figure 7 In the image, the arrow points in the opposite direction of the light path.

[0092] Under ideal imaging conditions, the light rays incident from the pattern layer onto the first reflective surface form a fixed angle with the normal vector at the intersection point of the pattern layer, and this angle is used as a preset angle. Optionally, the preset angle is 0°, meaning the light ray is incident perpendicular to the normal vector at the intersection point of the pattern layer. If the difference between the second angle and the preset angle exceeds a threshold, it indicates significant image distortion. In this case, the angle of the incident light ray needs to be corrected by fine-tuning the normal vector of the target second reflective surface sub-region corresponding to the incident light ray, thereby eliminating the distortion.

[0093] Specifically, the first reflective surface is divided into several independent second reflective surface sub-regions. The method of dividing these sub-regions can be customized by those skilled in the art when implementing the embodiments of this application. For example, in one specific implementation, the second reflective surface sub-regions are divided in a grid pattern. For all incident rays within any grid, the presence of ray deflection distortion is determined based on the angular difference between the second included angle and a preset included angle. If ray deflection distortion exists, the normal vector of the target second reflective surface sub-region corresponding to that incident ray is adjusted to correct the angle of the incident ray and eliminate the ray deflection distortion.

[0094] Taking any second reflective surface sub-region as the target second reflective surface sub-region, determine all incident rays from the pattern layer onto the first reflective surface of the target second reflective surface sub-region, forming a set of incident rays for the target second reflective surface sub-region; for each incident ray in the set of incident rays, determine the intersection point of the incident ray with the pattern layer, and solve for the normal vector of the intersection point on the pattern layer surface, and calculate the second angle between the incident ray and the normal vector of the intersection point; calculate the angle difference between the second angle of each incident ray and the preset angle; if the angle difference of any incident ray exceeds the preset angle threshold, it is determined that the target second reflective surface sub-region has optical distortion and the normal vector needs to be adjusted; if the angle difference of all incident rays is not greater than the preset angle threshold, it is determined that the target second reflective surface sub-region has no distortion and no adjustment is needed, and the next target second reflective surface sub-region is detected.

[0095] In this embodiment, for the distorted target second reflective surface sub-region, only the normal vector of the target second reflective surface sub-region is adjusted. For example, based on the magnitude and direction of the angle difference, the normal vector of each point within the target second reflective surface sub-region is changed so that the angle between the incident light ray and the normal vector of the intersection point of the pattern layer approaches a preset angle. When adjusting the normal vector of each point within the target second reflective surface sub-region, the adjustment range can be determined based on the magnitude of the angle difference, or a fixed angle can be adjusted each time. After adjustment, the second angle and its angle difference with the preset angle are recalculated; the above operation is repeated until the angle difference between the second angle of all incident light rays within the target second reflective surface sub-region and the preset angle does not exceed the preset angle threshold, at which point the distortion of the sub-region is determined to be eliminated; the above process is repeated for all second reflective surface sub-regions of the first reflective surface to complete the distortion optimization adjustment of the entire surface.

[0096] In one embodiment, after the field of view of the camera module covers the pattern layer, the method further includes: determining the incident light rays from the pattern layer onto the target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region is any second reflective surface sub-region on the first reflective surface; calculating the distance between any two adjacent intersection points of each incident light ray in the pattern layer; if the distance difference between any two distances exceeds a preset distance threshold, adjusting the normal vector direction of the target second reflective surface sub-region until the distance difference does not exceed the preset distance threshold.

[0097] Under ideal imaging conditions, light rays incident on the same sub-region of the first reflective surface intersect the pattern layer at points uniformly distributed within the first plane, with minimal distance differences between adjacent intersection points. If the distance difference between any two adjacent intersection points exceeds a preset threshold, it indicates stretching or compression distortion in the pattern layer imaging. In this case, the light propagation path needs to be corrected by fine-tuning the normal vector of the target's second reflective surface sub-region to restore the uniform distribution of intersection points. (See also...) Figure 8 , Figure 8 This is a schematic diagram showing the distance between any two adjacent intersection points of incident rays in the pattern layer under ideal imaging conditions. In the diagram, the distance between adjacent intersection points is d1=d2=d3. Therefore, the distance difference between adjacent intersection points is d3-d2=d2-d1=0.

[0098] In one embodiment, all incident rays from the pattern layer onto the target second reflective surface sub-region on the first reflective surface are determined, and the spatial intersection point of each incident ray with the pattern layer is determined. All spatial intersection points are vertically projected onto the first plane to obtain the two-dimensional projection coordinates of each intersection point in the first plane. The projection coordinates are sorted according to the X-axis or Y-axis direction to form an ordered set of intersection point projection coordinates. Based on the ordered set of intersection point projection coordinates, the planar straight-line distance between any two adjacent intersection points is calculated sequentially, and all calculated distance values ​​are recorded to form a set of adjacent intersection point distances. The distance difference between any two distance values ​​in the distance set is calculated. The uniformity of this distance difference directly reflects the degree of distortion in the pattern layer imaging. The greater the fluctuation of the distance difference, the more severe the stretching / compression distortion. If any distance difference exceeds a preset distance threshold, it is determined that the target second reflective surface sub-region has stretching / compression distortion and the normal vector needs to be adjusted. If all distance differences are not greater than the preset distance threshold, it is determined that the second reflective surface sub-region has no distortion and no adjustment is needed. The next target second reflective surface sub-region is then detected.

[0099] For the distorted target second reflective surface sub-region, only the normal vector of this sub-region is adjusted. For example, based on the distribution characteristics of the distance difference, the normal vector of each point in the target second reflective surface sub-region is slightly changed to correct the propagation path of the incident light and make the distribution of the intersection points of the light and the pattern layer more uniform. The angle step size of each adjustment is a unit angle to avoid deviation of the achieved field of view coverage due to large adjustments. After adjustment, the intersection points are re-extracted and the distance and distance difference between adjacent intersection points are calculated.

[0100] It should be understood that, through Figure 7 and Figure 8 When optimizing and adjusting the distortion of the curved surface in corresponding embodiments, in some embodiments, it may be based solely on... Figure 7 The corresponding embodiments are optimized and adjusted so that the angle difference between the second included angle and the preset included angle does not exceed a preset angle threshold. In some embodiments, it may also be based solely on... Figure 8 The corresponding embodiments are optimized and adjusted so that the distance between any two adjacent intersection points of each incident ray in the pattern layer does not exceed a preset distance threshold. In other embodiments, the distance between any two adjacent intersection points of each incident ray in the pattern layer can be optimized. Figure 7 and Figure 8 The corresponding technical solution combines the following: while ensuring that the angle difference between the second included angle and the preset included angle does not exceed the preset angle threshold, the distance between any two adjacent intersection points of each incident ray in the intersection point of the pattern layer does not exceed the preset distance threshold.

[0101] This application also provides an adjustment device for a reflective surface, including a determining module and an adjusting module. The determining module is used to determine whether the field of view of the camera module covers the pattern layer, and the adjusting module is used to adjust the normal vector of the first reflective surface when the field of view of the camera module does not cover the pattern layer. The specific functions of each module correspond to the steps in the above-described reflective surface adjustment method embodiments, and will not be repeated here.

[0102] like Figure 9 As shown, Figure 9 This is a schematic diagram of a computer device provided in an embodiment of this application; it includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114, and the memory 113 is used to store computer programs. In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the reflective surface adjustment method provided in any of the aforementioned method embodiments.

[0103] This application also provides a computer-readable storage medium storing a computer program that, when executed by a computer, causes the computer to perform the reflective surface adjustment method of any of the above embodiments.

[0104] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0106] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A visual-tactile sensor, characterized in that, include: Camera module; A transparent elastomer with a patterned layer; A transparent support is disposed in the optical path between the camera module and the transparent elastomer; The transparent support includes an inner surface, an outer surface, and a side surface extending between the inner and outer surfaces; the transparent elastomer is coupled to the inner surface; The outer surface is integrated with a first reflective surface; the first reflective surface is configured to reflect light from the pattern layer to the camera module, so that the field of view of the camera module covers the pattern layer. The method for adjusting the reflection surface of the first reflection surface includes: Determine whether the field of view of the camera module covers the pattern layer; If the field of view of the camera module does not cover the pattern layer, the normal vector of the first reflective surface is adjusted based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer until the field of view of the camera module covers the pattern layer.

2. The visual-tactile sensor as described in claim 1, characterized in that, The outer surface is curved, and a reflective coating is provided on the outer surface; the first reflective curved surface is composed of the curved surface and the reflective coating.

3. The visual-tactile sensor as described in claim 1, characterized in that, The side surface is provided with a load-bearing structure, the load-bearing structure includes at least one curved surface, and a second reflective curved surface is integrated on the curved surface; The first reflective surface receives incident light from the pattern layer and reflects the incident light to the second reflective surface, which then reflects the incident light to the camera module so that the field of view of the camera module covers the pattern layer.

4. The visual-tactile sensor as described in claim 3, characterized in that, The supporting structure also includes at least one reflective surface; The first reflective surface receives incident light from the pattern layer and reflects the incident light to the second reflective surface, which in turn reflects the incident light to the at least one reflective surface. The at least one reflective surface sequentially reflects the incident light to the camera module so that the field of view of the camera module covers the pattern layer.

5. A method for adjusting a reflective surface, characterized in that, The method is applied to the visual-tactile sensor as described in any one of claims 1-4; the method includes: Determine whether the field of view of the camera module covers the pattern layer; If the field of view of the camera module does not cover the pattern layer, the normal vector of the first reflective surface is adjusted based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer until the field of view of the camera module covers the pattern layer.

6. The method for adjusting the reflective surface as described in claim 5, characterized in that, The step of adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer includes: Determine multiple boundary rays corresponding to the field of view boundary of the camera module; Calculate the first projected coordinates of the intersection point of the target boundary ray and the pattern layer on the first plane; and calculate the second projected coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane; wherein the target boundary ray is any one of the plurality of boundary rays; Based on the coordinate deviation between the first projected coordinates and the second projected coordinates, the first included angle corresponding to the target boundary ray is adjusted to adjust the normal vector of the first reflective surface; wherein, the first included angle is the angle between the normal vector of the incident ray of the target boundary ray on the first reflective surface and the reflected ray.

7. The method for adjusting the reflective surface as described in claim 6, characterized in that, The calculation of the second projection coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane includes: Determine the target boundary point of the pattern layer that is closest to the intersection point; Calculate the second projected coordinates of the target boundary point on the first plane.

8. The method for adjusting the reflective surface as described in claim 6, characterized in that, The first projection coordinates include a first coordinate value in a first direction and a second coordinate value in a second direction; the second projection coordinates include a third coordinate value in the first direction and a fourth coordinate value in the second direction; adjusting the first included angle corresponding to the target boundary ray based on the coordinate deviation between the first projection coordinates and the second projection coordinates specifically includes: Based on the deviation between the first coordinate value and the third coordinate value, adjust the angular component of the first included angle in the first direction; Based on the deviation between the second coordinate value and the fourth coordinate value, the angular component of the first included angle in the second direction is adjusted.

9. The method for adjusting the reflective surface as described in claim 8, characterized in that, The step of adjusting the angle component of the first included angle in the first direction based on the deviation between the first coordinate value and the third coordinate value specifically includes: If the deviation is positive, then the angular component of the corresponding first included angle in the first direction is increased; If the deviation is negative, then the angular component of the corresponding first included angle in the first direction is reduced.

10. The method for adjusting the reflective surface as described in claim 5, characterized in that, If the field of view of the camera module does not cover the pattern layer, then based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer, the normal vector of the first reflective surface is adjusted, including: If the field of view of the camera module does not cover the pattern layer, determine the target pattern sub-region in the pattern layer that is not covered by the field of view of the camera module; Based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer, the normal vector of the target first reflective surface sub-region on the first reflective surface is adjusted; the target first reflective surface sub-region is the reflective surface sub-region on the first reflective surface corresponding to the target pattern sub-region; the reflected light of each first reflective surface sub-region corresponds to a pattern sub-region of the pattern layer.

11. The method for adjusting the reflective surface as described in claim 6, characterized in that, The step of adjusting the normal vector of the first reflective surface based on the deviation between the field of view boundary of the camera module and the boundary of the pattern layer until the field of view of the camera module covers the pattern layer includes: The first projection coordinates of the intersection point of the field-of-view boundary ray of the camera module and the pattern layer on the first plane are determined; and the second projection coordinates of the boundary point of the pattern layer corresponding to the intersection point on the first plane are calculated; wherein, the target boundary ray is any one of the plurality of boundary rays; Calculate the coordinate deviation between the first projected coordinates and the second projected coordinates; Based on the first coordinate deviation component of the coordinate deviation in the first direction of the first plane, adjust the normal vector of the first reflective surface in the first direction until the field of view of the camera module in the first direction covers the pattern layer; Based on the second coordinate deviation component of the coordinate deviation in the second direction of the first plane, the normal vector of the second reflective surface in the second direction is adjusted until the field of view of the camera module in the second direction covers the pattern layer.

12. The method for adjusting the reflective surface as described in claim 5, characterized in that, After the field of view of the camera module covers the pattern layer, the method further includes: Determine the incident light rays that are incident from the pattern layer onto the target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region is any second reflective surface sub-region on the first reflective surface; Determine the second angle between the normal vector of each intersection point of the incident ray and the pattern layer and the corresponding incident ray; If the angle difference between the second included angle and the preset included angle exceeds the preset angle threshold, then the normal vector direction of the target second reflective surface sub-region is adjusted until the angle difference does not exceed the preset angle threshold.

13. The method for adjusting the reflective surface as described in claim 5, characterized in that, After the field of view of the camera module covers the pattern layer, the method further includes: Determine the incident light rays that are incident from the pattern layer onto the target second reflective surface sub-region on the first reflective surface; the target second reflective surface sub-region is any second reflective surface sub-region on the first reflective surface; Calculate the distance between any two adjacent intersection points of each incident ray in the pattern layer; If the distance difference between any two of the distances exceeds a preset distance threshold, the normal vector direction of the second reflective surface sub-region of the target is adjusted until the distance difference does not exceed the preset distance threshold.

14. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 5-13.

Images