Visual tactile system
By using event cameras instead of ordinary cameras and eliminating LED fill light, the problems of high power consumption, heat generation and increased costs in visual tactile technology are solved, and normal operation and improved robustness in low-light conditions are achieved.
Patent Information
- Application Number
- CN202510960655.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-11
- Publication Date
- 2025-09-26
AI Technical Summary
The use of LED fill light in existing visual and tactile technology solutions leads to problems such as high power consumption, severe heat generation, reduced system robustness and increased costs.
An event camera is used to replace an ordinary camera, and its ability to work in dark light conditions is utilized. The lighting module is eliminated, and weak external light is allowed to enter through the cavity structure to meet the lighting requirements.
The power consumption and heat generation are reduced, the system robustness is improved, the cost is reduced, and the normal operation of the tactile sensor is achieved.
Smart Images
Figure CN120697099A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot tactile perception, and in particular to a visual-tactile system. Background Art
[0002] Visual tactile technology is a technology that uses visual technology to simulate the tactile perception of robots. Its system mainly consists of the following parts: (1) Contact module: It contains a contact body, a reflective layer, a marking layer, and a protective layer. The contact body is usually made of silicone or latex material and is used to directly contact the object and sense its surface texture and deformation; (2) Illumination module: It provides light source to help capture the deformation and texture information of the contact object; (3) Image acquisition module: It is usually composed of a camera and is used to capture images of the contact object and capture the deformation and texture changes during contact; (4) Information processing module: It is responsible for analyzing and processing image data and extracting tactile information, such as the hardness, texture, contact force, etc. of the object.
[0003] Through the collaborative work of these modules, visual-tactile technology can sensitively perceive rich information about contact with objects. It is widely used in fields such as robotic grasping, object recognition, and human-computer interaction. Currently, visual-tactile technology solutions use event cameras. Event-based cameras are a new type of biologically inspired vision sensor, sometimes also called dynamic vision sensor (DVS), and the derived DAVIS (Dynamic and Active-Pixel Vision Sensor) sensor.
[0004] For example, the paper "Evetac: An Event-based Optical Tactile Sensor for Robotic Manipulation" (https: / / ieeexplore.ieee.org / abstract / document / 10598389) (hereinafter referred to as the related technology) uses an illumination system (with built-in LEDs) like traditional technologies. This leads to the following deficiencies or technical problems: 1. Increased power consumption and heat generation; 2. Adding a module that requires maintenance to the system, thereby reducing system robustness; and 3. Increased system cost. Summary of the Invention
[0005] In order to alleviate or partially alleviate the above-mentioned technical problems, during the process of realizing the present invention, the inventors discovered that the visual and tactile technical solutions of relevant dynamic vision sensor cameras all adopted the red, green and blue color mode (RGB) solution, which required light-emitting diode (LED) fill light. Due to the low-light nature of RGB cameras, methods such as obtaining more three-dimensional graphics by filling in different colors of light have also evolved.
[0006] However, while researching existing DVS visual and tactile technology solutions, the inventors discovered an unexpected and astonishing fact: the same or even better effects can be achieved without fill light. This is because a typical cavity cannot be completely dark. This suddenly enlightened the inventors to the existing inertia of thinking or technical bias: when developing with DVS, people always reflexively or unquestionably must use fill light solutions. All efforts are directed towards designing better fill light solutions, but the idea of removing the lighting system has never been considered.
[0007] The principle behind the ability of event cameras to operate in low light (0.1 lux) is as follows: A contact image sensor (CIS) camera is a scanning camera that directly collects light information reflected from the scanned object by closely arranged photosensitive cells. Because it does not require a lens system, it is more compact and less expensive. Traditional CIS cameras expose the entire image, then control the charge through a readout circuit, such as an analog-to-digital converter (ADC), to convert it into a digital image. During exposure, charge accumulates on the pixel, and when the full well capacity (FWC) is reached, the pixel is overexposed.
[0008] The pixels of an event camera continuously monitor changes in illumination. Whenever the illumination changes by a certain amount, the pixel will output a signal, called an event, and the pixel will be reset. Figure 1 In the same figure, the pixel structure of DAVIS is drawn. It includes the traditional active pixel sensor (APS) and the DVS pixel structure that generates events. The light intensity signal is compared by the photoreceptor, the differentiator and the threshold comparator. If it exceeds the set threshold, an event is output. An event contains four pieces of information (x, y, t, p): position, time, and polarity (brightness or darkening). According to the working principle of traditional CIS cameras and event cameras, Figure 1 It can be seen that the event camera is not sensitive to absolute illumination.
[0009] On this basis, the inventors designed the solution of the present invention, that is, after removing the lighting system, because the event camera can operate at a minimum of 0.1 lux, the visual-tactile system solution of the present invention can solve the technical problems mentioned in the background technology part, while also completing the expected functions of the tactile sensor design.
[0010] Figure 2 and Figure 3 This is a related visual-tactile technology solution, including a contact module, an illumination module, an image acquisition module, and an information processing module. Obviously, the contact module, the illumination module, and the image acquisition module can be arranged in a structural cavity.
[0011] The visual-tactile system proposed in this invention replaces the image acquisition module in related visual-tactile technology solutions with a camera that is insensitive to low light, such as an event camera. The system is then modified to include the following modules: a contact module, an image acquisition module, and an information processing module. The image acquisition module is an event camera.
[0012] Furthermore, the visual-tactile system may further include a cavity structure, in which the contact module and the image acquisition module may be disposed. The structural cavity may specifically be a 3D-printed camera housing.
[0013] The visual-tactile system solution of the present invention has the following beneficial technical effects: the device design can remove the lighting module and still function normally. It also solves the problems associated with the existing lighting system, reduces power consumption and heat generation, improves device robustness, and reduces device costs. Other beneficial effects of the present invention will be discussed in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 The principles of CIS cameras and DVS cameras in related technologies;
[0015] Figure 2 It is a schematic block diagram of the structure of the visual-tactile technology solution in the related art;
[0016] Figure 3 An exploded diagram of the internal structural components of the visual-tactile technology solution in the related art;
[0017] Figure 4 This is a schematic block diagram of the structure of the visual-tactile technology solution in an embodiment of the present invention;
[0018] Figure 5 This is an exploded diagram of the internal structural components of the visual touch technology solution in an embodiment of the present invention.
[0019] The meanings of the reference numerals in the figure are: 1. contact module; 2. lighting module; 3. image acquisition module; 4. information processing module; 5. cavity structure. DETAILED DESCRIPTION
[0020] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0021] See also Figure 1 ,The pixel structure of DAVIS is drawn in the same figure, which includes the traditional active pixel sensor (APS) and the DVS pixel structure that generates events. Figure 1 In the image processing, the light intensity signal is compared by the photoreceptor, the differentiator and the threshold comparator. If the light intensity signal exceeds the set threshold, an event is output. An event contains four pieces of information (x, y, t, p): position, time, and polarity (brightness or darkening). Figure 1 It can be seen that the event camera is not sensitive to absolute illumination.
[0022] See also Figure 2 and Figure 3 The structure of the visual-tactile technology solution in the related visual-tactile technology includes four modules: a contact module 1, a lighting module 2, an image acquisition module 3, and an information processing module 4. The contact module 1, the lighting module 2, and the image acquisition module 3 can be arranged in a cavity structure 5, and the lighting module 2 is specifically an LED strip.
[0023] As described in the background section, the contact module 1 comprises a contact body, a reflective layer, a marking layer, and a protective layer (not shown). The contact body is typically made of silicone or latex material and is used to directly contact an object and sense its surface texture and deformation. The lighting module provides a light source to help capture the deformation and texture information of the contacted object. The image acquisition module, typically composed of a camera, captures images of the contacted object and captures deformation and texture changes during contact. The information processing module analyzes and processes image data to extract tactile information, such as the object's hardness, texture, and contact force.
[0024] Through the collaborative operation of the above modules, the related visual-tactile technology solution can highly sensitively perceive rich information about contact with objects, and is widely used in fields such as robotic grasping, object recognition, and human-computer interaction. However, since the image acquisition module 3 generally uses a conventional camera, and the structural cavity 5 is generally dark or dimly lit, the related visual-tactile technology solution requires the installation of the lighting module 2 within the structural cavity 5 to function properly.
[0025] For this, see Figure 4and Figure 5 In one embodiment of the present invention, the visual-tactile system solution includes: a contact module 1, an image acquisition module 3, and an information processing module 4. The image acquisition module 3 uses an event camera to replace the camera in the related art. This utilizes the event camera's ability to operate in dim light (0.1 lux), eliminating the lighting module 2 in the related art.
[0026] Furthermore, the visual-tactile system may further include a cavity structure 5, and the contact module 1 and the image acquisition module 3 may be disposed within the cavity structure 5. The overall dimensions of the visual-tactile system solution of one embodiment of the present invention are 32 mm in width, 33 mm in height, and 65 mm in length, and the structural cavity 5 is specifically a 3D-printed camera housing.
[0027] As an advantage of the present invention, the visual-tactile system of the present invention does not include an illumination module. In order to adapt to no-lighting conditions, the visual-tactile system of the present invention only requires a small amount of light in the cavity structure 5, which can be achieved in at least one of the following ways:
[0028] (1) A local area on at least one side of the cavity structure 5 (or transparent glass or similar transparent material is further added to prevent dust from entering the cavity) is transparent or hollowed out to allow a small amount of light to penetrate into the cavity. The local area here can be an entire surface or an opening on the surface; (2) In the assembly gap of the cavity structure 5, such as the assembly gap between the contact module 1 and / or the image acquisition module 3, some gaps are intentionally left to allow some light to penetrate into the cavity; (3) At least one component connected to the cavity structure 5 can be intentionally made of a transparent material. For example, the assembly gap is filled with transparent material to allow weak external light to penetrate into the cavity, such that the lighting module 2 no longer has a lighting function and is only a transparent component.
[0029] In one class of embodiments, the above approach allows the brightness within the cavity to be at least 0.1 lux.
[0030] After removing the lighting system, the visual-tactile system solution of the present invention can solve the technical problems mentioned in the background technology section because the event camera can operate at a minimum of 0.1 lux, while also completing the expected functions of the tactile sensor design.
[0031] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.
Claims
1. A visual tactile system, characterized in that: include: Contact module, image acquisition module, information processing module; The image acquisition module adopts an event camera.
2. The visual-tactile system according to claim 1, wherein: It also includes a cavity structure, in which the contact module and the image acquisition module are arranged.
3. The visual-tactile system according to claim 1, wherein: The contact module includes a contact body, a reflective layer, a marking layer and a protective layer.
4. The visual-tactile system according to claim 3, wherein: The contact body is made of silicone or latex material and is used to directly contact an object and sense its surface texture and deformation.
5. The visual-tactile system according to claim 1, wherein: The image acquisition module is used to capture deformation and texture changes during contact.
6. The visual-tactile system according to claim 1, wherein: The information processing module is used to analyze and process image data and extract tactile information.
7. The visual-tactile system according to claim 6, wherein: The tactile information includes the hardness, texture, and contact force of the object.
8. The visual-tactile system according to claim 1, wherein: The event camera can work in low light of 0.1 lux.
9. The visual-tactile system according to claim 1, wherein: The visual-tactile system does not include a lighting module.
10. The visual-tactile system according to claim 2, wherein: At least one local area on one side of the cavity structure is transparent or hollow, or a gap is left at the assembly gap of the cavity structure, or at least one component connected to the cavity structure is made of a transparent material to allow light to penetrate into the cavity structure.