Method for implementing stereoscopic tactile simulation, display module and display device

Abstract

The application relates to the technical field of display, in particular to a method for realizing stereoscopic tactile simulation, a display module and a display device. The method comprises the following steps: acquiring display data and touch information of a display screen, wherein the display data comprises color information and contour information of a display picture, and the touch information comprises coordinate information of an operator touching a display screen area; controlling an electroosmotic flow pump array to deform according to the color information and the touch coordinate, which comprises the following steps: controlling electrodes of electroosmotic flow cavities in a region associated with the touch coordinate on the electroosmotic flow pump array to form an electric field, under the action of the electric field, the electroosmotic flow cavities in the associated region deform to form a plurality of protrusions in the direction of protruding the display screen, and the plurality of protrusions are matched with the contour information; and changing the surface characteristics of the display screen by using the plurality of protrusions, which comprises the following steps: locally lifting a flexible stretchable screen body arranged on the electroosmotic flow pump array by using the plurality of protrusions, and forming a topography matched with the contour information in the area represented by the coordinate information.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a method, display module and display device for realizing stereoscopic haptic simulation. Background Technology

[0002] With the emergence of tactile simulation technology, various ideas have come up one after another. However, the tactile simulation technology for displays focuses more on surface tactile sensations, such as material, temperature, and roughness, and is still lacking in terms of three-dimensional tactile sensation.

[0003] In related technologies, touch and haptic feedback layers are often set on both sides of a glass substrate, respectively, to achieve stripes, graininess, and resistance through electrical pulse stimulation; or piezoelectric ceramic vibration is used to simulate the tactile feel of materials. However, due to the planarity of the display screen, both methods suffer from the problem that the tactile simulation of the display screen is not three-dimensional enough. Furthermore, the former has the drawback of a small dynamic range of electrical pulse stimulation, making it difficult to accurately simulate complex tactile feedback, and electrical stimulation can easily cause user discomfort, such as electric stinging, especially under prolonged use or high-intensity stimulation, which may lead to a decline in user experience. The latter has the drawback of high brittleness and limited mechanical reliability, which may cause internal microcracks to propagate if subjected to external impact during use. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method, display module, and display device for simulating stereoscopic haptic feedback, which can stably simulate stereoscopic haptic feedback on a display screen, providing a superior user experience and a long service life.

[0005] In a first aspect, one embodiment of this application provides a method for realizing stereoscopic tactile simulation, comprising: acquiring display data and touch information of a display screen, wherein the display data includes color information and contour information of the displayed image, and the touch information includes coordinate information of the area touched by the operator on the display screen; controlling the deformation of an electroosmotic pump array according to the color information and coordinate information, comprising: controlling the electrodes of the electroosmotic cavity in the region associated with the coordinate information on the electroosmotic pump array to form an electric field, and under the action of the electric field, the electroosmotic cavity in the associated region deforms to form multiple protrusions in the direction of protrusion from the display screen, the multiple protrusions matching the contour information; and using the multiple protrusions to change the surface features of the display screen, comprising: using the multiple protrusions to lift up a local area of ​​a flexible and stretchable screen body disposed on the electroosmotic pump array, and forming a shape matching the contour information in the region characterized by the coordinate information.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, controlling the formation of an electric field by the electrodes of the electroosmotic cavity in the region associated with the touch coordinates on the electroosmotic pump array includes: controlling the pressure difference between two electrodes in the electroosmotic cavity in the region associated with the touch coordinates on the electroosmotic pump array according to color information, thereby controlling the protrusion height of the protrusion.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, color information includes hue, saturation, and brightness.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the boundary information of multiple protrusions is determined based on hue.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the height of the protrusion is determined based on the saturation, and the height of the protrusion is positively correlated with the saturation.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the protrusion height is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

[0011] Secondly, one embodiment of this application provides a display module for realizing stereoscopic tactile simulation, comprising: an electroosmotic pump array layer and a stretchable screen. The electroosmotic pump array layer includes: a first support layer; multiple independent cavities arranged in an array on one side of the first support layer, each independent cavity being filled with an electrolyte; a second support layer located on the side of the multiple independent cavities away from the first support layer; and multiple sets of electrode assemblies, each set of electrodes including a first electrode and a second electrode disposed at both ends within an independent cavity. The electrode assemblies are configured to form an electric field within the independent cavity, and the electrolyte forms an electroosmotic flow under the action of the electric field. The fluid pressure of the electrolyte changes the shape of the independent cavity, forming a protruding structure protruding from the electroosmotic pump array layer. The stretchable screen is located on one side of the electroosmotic pump array layer, wherein the stretchable screen forms a predetermined morphology in response to the protruding structure.

[0012] In conjunction with the second aspect, in some implementations of the second aspect, the display module for realizing stereoscopic haptic simulation further includes a touch layer and a touch feedback layer. The touch layer is located on the side of the stretchable screen away from the electroosmotic pump array layer. The touch feedback layer is located on the side of the touch layer away from the electroosmotic pump array layer, and the touch feedback layer is configured to form capacitance with the touching finger to generate haptic feedback and simulate the tactile sensation of an object on the touchscreen.

[0013] In conjunction with the second aspect, in some implementations of the second aspect, the touch electrodes of the touch layer are reused as sensors and drivers for haptic feedback in the touch feedback layer.

[0014] In conjunction with the second aspect, in some implementations of the second aspect, the touch feedback layer also includes a protective film layer located on the side of the touch layer away from the electroosmotic pump array layer.

[0015] In conjunction with the second aspect, in some implementations of the second aspect, the protective film layer is a flexible protective layer or a hard coating protective layer or a composite film layer of both.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the first electrode and the second electrode are thin-film electrodes.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the first support layer and the second support layer are flexible and stretchable insulating materials.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, the materials of the first support layer and the second support layer include any or a combination of the following: polyimide, polydimethylsiloxane, polyethylene terephthalate, polyurethane, epoxy resin, polyester film, and cyclic olefin polymer film.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the electrolyte includes any of the following: an aqueous electrolyte, an ionic liquid, a gel electrolyte, or a porous media-enhanced electrolyte.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes: a substrate, which is a flexible support membrane layer, wherein the electroosmotic pump array is located within the substrate.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the protrusion parameters of the protrusion structure are related to the color information of the image displayed on the stretchable screen.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, color information includes hue, saturation, and brightness.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the boundary information of multiple protrusions is determined based on hue.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the height of the protrusion is determined based on the saturation, and the height of the protrusion is positively correlated with the saturation.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the protrusion height is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the stretchable screen includes: a light-emitting layer and an encapsulation layer. The light-emitting layer is located on one side of the electroosmotic pump array layer. The encapsulation layer is located on the side of the light-emitting layer away from the electroosmotic pump array layer.

[0027] In conjunction with the second aspect, in some implementations of the second aspect, the light-emitting layer includes an active driving layer, an anode layer, a hole transport layer, a pixel layer, an electron layer, and a cathode layer stacked sequentially, with the active driving layer located on the side of the light-emitting layer away from the electroosmotic pump array layer.

[0028] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes: a timing control module and an electroosmotic pump control module. The timing control module is configured to acquire display image information of the stretchable screen and receive coordinate information of the touch feedback layer, and generate a tactile trigger signal. The electroosmotic pump control module acquires the tactile trigger signal and is configured to independently control the voltage in each independent cavity of the electroosmotic pump array layer according to the tactile trigger signal, so as to cause local deformation of the electroosmotic pump array layer.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes a display driver module, which is configured to receive display image information output by the timing control module and convert it into an electrical signal, and the electrical signal is configured to guide the stretchable screen to display the image.

[0030] Thirdly, one embodiment of this application provides a display device including the display module of the foregoing embodiment.

[0031] The method, display module, and display device for realizing stereoscopic haptic simulation according to the embodiments of this application simulate stereoscopic haptic sensation by setting a micro electroosmotic pump array layer below a stretchable screen and using touch information obtained by setting a touch feedback layer above the stretchable screen to control the deformation of independent cavities on the electroosmotic pump array layer corresponding to the user's touch area to simulate stereoscopic haptic sensation. Since both the electroosmotic pump array layer and the stretchable screen are made of flexible materials, the stereoscopic haptic simulation of this application has a good user experience, long service life, and high stability. Attached Figure Description

[0032] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0033] Figure 1 The diagram shows a structural schematic of one method for achieving tactile simulation in related technologies.

[0034] Figure 2 The diagram shows a structural schematic of another method for achieving tactile simulation in related technologies.

[0035] Figure 3 The diagram shown is a flowchart of a method for implementing stereoscopic tactile simulation according to an embodiment of this application.

[0036] Figure 4 The diagram shown is a structural schematic of a display module for implementing stereoscopic haptic simulation provided in an embodiment of this application.

[0037] Figure 5 As shown Figure 4 The diagram shows the structure of the electroosmotic pump array layer of the display module.

[0038] Figure label:

[0039] E11, Cover plate; E12, Haptic feedback layer; E13, Glass substrate; E14, Touch film layer; E15, Display panel layer; E21, Cover glass sheet; E22, Transmission device; E23, Touch screen; E24, First stretchable plastic box; E25, Chip; E26, Second stretchable plastic box; 1, Electroosmotic pump array layer; 11, First support layer; 12, Independent cavity; 13, Second support layer; 14, Electrode assembly; 2, Stretchable screen; 21, Active drive layer; 22, Anode layer; 23, Hole transport layer; 24, Pixel layer; 25, Electronic layer; 26, Cathode layer; 27, Encapsulation layer; 3, Polarizing film layer; 4, Touch layer; 5, Protective film layer. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] Figure 1 The diagram shown is a structural schematic of one method for achieving tactile simulation in related technologies. Figure 2 The diagram shows a structural schematic of another method for achieving tactile simulation in related technologies.

[0042] In related technologies, to achieve tactile simulation on the display screen and make the screen more three-dimensional in both visual and tactile aspects, Tanvas Touch technology can be used. Specifically, for example... Figure 1 As shown, the structure, from top to bottom, includes: a cover plate E11, a haptic feedback layer E12, a glass substrate E13, a touch film layer E14, and a display panel layer E15. The touch film layer E14 and the haptic feedback layer E12 are respectively disposed on both sides of the glass substrate E13. Stripes, graininess, and resistance are achieved through electrical pulse stimulation. However, the feedback strength of the above technical solution is insufficient; relying on electrostatic field modulation of surface friction can only simulate micron-level tactile differences (such as roughness), and it is greatly affected by humidity. Alternatively, piezoelectric ceramic vibration can be used to achieve tactile simulation on the display screen, making the screen more three-dimensional in both visual and tactile aspects. Specifically, as... Figure 2As shown, the structure, from top to bottom, includes: a cover glass E21, a transmission device E22, a touch screen E23, a first stretchable plastic box E24, a chip E25, and a second stretchable plastic box E26. By integrating the transmission device E22 on the touch screen E23, the transmission device E22 generates vibrations to simulate a rough surface: high-frequency micro-amplitude vibrations (such as the feel of sandpaper), and through the micron-level deformation of piezoelectric ceramics (such as a protrusion height of 5-50μm), a tactile texture (such as wood grain or fabric) is generated in real time. However, this technology has the drawbacks of excessive power consumption, poor stability, and susceptibility to damage from external impacts.

[0043] Figure 3 The diagram shown is a flowchart of a method for implementing stereoscopic tactile simulation according to an embodiment of this application.

[0044] To address the aforementioned technical problems, embodiments of this application in the first aspect provide a method for implementing stereoscopic tactile simulation, such as... Figure 3 As shown, the method includes operations S1 to S3.

[0045] Operation S1 includes: acquiring display data and touch information from the display screen, wherein the display data includes color information and outline information of the displayed image, and the touch information includes coordinate information of the area of ​​the display screen touched by the operator.

[0046] Operation S2 includes: controlling the electroosmotic pump array to deform according to color information and coordinate information, including: controlling the electrodes of the electroosmotic cavity in the region associated with the coordinate information on the electroosmotic pump array to form an electric field, and under the action of the electric field, the electroosmotic cavity in the associated region deforms to form multiple protrusions in the direction of protruding from the display screen, and the multiple protrusions are matched with the contour information.

[0047] Operation S3 includes: using multiple protrusions to change the surface features of the display screen, including: using multiple protrusions to lift up a local area of ​​the flexible stretchable screen body set on the electroosmotic pump array, and forming a shape that matches the contour information in the area characterized by coordinate information.

[0048] In this embodiment, by setting a micro-electroosmotic pump array layer below the stretchable screen, and using the touch information obtained by setting a touch feedback layer above the stretchable screen, the independent cavities on the electroosmotic pump array layer corresponding to the user's touch area are deformed to simulate a three-dimensional tactile sensation. Since both the electroosmotic pump array layer and the stretchable screen are made of flexible materials, the three-dimensional tactile simulation of this application has a good user experience, a long service life, and high robustness.

[0049] In conjunction with the first aspect, in some implementations of the first aspect, in operation S1, a convolutional neural network (CNN) can be used to analyze the RGB-D data (color + contour depth) of the display screen in real time, and combine it with capacitive touch coordinates (accuracy ±0.5mm) to obtain the relevant color information, contour information and touch information of the display screen.

[0050] Specifically, display data refers to the content information being displayed on the display panel. Color information encompasses the color attributes of each pixel in the displayed image, such as the RGB (red, green, blue) primary color components, which collectively determine the final color of each pixel. Contour information describes the shape and boundaries of objects in the displayed image, and can be extracted from the displayed image using edge detection algorithms (such as Sobel and Canny). For example, when displaying a picture of fruit, the outline of the fruit is the contour information, while the red, green, and other colors of the fruit are the color information.

[0051] Touch information is obtained through a touch feedback layer, which typically employs capacitive, resistive, or optical touch technologies. Capacitive touch detects the touch location by utilizing the capacitance change between the human body and electrodes; resistive touch determines the location by generating an electrical signal through the contact of two conductive films under pressure; and optical touch determines the touch coordinates by detecting light occlusion. When an operator touches the display screen, the touch feedback layer can accurately acquire the coordinate information of the touch area, in pixels. For example, on a 1920×1080 resolution screen, the coordinate information can be accurate to the specific row and column values.

[0052] Accurately acquiring display data and touch information is fundamental to achieving 3D haptic simulation. Display data provides the basis for subsequent simulations that match the tactile sensation to the screen content, while touch information clarifies the specific location of the user's operation, enabling haptic feedback to precisely target the area touched by the user and enhance the user experience.

[0053] In conjunction with the first aspect, in some implementations of the first aspect, during operation S2, the electroosmotic pump array layer consists of multiple independent cavities, each filled with electrolyte and equipped with a first electrode and a second electrode at both ends. When a voltage is applied to the electrodes, an electric field is generated within the cavity. Under the influence of the electric field, the electrolyte undergoes electroosmotic flow, that is, directional flow of liquid in the electrolyte solution relative to a stationary charged surface. This flow generates fluid pressure, causing the independent cavities to deform and form bulge structures. For example, when positive and negative voltages are applied to the two ends of an independent cavity, positive ions move towards the negative electrode, and negative ions move towards the positive electrode, driving the surrounding liquid to flow, thereby causing the cavity to expand and form bulges.

[0054] By obtaining the touch coordinates from the touch feedback layer, the specific area touched by the user can be determined. Then, an electric field is created by controlling the electrodes of the electroosmotic cavity associated with that area, causing only the cavity in the touched area to deform and form a protrusion. For example, when a user touches an image of a gemstone on the screen, only the electroosmotic cavity corresponding to the area where the gemstone is located deforms, simulating the three-dimensional feel of the gemstone.

[0055] In accordance with the first aspect, in certain implementations of the first aspect, during operation S3, the flexible and stretchable screen exhibits good flexibility and stretchability, enabling it to undergo localized deformation under the convex effect formed by the electroosmotic pump array. Such screens typically employ technologies such as organic light-emitting diodes (OLEDs), which possess advantages such as self-illumination, wide viewing angle, and fast response speed, and whose thin-film structure facilitates flexible screen fabrication.

[0056] Once multiple protrusions are formed on the electroosmotic pump array, they push upwards on a localized area of ​​the flexible, stretchable screen. Because the protrusions match the contour information, the screen will form a shape consistent with the outline of an object in the displayed image in the area touched by the user. For example, when the displayed image is a circular button, the protrusions will push the screen upwards to form a circular raised area, allowing the user to feel the three-dimensional shape of the button when touched.

[0057] By altering the surface features of the display screen with raised bumps, users can directly perceive the shape and contour of objects when touching the screen, further enhancing the realism of tactile feedback. Simultaneously, the use of a flexible and stretchable screen body makes the display module thinner, lighter, and more flexible, suitable for devices of various shapes, such as wearable devices and curved-screen phones.

[0058] In conjunction with the first aspect, in some implementations of the first aspect, controlling the formation of an electric field by the electrodes of the electroosmotic cavity in the region associated with the touch coordinates on the electroosmotic pump array includes: controlling the pressure difference between two electrodes in the electroosmotic cavity in the region associated with the touch coordinates on the electroosmotic pump array according to color information, thereby controlling the protrusion height of the protrusion.

[0059] In conjunction with the first aspect, in some implementations of the first aspect, color information includes hue, saturation, and brightness.

[0060] Hue, saturation, and brightness in color information all affect the formation of embossing. Hue determines the boundary information of the embossing; different hues represent different color areas, and different color areas may require different tactile boundaries when simulating tactile sensation. For example, when displaying a map, land and ocean are different colors, and their corresponding tactile boundaries should also be different. Saturation and brightness are positively correlated with the height of the embossing; the higher the saturation and the brighter the brightness, the higher the embossing height. This is because highly saturated and bright colors usually give people a stronger and more prominent visual experience, and in tactile simulation, a higher embossing height is needed to enhance this experience.

[0061] In conjunction with the first aspect, in some implementations of the first aspect, the boundary information of multiple protrusions is determined based on hue. For example, tomatoes are red and leaves are green, and the area where red and green intersect is the boundary. Different hues result in different protrusion heights, with different heights on both sides of the boundary, thus visually distinguishing tomatoes and leaves through touch.

[0062] In conjunction with the first aspect, in some implementations of the first aspect, the height of the protrusion is determined based on saturation, and the protrusion height is positively correlated with saturation. Optionally, for example, the protrusion for deep red is greater than that for bright red, which is greater than that for light red.

[0063] In this embodiment, highly saturated colors (such as pure red) are visually more "prominent," and multi-sensory consistency can be enhanced by increasing the corresponding tactile feedback intensity (protrusion height). Studies have shown that tactile intensity and color saturation have a linear perceptual correlation (Weber-Fechner's law). Specifically, a mapping between saturation and driving voltage can be established, that is, the electric field strength is proportional to the electrolyte displacement (Δz∝E).

[0064] In conjunction with the first aspect, in some implementations of the first aspect, the protrusion height is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

[0065] In this embodiment, high-brightness areas (such as white) are visually perceived as "proximal" or "protruding," and the sense of depth can be enhanced by matching tactile protrusions. Experiments show that for every 10% increase in brightness, the perceived tactile height increases by approximately 8% (data from the MIT Haptic Lab).

[0066] In conjunction with the first aspect, in some implementations of the first aspect, multiple parameters affecting the height of the protrusion can be coordinated and controlled by weighting and weighted summation.

[0067] Figure 4 The diagram shown is a structural schematic of a display module for implementing stereoscopic haptic simulation according to an embodiment of this application. Figure 5 As shown Figure 4 The diagram shows the structure of the electroosmotic pump array layer of the display module.

[0068] Secondly, one embodiment of this application provides a display module for realizing stereoscopic haptic simulation, such as... Figure 4 and Figure 5 As shown, the display module includes an electroosmotic pump array layer 1 and a stretchable screen 2. The electroosmotic pump array layer 1 includes: a first support layer 11; multiple independent cavities 12 arranged in an array on one side of the first support layer 11, each independent cavity 12 being filled with electrolyte; a second support layer 13 located on the side of the multiple independent cavities 12 away from the first support layer 11; and multiple sets of electrode assemblies 14, each set of electrodes including a first electrode and a second electrode disposed at both ends within an independent cavity 12. The electrode assemblies 14 are configured to form an electric field within the independent cavities 12, and the electrolyte forms an electroosmotic flow under the action of the electric field. The fluid pressure of the electrolyte changes the shape of the independent cavities 12, forming a protruding structure protruding from the electroosmotic pump array layer 1. The stretchable screen 2 is located on one side of the electroosmotic pump array layer 1, wherein the stretchable screen 2 forms a preset morphology in response to the protruding structure.

[0069] In this embodiment, by setting a micro electroosmotic pump array layer 1 below the stretchable screen 2, and using the touch information obtained by setting a touch feedback layer above the stretchable screen 2, the independent cavity 12 on the electroosmotic pump array layer 1 corresponding to the user's touch area is deformed to simulate a three-dimensional tactile sensation. Since both the electroosmotic pump array layer 1 and the stretchable screen 2 are made of flexible materials, the three-dimensional tactile simulation of this application has a good user experience, a long service life, and high stability.

[0070] In conjunction with the second aspect, in some implementations of the second aspect, the first support layer 11 and the second support layer 13 serve to support and fix the independent cavity 12. They are made of flexible and stretchable insulating materials, such as polyimide and polydimethylsiloxane. Polyimide has excellent thermal stability, mechanical properties, and insulation properties, and can remain stable in high-temperature and complex environments; polydimethylsiloxane has good flexibility and biocompatibility, making it suitable for applications with high flexibility requirements. The selection of these materials ensures that the electroosmotic pump array layer 1 can deform with the stretchable screen 2, while avoiding problems such as leakage.

[0071] In conjunction with the second aspect, in some implementations of the second aspect, multiple independent cavities 12 are arranged in an array on the first support layer 11, with each cavity equivalent to an independent electroosmotic pump unit. The electrolyte filled in the cavity is the key medium for generating electroosmotic flow; common electrolytes include water-based electrolytes and ionic liquids. Water-based electrolytes are low in cost and have good conductivity, but are prone to volatility; ionic liquids have advantages such as low volatility and a wide electrochemical window, but are more expensive. The choice of different electrolytes will affect the performance and service life of the electroosmotic pump. The first and second electrodes of each electrode assembly 14 are set at both ends of the independent cavity 12, and an electric field is formed in the cavity by applying a voltage. The electrodes are usually thin-film electrodes, such as metal thin-film electrodes (e.g., copper, aluminum, etc.) or conductive polymer thin-film electrodes. Metal thin-film electrodes have good conductivity, but relatively poor flexibility; conductive polymer thin-film electrodes have better flexibility and processability. The design and layout of the electrodes will affect the distribution and intensity of the electric field, thereby affecting the generation of electroosmotic flow and the deformation effect of the cavity.

[0072] In conjunction with the second aspect, in some implementations of the second aspect, the display module for realizing stereoscopic haptic simulation further includes a touch layer 4 and a touch feedback layer. The touch layer 4 is located on the side of the stretchable screen 2 away from the electroosmotic pump array layer 1. The touch feedback layer is located on the side of the touch layer 4 away from the electroosmotic pump array layer 1, and is configured to form capacitance with the touching finger to generate haptic feedback and simulate the tactile sensation of an object on the touchscreen.

[0073] Touch layer 4 employs capacitive, resistive, or optical touch technologies. A capacitive touch layer 4 consists of multiple conductive films. When a finger touches the screen, the capacitance between the human body and the conductive films changes; the touch coordinates are determined by detecting the location of this capacitance change. A resistive touch layer 4 consists of two conductive films separated by tiny insulating dots. When a finger presses the screen, the two conductive films contact each other, generating an electrical signal to determine the touch location. An optical touch layer 4 uses infrared emitters and receivers at the screen edge to form an infrared grating. When a finger touches the screen, it blocks some infrared light; the touch coordinates are determined by detecting the location of this blocking.

[0074] The haptic feedback layer is an interaction enhancement module independent of Touch Layer 4. It conveys non-visual information such as operation confirmation and interface state changes to the user through vibration, tactile simulation, or light effects. Its core value lies in enhancing the intuitiveness and immersion of the interaction, serving as the "feedback channel" for touch interaction.

[0075] In this embodiment, the touch feedback layer relies on capacitive or resistive technology to calculate touch coordinates by detecting changes in the electric field or pressure on the screen surface. For example, the capacitive touch layer 4 utilizes the electrostatic field of the human body to change the capacitance value between electrodes, thereby achieving multi-touch.

[0076] In conjunction with the second aspect, in some implementations of the second aspect, the touch electrodes of the touch layer 4 are reused as sensors and drivers for haptic feedback in the touch feedback layer, that is, the touch layer 4 and the touch feedback layer can be the same layer structure.

[0077] In this embodiment, based on the traditional touch layer 4, this application embodiment assigns different functions to the same set of TP patterns (Touch Panel Patterns) at different time periods. When haptic feedback is required, the system dynamically adjusts the driving signal of the TP pattern to form a dynamically changing capacitive coupling with the finger. This capacitive change generates a weak electric field force, which acts on the receptors on the finger skin, thereby simulating tactile sensation. Through precise time division, the use of the TP pattern is divided into a touch detection stage and a haptic feedback stage. In the touch detection stage, the TP pattern works normally to identify the touch position; in the haptic feedback stage, it switches to the haptic signal output mode to avoid mutual interference between touch detection and haptic feedback signals, ensuring that the two functions operate independently and efficiently.

[0078] Specifically, by rapidly and precisely controlling the capacitance changes in different areas of the TP pattern, the microscopic undulations of an object's surface are simulated. When a finger slides across the screen, the frequency and amplitude differences in capacitance changes in different areas are perceived by the skin, thus producing a tactile sensation similar to real texture. For example, when simulating wood grain, the capacitance change sequence of the TP pattern can be designed according to the direction of the wood grain, allowing the finger to feel a texture and unevenness similar to real wood.

[0079] The frequency and intensity of capacitance changes in the TP pattern are used to adjust the amount of friction felt by the fingers. High-frequency, small-amplitude capacitance changes simulate a smooth surface, making the fingers glide smoothly; low-frequency, large-amplitude capacitance changes simulate a rough surface, increasing the resistance to finger gliding. By adjusting these parameters in real time, it is possible to simulate the tactile sensations of various surfaces, from smooth to rough.

[0080] Different materials possess unique tactile properties, such as the coldness of metal and the smoothness of silk. Haptic feedback systems can comprehensively simulate the tactile sensation of different materials by controlling the capacitance changes of the TP pattern and combining this with temperature simulation techniques (such as localized heating or cooling). For example, when simulating metal, while simulating texture through capacitance changes, appropriate cooling can be applied to a localized area of ​​the screen to give the fingers a similar cold, metallic feel. Localized heating can be achieved through methods such as heating with transparent conductive films, microfluidic liquid circulation, and energy storage using phase change materials; localized cooling can be achieved using semiconductor cooling chips utilizing the Peltier effect; or localized cooling can be achieved using aerogel insulation layers to assist in cooling.

[0081] In conjunction with the second aspect, in some implementations of the second aspect, the touch layer 4 includes driving electrodes (Tx) and receiving electrodes (Rx), which are arranged in a grid array and serve as the TP pattern of the touch feedback layer. Specifically, the Tx electrodes actively transmit electrical signals (such as sine waves or pulses), while the Rx electrodes passively receive signals and detect capacitance changes caused by finger touches. Signal sensitivity and transmittance are optimized through rhomboid, rhomboid-rhomboid nested, and snowflake-shaped topologies.

[0082] In conjunction with the second aspect, in some implementations of the second aspect, the touch feedback layer also includes a protective film layer 5, which is located on the side of the touch layer 4 away from the electroosmotic pump array layer 1.

[0083] In conjunction with the second aspect, in some implementations of the second aspect, the protective film layer 5 is a flexible protective layer or a hard coating protective layer or a composite film layer of both.

[0084] In this embodiment, the protective film layer 5, as the outermost layer of the display module, can prevent external physical damage (such as scratches and impacts) from damaging the touch layer 4 and the electroosmotic pump array layer 1, thus extending the service life of the device. In addition, it also has a chemical protective function. For example, it reduces fingerprints and oil stains by using a hydrophobic and oleophobic coating (such as fluoride), maintaining screen cleanliness and reducing maintenance costs.

[0085] Flexible protective layers can be made of materials such as polyurethane (TPU) and polyethylene terephthalate (PET), possessing high flexibility (bending radius <1mm), impact resistance (impact absorption rate >90%), and self-healing capabilities (e.g., TPU repairs scratches after 24 hours at 40°C), making them suitable for wearable devices, foldable screens, and other applications. Hard coating protective layers can be made of inorganic nano-coatings such as silica (SiO2) and alumina (Al2O3), or organic-inorganic composite coatings (e.g., polysiloxane-SiO2). Composite films can be made of a flexible substrate (e.g., PET) + a hard coating (e.g., SiO2) + a functional coating (e.g., anti-fingerprint coating).

[0086] In conjunction with the second aspect, in some implementations of the second aspect, the first electrode and the second electrode are thin-film electrodes. Optionally, the thin-film electrode includes a metal thin-film electrode, a conductive polymer thin-film electrode, a carbon-based thin-film electrode, etc.

[0087] In conjunction with the second aspect, in some implementations of the second aspect, the first support layer 11 and the second support layer 13 are flexible and stretchable insulating materials.

[0088] In conjunction with the second aspect, in some implementations of the second aspect, the materials of the first support layer 11 and the second support layer 13 include any or a combination of the following: polyimide, polydimethylsiloxane, polyethylene terephthalate, polyurethane, epoxy resin, polyester film, and cyclic olefin polymer film.

[0089] In conjunction with the second aspect, in some implementations of the second aspect, the electrolyte includes any of the following: an aqueous electrolyte, an ionic liquid, a gel electrolyte, or a porous media-enhanced electrolyte.

[0090] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes: a substrate, which is a flexible support membrane layer, wherein the electroosmotic pump array is located within the substrate.

[0091] In conjunction with the second aspect, in some implementations of the second aspect, the protrusion parameters of the protrusion structure are related to the color information of the image displayed on the stretchable screen 2.

[0092] In conjunction with the second aspect, in some implementations of the second aspect, color information includes hue, saturation, and brightness.

[0093] In conjunction with the second aspect, in some implementations of the second aspect, the boundary information of multiple protrusions is determined based on hue.

[0094] In conjunction with the second aspect, in some implementations of the second aspect, the height of the protrusion is determined based on the saturation, and the height of the protrusion is positively correlated with the saturation.

[0095] In conjunction with the second aspect, in some implementations of the second aspect, the protrusion height is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

[0096] In conjunction with the second aspect, in some implementations of the second aspect, the stretchable screen 2 includes: a light-emitting layer and an encapsulation layer 27. The light-emitting layer is located on one side of the electroosmotic pump array layer 1. The encapsulation layer 27 is located on the side of the light-emitting layer away from the electroosmotic pump array layer 1.

[0097] In conjunction with the second aspect, in some implementations of the second aspect, the light-emitting layer includes an active driving layer 21, an anode layer 22, a hole transport layer 23, a pixel layer 24, an electron layer 25, and a cathode layer 26 stacked sequentially, with the active driving layer 21 located on the side of the light-emitting layer away from the electroosmotic pump array layer 1.

[0098] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes a timing control module and an electroosmotic pump control module. The timing control module is configured to acquire display image information of the stretchable screen 2 and receive coordinate information of the touch feedback layer, and generate a tactile trigger signal. The electroosmotic pump control module acquires the tactile trigger signal and is configured to independently control the voltage within each independent cavity 12 of the electroosmotic pump array layer 1 according to the tactile trigger signal, so as to cause local deformation of the electroosmotic pump array layer 1.

[0099] In conjunction with the second aspect, in some implementations of the second aspect, the display module further includes a display driver module, which is configured to receive display image information output by the timing control module and convert it into an electrical signal, and the electrical signal is configured to guide the stretchable screen 2 to display an image.

[0100] In this embodiment, the timing control module is the "brain" of the display module, responsible for coordinating the work of each module. It acquires the display image information of the stretchable screen 2 and the coordinate information of the touch feedback layer, and generates tactile trigger signals based on this information. For example, when a user touches a specific area on the screen, the timing control module generates a corresponding tactile trigger signal based on the color and contour information of that area, controlling the electroosmotic pump array layer 1 to produce a corresponding protrusion deformation.

[0101] The electroosmotic pump control module receives a tactile trigger signal generated by the timing control module and independently controls the voltage within each independent cavity 12 of the electroosmotic pump array layer 1 based on this signal. By precisely controlling the voltage of each cavity, local deformation of the electroosmotic pump array layer 1 can be achieved, thereby simulating a three-dimensional tactile sensation that matches the displayed content. For example, for areas requiring a high protrusion height, the control module increases the voltage of the corresponding cavity to enhance electroosmotic flow, causing the cavity to expand higher.

[0102] The display driver module receives the display image information output by the timing control module and converts it into electrical signals to guide the stretchable screen 2 in displaying the image. It is responsible for controlling the luminous intensity and color of each pixel, ensuring the quality and clarity of the displayed image. The display driver module typically uses integrated circuit chips, featuring high speed and high precision.

[0103] The various components work together to enable the display module to provide both display and haptic feedback. The substrate provides stable support, the timing control module ensures the coordinated operation of all modules, the electroosmotic pump control module achieves accurate haptic simulation, and the display driver module guarantees high-quality image display.

[0104] Thirdly, one embodiment of this application provides a display device, including the display module described in the foregoing embodiments. It should be noted that the display device can be various electronic display products, specifically including but not limited to at least one of mobile phones, tablet computers, e-book readers, media players, digital cameras, laptops, in-vehicle computers, desktop computers, set-top boxes, smart TVs, and wearable devices.

[0105] In addition, depending on actual needs, the display device may also include other structures such as a touch panel.

[0106] Since the display device in this application embodiment includes the above-mentioned... Figures 3 to 5 All the technical solutions in the embodiments shown can achieve at least all the above-mentioned technical effects, and will not be repeated here.

[0107] In the embodiments of this disclosure, unless otherwise specified, the connection can be a detachable connection using bolts, nuts, screws, clips, magnets, etc. In some connections where there is no particular requirement for a detachable fit, a non-detachable connection can be achieved through welding, bonding, etc.

[0108] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0109] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0110] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0111] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0112] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for realizing stereoscopic tactile simulation, characterized in that, include: The display data and touch information of the display screen are acquired, wherein the display data includes color information and outline information of the displayed image, and the touch information includes coordinate information of the area of ​​the display screen touched by the operator; Controlling the deformation of the electroosmotic pump array according to the color information and the coordinate information includes: controlling the electrodes of the electroosmotic cavity in the region associated with the coordinate information on the electroosmotic pump array to form an electric field, and under the action of the electric field, the electroosmotic cavity in the associated region deforms to form multiple protrusions in the direction of protruding from the display screen, and the multiple protrusions are matched with the contour information; Changing the surface features of the display screen by using the multiple protrusions includes: using the multiple protrusions to lift up a local area of ​​the flexible stretchable screen body disposed on the electroosmotic pump array, and forming a shape that matches the contour information in the area characterized by the coordinate information.

2. The method for realizing stereoscopic tactile simulation according to claim 1, characterized in that, The process of controlling the formation of an electric field by the electrodes of the electroosmotic cavity in the region associated with the touch coordinates on the electroosmotic pump array includes: The pressure difference between the two electrodes in the electroosmotic cavity within the area associated with the touch coordinates on the electroosmotic pump array is controlled according to the color information, thereby controlling the protrusion height of the protrusion; Preferably, the color information includes hue, saturation, and brightness; Preferably, the boundary information of the plurality of protrusions is determined based on the hue; Preferably, the protrusion height is determined based on the saturation, and the protrusion height is positively correlated with the saturation. Preferably, the protrusion height of the protrusion is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

3. A display module for realizing stereoscopic haptic simulation, characterized in that, include: The electroosmotic pump array layer includes: First support layer; Multiple independent cavities are arranged in an array on one side of the first support layer, and each of the independent cavities is filled with electrolyte. The second support layer is located on the side of the plurality of independent cavities away from the first support layer; Multiple sets of electrode assemblies, each set of electrode assemblies including a first electrode and a second electrode disposed at both ends of an independent cavity, the electrode assemblies being configured to form an electric field within the independent cavity, the electrolyte forming an electroosmotic flow under the action of the electric field, and the shape of the independent cavity being changed by the fluid pressure of the electrolyte to form a protruding structure protruding from the electroosmotic flow pump array layer. A stretchable screen is located on one side of the electroosmotic pump array layer, wherein the stretchable screen forms a predetermined shape in response to the protruding structure.

4. The display module for realizing stereoscopic haptic simulation according to claim 3, characterized in that, Also includes: The touch layer is located on the side of the stretchable screen away from the electroosmotic pump array layer; A touch feedback layer is located on the side of the touch layer away from the electroosmotic pump array layer. The touch feedback layer is configured to form a capacitance with the touching finger to generate tactile feedback and simulate the touch of an object on the touch screen. Preferably, the touch electrodes of the touch layer are reused as sensors and drivers for tactile feedback in the touch feedback layer; Preferably, the touch feedback layer further includes: A protective film layer is located on the side of the touch layer away from the electroosmotic pump array layer; Preferably, the protective film layer is a flexible protective layer, a hard coating protective layer, or a composite film layer of both.

5. The display module for realizing stereoscopic haptic simulation according to claim 3, characterized in that, The first electrode and the second electrode are thin-film electrodes; Preferably, the first support layer and the second support layer are flexible and stretchable insulating materials; Preferably, the materials of the first support layer and the second support layer include any one or a combination of the following: polyimide, polydimethylsiloxane, polyethylene terephthalate, polyurethane, epoxy resin, polyester film, and cyclic olefin polymer film. Preferably, the electrolyte includes any of the following: water-based electrolyte, ionic liquid, gel electrolyte, and porous media-enhanced electrolyte; Preferably, the display module further includes a substrate, wherein the substrate is a flexible support film layer, and the electroosmotic pump array is located within the substrate.

6. The display module for realizing stereoscopic haptic simulation according to claim 3, characterized in that, The protrusion parameters of the protrusion structure are related to the color information of the image displayed on the stretchable screen. Preferably, the color information includes hue, saturation, and brightness; Preferably, the boundary information of the plurality of protrusions is determined based on the hue; Preferably, the protrusion height is determined based on the saturation, and the protrusion height is positively correlated with the saturation. Preferably, the protrusion height of the protrusion is determined based on the brightness, and the protrusion height is positively correlated with the brightness.

7. The display module for realizing stereoscopic haptic simulation according to claim 3, characterized in that, The stretchable screen body includes: A light-emitting layer is located on one side of the electroosmotic pump array layer; The encapsulation layer is located on the side of the light-emitting layer away from the electroosmotic pump array layer; Preferably, the light-emitting layer includes an active driving layer, an anode layer, a hole transport layer, a pixel layer, an electron layer, and a cathode layer stacked sequentially, with the active driving layer located on the side of the light-emitting layer away from the electroosmotic pump array layer.

8. The display module for realizing stereoscopic haptic simulation according to claim 4, characterized in that, The display module also includes: The timing control module is configured to acquire the display image information of the stretchable screen and receive the coordinate information of the touch feedback layer, and generate a tactile trigger signal; An electroosmotic pump control module acquires the tactile trigger signal and is configured to independently control the voltage in each of the independent cavities in the electroosmotic pump array layer according to the tactile trigger signal, so as to cause local deformation of the electroosmotic pump array layer.

9. The display module for realizing stereoscopic haptic simulation according to claim 8, characterized in that, The display module also includes: The display driver module is configured to receive display image information output by the timing control module and convert it into an electrical signal, which is configured to guide the stretchable screen to display images.

10. A display device for simulating stereoscopic tactile sensation, characterized in that, Includes a display module that realizes stereoscopic haptic simulation as described in any one of claims 3 to 9.

Images