Force sensing method, electronic module capable of facilitating force sensing, and computing device

By combining the data of the touch sensor and fingerprint sensor, efficient force sensing in computing devices is achieved, solving the problems of increasing hardware complexity and cost in the prior art, and improving the user experience.

CN114077337BActive Publication Date: 2024-11-26NOVATEK MICROELECTRONICS CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110043740.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-08-18
Filing Date
2021-01-13
Publication Date
2024-11-26
Estimated Expiration
2041-01-13

AI Technical Summary

Technical Problem

When implementing force sensing on touch screens, the prior art needs to increase the volume of computing devices and hardware manufacturing costs, which cannot effectively reduce the complexity of hardware design.

Method used

By combining the information of the touch sensor and fingerprint sensor, using the touch sensing data and fingerprint sensing data, sensing information representing the relative stress of the object pressing is generated, and force sensing is achieved, avoiding the mechanical structure and electronic circuits of the panel module. Additional design.

Benefits of technology

It realizes efficient force sensing functions in computing devices, reduces hardware volume and manufacturing costs, and improves user interactive experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114077337B_ABST
    Figure CN114077337B_ABST
Patent Text Reader

Abstract

The present invention provides a force sensing method, an electronic module capable of facilitating force sensing, and a computing device. A method for force sensing using information from a display panel having a touch sensor and a fingerprint sensor. The method comprises: obtaining first sensing information representing the distribution of capacitance values ​​on a touch sensing area due to the pressing of an object on the display panel based on touch sensing data from a touch sensing circuit coupled to the touch sensor; obtaining second sensing information representing an effective pressing area due to the pressing of the object based on fingerprint sensing data from a fingerprint sensing circuit coupled to the fingerprint sensor; and generating third sensing information representing a force corresponding to the pressing of the object based on the first sensing information and the second sensing information. An electronic module capable of facilitating force sensing and a computing device capable of force sensing are also provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a technique for force sensing in a computing device, and more particularly to a method for force sensing, an electronic module capable of facilitating force sensing, and a computing device. Background Art

[0002] For computing devices, such as smart phones, tablet computers or other information processing devices, touch screens have become an essential component of computing devices for user interaction. Users can contact the touch screen with one or more fingers or a stylus to give input or control the computing device through single-touch or multi-touch gestures.

[0003] The user's demand for better touch effects on the touch screen of the computing device continues to increase. In particular, it is expected that force sensing can be implemented on the touch screen to provide information on the size of the force corresponding to the touch on the touch screen, so as to promote a better user experience for practical applications of user interaction. In order to meet this, the existing method is to set a dedicated force sensor, for example, under the LCD and the touch sensor and the backlight unit when implementing a panel module such as a liquid crystal module. This method requires the use of additional force sensors to redesign the mechanical structure and electronic circuit of the panel module, so it inevitably increases the volume of the computing device as well as the hardware and manufacturing costs. Summary of the invention

[0004] The object of the present invention is to provide a technique for force sensing in a computing device, wherein force sensing can be achieved using information from a touch sensor and a fingerprint sensor. Force sensing can be achieved through the contribution of fingerprint sensing data obtained from the signal of the fingerprint sensor and touch sensing data obtained from the signal of the touch sensor.

[0005] In order to at least achieve the above-mentioned purpose, the present invention provides a method for force sensing using information from a display panel having a touch sensor and a fingerprint sensor. The method comprises: obtaining first sensing information based on touch sensing data from a touch sensing circuit coupled to the touch sensor, the first sensing information representing the distribution of capacitance values ​​on a touch sensing area due to the pressing of an object on the display panel; obtaining second sensing information based on fingerprint sensing data from a fingerprint sensing circuit coupled to the fingerprint sensor, the second sensing information representing second sensing information of an effective pressing area due to the pressing of the object; and generating the third sensing information based on the first sensing information and the second sensing information, the third sensing information representing the force corresponding to the pressing of the object.

[0006] Optionally, the effective pressing area is within the touch sensing area and is smaller than the touch sensing area.

[0007] Optionally, the capacitance value distribution represents that multiple positions in the touch sensing area have respective capacitance values, and the capacitance values ​​can be obtained based on the touch sensing data, and the touch sensing data is generated due to deformation of the touch sensor caused by pressing of the object.

[0008] Optionally, the second sensing information representing the effective pressing area due to the pressing of the object is obtained by averaging the fingerprint sensing values ​​in each of the multiple blocks on the fingerprint sensing area based on the fingerprint sensing data.

[0009] Optionally, the step of obtaining the second sensing information includes: determining a touch-related area based on the first sensing information and a first threshold; and determining the second sensing information based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area.

[0010] Optionally, the touch-related area is determined, wherein each touch sensing value of the touch-related area exceeds or is equal to the first threshold.

[0011] Optionally, the second sensing information is determined, wherein each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold.

[0012] Optionally, the third sensing information is generated by performing a calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information, and the touch sensing values ​​corresponding to the effective pressing area have a lower contribution to the calculation.

[0013] Optionally, the calculation completely excludes any touch sensing value corresponding to the effective pressing area.

[0014] Optionally, the calculation does not completely exclude the touch sensing value corresponding to the effective pressing area.

[0015] Optionally, the third sensing information is generated by performing the calculation of a second weighted sum of multiple fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing value not corresponding to the effective pressing area dominates the result of the calculation.

[0016] In order to at least achieve the above-mentioned purpose, the present invention further provides an electronic module capable of facilitating force sensing, wherein the electronic module is used to be coupled to a display panel having a touch sensor and a fingerprint sensor. The electronic module includes a touch sensing circuit, a fingerprint sensing circuit, and at least one control unit. The touch sensing circuit is used to couple to the touch sensor and output touch sensing data. The fingerprint sensing circuit is used to couple to the touch sensor and output fingerprint sensing data. The at least one control unit is coupled to the touch sensing circuit and the fingerprint sensing circuit, and the at least one control unit is configured to perform multiple operations for force sensing, the operations including: based on the touch sensing data, obtaining first sensing information, the first sensing information representing the distribution of capacitance values ​​on the touch sensing area due to the pressing of an object on the display panel; based on the fingerprint sensing data, obtaining second sensing information, the second sensing information representing the effective pressing area due to the pressing of the object; and, based on the first sensing information and the second sensing information, generating third sensing information, the third sensing information representing the force corresponding to the pressing of the object.

[0017] Optionally, the effective pressing area is within the touch sensing area and is smaller than the touch sensing area.

[0018] Optionally, the capacitance value distribution represents that multiple positions in the touch sensing area have respective capacitance values, and the capacitance values ​​can be obtained based on the touch sensing data. The touch sensing data is generated by deformation of the touch sensor caused by the pressing of the object.

[0019] Optionally, the at least one control unit is configured to average the fingerprint sensing values ​​within each of the multiple blocks on the fingerprint sensing area based on the fingerprint sensing data so as to obtain second sensing information representing the effective pressing area generated due to the pressing of the object.

[0020] Optionally, the at least one control unit is configured to determine a touch-related area based on the first sensing information and a first threshold, and to determine the second sensing information based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area.

[0021] Optionally, the at least one control unit is configured to determine the touch-related area, wherein each touch sensing value in the touch-related area exceeds or is equal to the first threshold.

[0022] Optionally, the at least one control unit is configured to determine the second sensing information, wherein each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold.

[0023] Optionally, the at least one control unit performs calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information to generate the third sensing information, and reduces the contribution of the touch sensing values ​​corresponding to the effective pressing area to the calculation.

[0024] Optionally, the calculation completely excludes any touch sensing value corresponding to the effective pressing area.

[0025] Optionally, the calculation does not completely exclude the touch sensing value corresponding to the effective pressing area.

[0026] Optionally, the at least one control unit performs the calculation of a second weighted sum of a plurality of fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing values ​​not corresponding to the effective pressing area dominate the result of the calculation.

[0027] Optionally, the electronic module may be implemented in a single chip.

[0028] In order to at least achieve the above-mentioned purpose, the present invention further provides a computing device, comprising a display panel, a first device, a second device and a third device. The display panel is provided with a touch sensor and a fingerprint sensor. The first device is coupled to the touch sensor and is used to obtain first sensing information, wherein the first sensing information represents the distribution of capacitance values ​​on the touch sensing area due to the pressing of an object on the display panel. The second device, coupled to the fingerprint sensor, is used to obtain second sensing information, wherein the second sensing information represents the effective pressing area due to the pressing of the object. The third device, coupled to the first device and the second device, is used to generate third sensing information based on the first sensing information and the second sensing information, wherein the third sensing information represents the force corresponding to the pressing of the object. The computing device is capable of selectively performing at least one operation based on the third sensing information.

[0029] Optionally, the capacitance value distribution represents that a plurality of positions in the touch sensing area have respective capacitance values, which are generated due to deformation of the touch sensor caused by the pressing of the object.

[0030] Optionally, the second device is configured to average the fingerprint sensing values ​​in each of the multiple blocks on the fingerprint sensing area, so as to obtain second sensing information representing the effective pressing area caused by the pressing of the object.

[0031] Optionally, the second device determines a touch-related area based on the first sensing information and a first threshold; and the second device determines the second sensing information based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area.

[0032] Optionally, the second device determines the touch-related area, wherein each touch sensing value of the touch-related area exceeds or is equal to the first threshold.

[0033] Optionally, the second device determines the second sensing information, wherein each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold.

[0034] Optionally, the third device performs calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information to generate the third sensing information, and the touch sensing values ​​corresponding to the effective pressing area have a lower contribution to the calculation.

[0035] Optionally, the calculation completely excludes any touch sensing value corresponding to the effective pressing area, so as to generate the third sensing information.

[0036] Optionally, the calculation does not completely exclude the touch sensing value corresponding to the effective pressing area.

[0037] Optionally, the third device performs the calculation of a second weighted sum of multiple fingerprint sensing values ​​further based on the second sensing information to generate the third sensing information, wherein the touch sensing value not corresponding to the effective pressing area dominates the result of the calculation.

[0038] Optionally, the computing device further comprises: a single chip and a processing unit. The single chip comprises the first device, the second device and the third device. The processing unit is coupled between the display panel and the single chip, and is configured to selectively perform at least one operation based on the third sensing information.

[0039] Optionally, the computing device further includes a single chip, which includes the first device and the second device, wherein the third device is a processing unit coupled between the display panel and the single chip, and the processing unit is configured to selectively perform at least one operation based on the third sensing information.

[0040] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, this description and the attached drawings are only used to illustrate the present invention, and do not impose any limitation on the scope of rights of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 is a block diagram showing an electronic module capable of facilitating force sensing according to an embodiment of the present invention employed in a computing device;

[0043] Figure 2 is a block diagram illustrating an electronic module capable of facilitating force sensing according to another embodiment of the present invention;

[0044] Figure 3A is a block diagram illustrating implementation of force sensing in a computing device according to an embodiment of the present invention;

[0045] Figure 3B is a block diagram illustrating implementation of force sensing in a computing device according to another embodiment of the present invention;

[0046] Figure 4 is a flow chart illustrating a method for force sensing using information from a display panel having a touch sensor and a fingerprint sensor according to various embodiments;

[0047] Figure 5 is a cross-sectional schematic diagram showing a display panel with a touch sensor and a fingerprint sensor according to an embodiment;

[0048] Figure 6 It is shown as in Figure 5 A schematic diagram of an example of a distribution of capacitance changes on a touch sensing area due to a pressing of an object on the display panel as shown in FIG.

[0049] Figure 7 It is shown in a three-dimensional view Figure 6 Schematic diagram of the capacitance change distribution;

[0050] Fig. 8A is a schematic diagram showing capacitance change distribution in various cases where a large force, a medium force, and a small force are applied, projected onto a two-dimensional plane about a touch sensing value and an X-axis;

[0051] Figure 8B is a schematic diagram showing capacitance change distribution in various cases where a large force, a medium force, and a small force are applied, projected onto a two-dimensional plane about a touch sensing value and a Y axis;

[0052] Fig. 9 is a schematic diagram showing an example in which the second sensing information represents an effective pressing area obtained based on fingerprint sensing data;

[0053] Fig.10 is a schematic diagram showing an example of first sensing information and second sensing information corresponding to a finger pressing the display panel with a large force;

[0054] Fig.11 is a schematic diagram showing an example of first sensing information and second sensing information corresponding to a palm pressing the display panel with a small force;

[0055] Fig.12 is a schematic diagram showing an example of generating third sensing information representing a force corresponding to a press of a finger;

[0056] Fig.13 is a schematic diagram showing an example of generating third sensing information representing a force corresponding to a press of a palm;

[0057] Fig.14 is a schematic diagram showing another embodiment of generating the second sensing information;

[0058] Fig.15 is a schematic diagram showing an example of generating second sensing information representing an effective pressing area.

[0059]

Explanation of symbols

[0060] 1 Computing device

[0061] 5 Processing Unit

[0062] 9 Display Panel

[0063] 10 Electronic modules

[0064] 11 Touch sensor module

[0065] 12 Fingerprint sensor module

[0066] 19 Display driver module

[0067] 21 First Chip

[0068] 22 Second chip

[0069] 91 Display Module

[0070] 93 Touch Sensor

[0071] 95 Fingerprint sensor

[0072] 110 Touch sensing circuit

[0073] 112 Control unit

[0074] 120 Fingerprint sensing circuit

[0075] 122 Control Unit

[0076] 200 Cover glass

[0077] 210 LCD unit

[0078] 220 Deformation Layer

[0079] 230 Backlight unit

[0080] 300 touch sensing area

[0081] 301 images

[0082] 302 images

[0083] 310 Touch related area

[0084] 311 Part of the touch sensing value

[0085] 400 Fingerprint sensing area

[0086] 410 corresponding area

[0087] 411 Corresponding part

[0088] 10A Electronic Module

[0089] 10B Electronic Module

[0090] 10C Electronic Module

[0091] 12A Fingerprint Sensor Module

[0092] 301A Sub-image

[0093] 301B Blank Area

[0094] 302A Sub-image

[0095] 302B Blank Area

[0096] 91A Display Module

[0097] 93A Touch Sensor

[0098] 95A Fingerprint Sensor

[0099] 9A Display Panel

[0100] EA Effective Compression Area

[0101] EA1 Effective compression area

[0102] EA2 Effective compression area

[0103] FPR Image

[0104] HX Curve

[0105] HY Curve

[0106] LX Curve

[0107] LY Curve

[0108] MX Curve

[0109] MY Curve

[0110] Steps S10 to S30

[0111] Steps S210 and S220 DETAILED DESCRIPTION

[0112] In order to facilitate understanding of the objects, features, and effects of the present invention, embodiments and drawings for detailed description of the present invention are provided.

[0113] Various embodiments of techniques for force sensing in a computing device are provided below, where force sensing can be implemented using information from a display panel having a touch sensor and a fingerprint sensor in the computing device. Force sensing can be achieved through contributions from fingerprint sensing data from the fingerprint sensor and touch sensing data from the touch sensor.

[0114] For ease of explanation, hereinafter, an embodiment of an electronic module capable of facilitating force sensing and a computing device (or apparatus) using the electronic module are first introduced. Then, an embodiment of a method for performing force sensing using information from a display panel having a touch sensor and a fingerprint sensor in the computing device is provided.

[0115] Please refer to Figure 1 , which is a block diagram of an embodiment of the present invention, showing an electronic module 10 capable of facilitating force sensing, which may be employed in a computing device 1. Figure 1 As shown, the computing device 1 includes a processing unit 5, a display panel 9 and an electronic module 10. The electronic module 10 capable of facilitating force sensing can be used to couple between the display panel 9 and the processing unit 5. As will be explained later by way of example, the electronic module 10 can be configured to perform Figure 4The method for force sensing shown in the figure, wherein the electronic module 10 is capable of generating sensing information representing the force corresponding to the pressure of an object (such as one or more fingers, a stylus, etc.) on the display panel 9, thereby achieving force sensing by utilizing the contribution of the fingerprint sensing data and the touch sensing data.

[0116] based on Figure 1 The structure of the computing device 1 shown can be implemented in any electronic device, such as a smart phone, a tablet computer or any other information processing device, wherein, as appropriate, the computing device 1 may further include but is not limited to additional components such as memory, circuits for wireless or wired communication, image capture devices, etc.

[0117] The processing unit 5 can be configured to control the display panel 9 to provide a specific function. For example, the processing unit 5 executes an application under the operating system to control the display panel 9 through the electronic module 10, so that the computing device 1 can interact with the user through the display panel 9. The computing device 1 (e.g., a smart phone, a tablet, etc.) can be configured to perform one of a plurality of operations (e.g., user operations or functions in a game) associated with different values ​​of the force represented by the sensing information generated by the electronic module 10.

[0118] The display panel 9 includes a display module 91, such as a liquid crystal display (LCD) module, an organic light emitting diode (OLED) module or the like, and is provided with a touch sensor 93 and a fingerprint sensor 95. The display panel 9, for example, can be implemented by an in-cell type or on-cell type display panel integrated with a fingerprint sensor, wherein the display module 91, the touch sensor 93 and the fingerprint sensor 95 can be layered or integrated in any appropriate manner. The touch sensor 93 can be implemented by using a capacitive touch sensor 93 in the form of a touch sensing array. The fingerprint sensor 95, for example, can be implemented by an optical fingerprint sensor, a capacitive fingerprint sensor, an ultrasonic fingerprint sensor, or any device of a sensor for sensing a fingerprint signal. In some embodiments, the fingerprint sensor 95 can be implemented to detect at least a portion of the visible area of ​​the image displayed by the display module 91. In a preferred embodiment, the fingerprint sensor 95 is a full-screen fingerprint sensor 95, which can cover the same area, substantially used for the image display of the display module 91 and the touch sensing of the fingerprint sensor. Of course, the implementation of the present invention is not limited to the above examples.

[0119] The electronic module 10 capable of facilitating force sensing can be used to couple to the display panel 9 having the touch sensor 93 and the fingerprint sensor 95. Figure 1 In the embodiment, the electronic module 10 may include a touch sensing circuit 110, a fingerprint sensing module 12 and a control unit 112. In an embodiment, the control unit 112 and the touch sensing circuit 110 may be implemented or regarded as a touch sensing module 11. The electronic module 10 may be implemented as a chip. From the perspective of the processing unit 5, the electronic module 10 may be used as a "bridge" between the display panel 9 and the processing unit 5. The processing unit 5 may be configured to control the electronic module 10 to obtain output touch data and / or output fingerprint data.

[0120] The touch sensing circuit 110 may be used to couple to the touch sensor 93 and obtain touch sensing data. For example, the touch sensing circuit 110 may include a touch analog front end (AFE) circuit for converting an analog touch signal received from the touch sensor 93 into a corresponding digital touch signal, such as a set of raw touch data, the set of raw touch data being associated with and distributed over an area on the screen of the display panel 9 where at least one touch is detected. The touch sensing data may be further obtained based on the digital touch data, for example, by the control unit 112.

[0121] The fingerprint sensing module 12 may be used to couple to the fingerprint sensor 95 and obtain fingerprint sensing data. For example, the fingerprint sensing module 12 may be implemented to convert the fingerprint signal received from the fingerprint sensor 95 into corresponding digital fingerprint data, such as a set of raw fingerprint data, which is associated with and distributed over the area where the fingerprint (or part of the fingerprint) is detected on the screen of the display panel 9. The fingerprint sensing data may be further obtained based on the digital fingerprint data, for example, by the fingerprint sensing module 12 or the control unit 112.

[0122] The control unit 112 may be coupled to the touch sensing circuit 110 and the fingerprint sensing module 12. In one example, the control unit 112 may obtain the touch sensing data based on the digital touch data output by the touch sensing circuit 110.

[0123] Please refer to Figure 2 , another embodiment of an electronic module capable of facilitating force sensing is shown in block diagram form. Figure 2 As shown, based on Figure 1The electronic module 10A of the embodiment of the electronic module 10 in the embodiment includes a touch sensing module 11 and a fingerprint sensing module 12A. The fingerprint sensing module 12A may include a fingerprint sensing circuit 120 and a control unit 122 for fingerprint sensing.

[0124] For example, the fingerprint sensing circuit 120 may include a fingerprint analog front end (AFE) circuit for converting a fingerprint signal received from the fingerprint sensor 95 into corresponding digital fingerprint data, such as a set of raw fingerprint data, the set of raw fingerprint data being associated with and distributed over an area where a fingerprint (or a portion of a fingerprint) is detected. The fingerprint sensing data may be further obtained based on the digital fingerprint data, for example, by the control unit 122.

[0125] The control unit 122 , coupled to the fingerprint sensing circuit 120 and the touch sensing module 11 , can be utilized for fingerprint sensing, for example, to generate fingerprint sensing data based on the digital fingerprint data.

[0126] In the above examples, the touch sensing circuit 110 or the fingerprint sensing circuit 120 may include an analog front-end circuit implemented by circuit components such as a low noise amplifier and an analog-to-digital converter. In the above examples, the control units 112 and 122 may be implemented by using a processor, a microcontroller, or a programmable circuit such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In one example, the control units 112 and 122 may be implemented by using a single control unit. Of course, the implementation of the present invention is not limited to the above examples.

[0127] In one embodiment, Figure 1 or Figure 2 The electronic module 10 shown in FIG. 1 can be implemented in a single chip, thereby bringing many benefits of circuit integration to the computing device 1 . Figure 3A Shown based on Figure 1 or Figure 2 An example of a single-chip implementation of the configuration of the electronic module 10 is shown in FIG. Figure 1 or Figure 2 The electronic module 10B is implemented in a single chip form, including a touch sensing module 11 and a fingerprint sensing module 12 (or 12A). The electronic module 10B may further include a display driving module 19 for connecting to the display module 91 .

[0128] It should be noted that although Figure 1 or Figure 2 The electronic module 10 shown in can be implemented in a single chip. In other implementations, the electronic module 10 can be implemented using multiple chips according to design requirements. For example, in some embodiments, the touch sensing module and the fingerprint sensing module can be implemented as different chips. Figure 3B In the embodiment shown, the electronic module 10C (for example, according to Figure 1 or Figure 2 The electronic module of the present invention comprises a first chip 21 and a second chip 22. The first chip 21 comprises a touch sensing module 11, and the second chip 22 comprises a fingerprint sensing module 12 (or 12A). The first chip 21 may further comprise a display driving module 19, which is used to connect to the display module 91 to drive the display module 91. Of course, the implementation of the present invention is not limited to these examples.

[0129] An embodiment of a method for performing force sensing using information from a display panel of a computing device having a touch sensor and a fingerprint sensor will be described below. Based on the method, an electronic module (e.g., 10, 10A, 10B, or 10C) as described above can be configured to achieve force sensing corresponding to a pressure of an object on the screen of the display panel, thereby facilitating the implementation of practical applications of force sensing information obtained according to the method (e.g., an application or user interface operation using the force sensing information).

[0130] Please refer to Figure 4 , which is a method for force sensing using information from a display panel having a touch sensor and a fingerprint sensor of a computing device according to an embodiment shown in the form of a flow chart. The method for force sensing can be implemented in an electronic module (e.g., 10, 10A, 10B, or 10C) that is used to couple to a display panel, such as a display panel 9 having a touch sensor 93 and a fingerprint sensor 95. Figure 4 As shown, the method comprises the following steps, wherein only for the sake of convenience of explanation is cited Figure 1 (or Figure 2 ) of the electronic module 10 (or 10A), but the implementation of the method is not limited to this.

[0131] As indicated in step S10, first sensing information is obtained based on touch sensing data from the touch sensing circuit 110 coupled to the touch sensor 93, wherein the first sensing information represents the touch of an object (e.g., one or more fingers, such as in Figure 5 As shown in , the capacitance value distribution on the touch sensing area caused by pressing of a touch pen or other object) on the display panel 9 of the computing device 1.

[0132] As indicated in step S20, based on the fingerprint sensing data from the fingerprint sensing circuit 120 (or the fingerprint sensing module 12) coupled to the fingerprint sensor 95, second sensing information is obtained, wherein the second sensing information represents second sensing information of the effective pressing area due to the pressing of the object.

[0133] As indicated in step S30, third sensing information is generated based on the first sensing information and the second sensing information, and the third sensing information represents a force corresponding to the pressing of the object. For example, step S30 can be performed by the control unit 112 (or the fingerprint sensing module 12, or the control unit 122) and / or the processing unit 5 (for example, performed separately or together).

[0134] In some embodiments, the force calculated in step S30 is positively correlated with the area difference between the touch sensing area and the effective pressing area. In other words, a larger force can be represented by a larger area difference between the touch sensing area and the effective pressing area; and, conversely, a smaller force can be represented by a smaller area difference between the touch sensing area and the effective pressing area.

[0135] In such Figure 4 The method shown in can achieve force sensing through the steps S10-S30 because the force corresponding to the pressing of an object (e.g., one or more fingers, a stylus, etc.) on the display panel 9 is at least related to the distribution of capacitance values ​​on the touch area, as will be explained later in the present invention by way of example, and the effective pressing area caused by the pressing of the object can be used as a correction in the generation process of the third sensing information representing the force. The touch sensing area, for example, represents at least a portion or the entire area used for touch sensing and provided on the display panel 9.

[0136] The following examples illustrate the correlation between force and compression of an object.

[0137] First, when an object (such as one or more fingers, a stylus, etc.) is pressed, the deformation of the display panel 9 can cause the internal capacitance value to change with the area corresponding to the pressure of the object, and can be measured by the touch sensor 93.

[0138] Please refer to Figure 5 , which shows a cross-sectional view of a display panel 9A (an example of the display panel 9) having a touch sensor and a fingerprint sensor according to an embodiment. Figure 5As shown in , the display panel 9A includes a cover glass 200, a display module 91A, a touch sensor 93A, and a fingerprint sensor 95A. The display module 91A may include a liquid crystal display (LCD) unit 210 having a display unit array, a deformable layer 220 (e.g., an air gap), and a backlight unit 230. The touch sensor 93A (e.g., a capacitive touch sensor array) may be embedded in the LCD unit 210, and the fingerprint sensor 95A (e.g., an optical fingerprint sensor array) may be disposed below the LCD unit. Figure 5 As described in, when an object (such as one or more fingers, a stylus, etc.) presses the display panel 9A, a force is applied to the cover glass 200, and the cover glass 200 will deform downward, so that the distance between the touch sensor 93A and the backlight unit 230 is reduced, resulting in an increase in the capacitance between the touch sensor 93A and the backlight unit 230. Of course, the implementation of the present invention is not limited to the above examples. For example, the display panel 9 with a touch sensor and a fingerprint sensor can have any suitable architecture, such as a structure under the display (under-display), on the display (on-display) or in the display (in-display) for a fingerprint sensor, or a structure for an embedded touch sensor, an in-cell or on-cell type touch sensor.

[0139] Therefore, the deformation of the display panel 9A may cause a change in the internal capacitance value at the area corresponding to the pressure of the object, and the internal capacitance value may be measured by the touch sensor 93A. Figure 6 As shown in Figure 5 An example of the distribution of capacitance changes on the touch sensing area due to the pressure of an object is shown in FIG. Figure 6 As shown in , the touch sensing area 300 represents the area corresponding to the touch sensor 93 (e.g., a capacitive sensor array) of the display panel 9, wherein the touch sensing area 300 may, for example, cover substantially the same range as the screen area used for image display of the display panel 9. Figure 6In the figure, a plurality of small blocks are schematically shown in grayscale within the touch sensing area 300 to represent an array of touch sensing components of the touch sensor 93 corresponding to a touch sensing resolution (e.g., 16×28). When an object is pressed on the touch sensing area 300, for example, a 500 gram (g) standard weight contained in an insulating box is placed approximately in the middle of the touch sensing area 300, the corresponding capacitance change distribution on the touch sensing area 300 can be determined and represented by using touch sensing data, wherein the touch sensing data includes a plurality of touch sensing values ​​associated with positions on the touch sensing area 300. In addition, the touch sensing value is associated with the deformation of the touch sensor 93, wherein the larger the touch sensing value, the larger the deformation. Please refer to Figure 7 , shown in a three-dimensional view Figure 6 The capacitance change distribution of the touch position (x, y) is represented by the z-axis. Figure 7 As shown in FIG. 3 , the curved surface of the touch sensing value corresponding to the pressing of the object (e.g., a 500 g standard weight contained in an insulating box placed in the middle of the touch sensing area 300) represents a mountain-shaped distribution of capacitance changes. Figure 6 In the example, small blocks of gray scale are shown only to represent the changes of the sensing values, and they are not actually displayed on the display panel 9. Of course, the implementation of the present invention is not limited to these examples.

[0140] Therefore, the force applied to the display panel 9 due to the pressing of the object may cause a capacitance change distribution, which is represented by the touch sensing value and can be visualized as a mountain shape. In addition, it is found that if the applied force becomes larger, the capacitance change distribution range is wider. Please refer to Fig. 8A , curves HX, MX, and LX represent the average values ​​of the projections of the capacitance change distribution on the two-dimensional plane relative to the Z axis (i.e., the axis for touch sensing value) and the X axis, respectively. Figure 6 Under the same conditions as the above example of the touch sensor 93 corresponding to the touch sensing resolution shown in , a large force (for example, using a 2kg standard weight), a medium force (for example, using a 1kg standard weight) and a small force (for example, using a 100g standard weight) are applied respectively. Please refer to Figure 8B The curves HY, MY, and LY represent the Y-direction average values ​​of the projections of the corresponding capacitance changes on the two-dimensional plane relative to the Z axis (i.e., the axis for touch sensing values) and the Y axis when a large force, a medium force, and a small force are applied, respectively. Fig. 8A and 8B It can be seen that if the applied force becomes larger, the range of the capacitance change distribution (the average value in the X direction or the Y direction) will become wider.

[0141] Various embodiments for implementing steps S10 , S20 , and S30 are provided below.

[0142] In an embodiment, in step S10, the capacitance value distribution represents that multiple positions within the touch sensing area (e.g., 300) have separate capacitance values, and the capacitance values ​​are obtained based on touch sensing data generated by deformation of the touch sensor (e.g., 93 or 93A) due to pressure from the object (e.g., one or more fingers, a stylus, etc.).

[0143] In some embodiments, in step S20, based on the fingerprint sensing data, second sensing information is obtained by averaging the fingerprint sensing values ​​in each block of multiple blocks on the fingerprint sensing area to obtain second sensing information representing the effective pressing area generated by the pressing of the object.

[0144] Please refer to Fig. 9 , which shows an example of obtaining second sensing information representing the effective pressing area based on fingerprint sensing data. Fig. 9 As shown in , the fingerprint sensing area 400 may, for example, cover substantially the same range as the screen area of ​​the display panel 9, representing an area corresponding to the fingerprint sensor 95 of the display panel 9. Fig. 9 In the figure, a plurality of small blocks are schematically shown in the fingerprint sensing area 400 to represent an array of fingerprint sensing components of the touch sensor 93 corresponding to a fingerprint sensing resolution, and the fingerprint sensing resolution may be greater than or equal to the touch sensing resolution. In addition, each block in the fingerprint sensing area 400 may be configured to represent an M×N number of fingerprint sensing components for acquiring an image (e.g., a grayscale image), where M and N are integers greater than or equal to two. The fingerprint sensing values ​​corresponding to each of the plurality of blocks on the fingerprint sensing area 400 may be averaged to obtain a corresponding representative value for the block (e.g., a normal arithmetic mean, a weighted arithmetic mean, etc.). Fig. 9 In the example, some blocks shown in black represent the image FPR of an object (e.g., a fingerprint) acquired by the fingerprint sensing components corresponding to these blocks. Therefore, these blocks can be used to determine the effective pressing area EA corresponding to the image of the object touching the display panel 9. In addition, it should be noted that although Fig. 9 The fingerprint image is taken as an example, but the acquired image may be one or more fingers, or a stylus or other objects.

[0145] based on Figure 4 The method can use the effective pressing area (for example, in Fig. 9The contribution of EA in (A) is used to achieve force sensing, the effective pressing area is obtained from the fingerprint sensing data, and can be utilized in generating the third sensing information.

[0146] In this regard, one concern is that similar capacitance value distributions (e.g. Fig.10 and Fig.11 ) may be caused by applying different forces to different objects, which makes it challenging to implement force sensing by using a distribution of capacitance values. Figure 4 The method utilizes second sensing information representing an effective pressing area to identify a touch position corresponding to the pressing of the object, thereby facilitating force sensing and overcoming the problem.

[0147] Fig.10 and Fig.11 Explain the above problem. Fig.10 , shows an example of first sensing information and second sensing information corresponding to a finger pressing the display panel with a large force. Fig.10 As shown on the left side of FIG, the capacitance value distribution on the touch sensing area 300 due to the heavy pressure of an object (e.g., a finger) on the display panel 9 is represented by an image 301, and represents first sensing information, wherein the first sensing information may include a touch sensing value associated with a corresponding position (e.g., the coordinates of a block for touch sensing). Fig.10 As shown on the right side of , the effective pressing area due to the heavy pressure of the object (e.g., finger) on the fingerprint sensing area 400 on the display panel 9 is represented by EA1 and represents the second sensing information, wherein the second sensing information may include a fingerprint sensing value associated with a corresponding position (e.g., the X, Y coordinates of a block or fingerprint sensing component for fingerprint sensing). Fig.10 ,exist Fig.11 An example of first sensing information and second sensing information corresponding to a palm touching the display panel with a small force is shown in FIG. Fig.11 As shown on the left side of FIG. 3 , the capacitance distribution on the touch sensing area 300 caused by a light pressure of an object (eg, a palm) on the display panel 9 is represented by an image 302 and represents the first sensing information. Fig.11 As shown on the right side of FIG. 1 , the effective pressing area due to the light pressure of the object (eg, palm) on the fingerprint sensing area 400 on the display panel 9 is represented by EA2 and represents the second sensing information.

[0148] and Fig.10 In comparison, the capacitance value distributions represented by images 301 and 302 (eg, Fig.10 and Fig.11) is similar, but the force associated with the heavy pressure of the finger is greater than the force associated with the light pressure of the palm. Fig.10 or Fig.11 As shown in FIG, the effective pressing area EA1 of heavy pressing (pressing with greater force) is smaller than the effective pressing area EA2 of light pressing (pressing with less force), wherein the effective pressing area EA1 or EA2 is within the touch sensing area 300 and is smaller than the touch sensing area 300. Figure 1 In the method, the second sensing information representing the effective pressing area can be used to identify the contact position corresponding to the pressing of the object on the touch sensing area, and the first sensing information representing the force information can be adopted or regarded as a correction to the above-mentioned problem in the generation of the third sensing information.

[0149] Therefore, in some embodiments, the force calculation in step S30 can be set to have a positive correlation with the area difference between the touch sensing area and the effective pressing area. In other words, a larger force can be represented by a larger area difference between the touch sensing area and the effective pressing area, such as Fig.10 As shown exemplarily in Fig.11 As exemplarily shown in FIG. 1 , a smaller force can be represented by a smaller area difference between the touch sensing area and the effective pressing area.

[0150] With regard to the above viewpoint, for example, the third sensing information may be generated based on the difference value (DV), as conceptually expressed by the following equation:

[0151] DV = T_SA - FP_EA (Equation 1)

[0152] Wherein T_SA represents the value representing the touch sensing area, and FP_EA represents the value representing the effective pressing area. In an example, T_SA may be defined as a first weighted sum of a plurality of touch sensing values ​​(T_SAi) of the first sensing information, and FP_EA may be defined as a second weighted sum of a plurality of fingerprint sensing values ​​(FP_EAj) of the second sensing information. By using these definitions, the difference value (DV) of Equation 1 may be expressed as:

[0153]

[0154] The weight a for the first weight sum can be set in an appropriate manner: i and the weight b for the second weight sum j , for example, weight a iis a positive value, so that the first weight sum contributes more to the difference DV than the second weight sum (e.g., T_SA>FP_EA), thereby satisfying that the touch sensing value not corresponding to the effective press area dominates the result of the calculation for the third sensing information. In another example, T_SA (e.g., the first weight sum) can be conceptually equal to the sum of the weight sum of the touch sensing values ​​not corresponding to the effective press area (represented by T_XEA) and the weight sum of the touch sensing values ​​corresponding to the effective press area (represented by T_EA), and Equation 2 can be expressed as follows.

[0155] DV = T_XEA + T_EA - FP_EA (Equation 3)

[0156] For example, in equation 3, FP_EA = ∑ j b j FP_EA j , where the weight b for the second weight sum can be set in an appropriate manner j , for example, weight b j is a non-negative value, so that the first weight sum contributes more to the difference DV than the second weight sum. For example, the weight b can be set j , so that T_XEA>T_EA-FP_EA; or even T_EA is substantially equal to FP_EA. Of course, the implementation of the present invention is not limited to the example.

[0157] The following embodiments are about the generation of third sensing information representing force in step S30 .

[0158] In some embodiments of step S30, the third sensing information is generated by performing a calculation of a first weighted sum of a plurality of touch sensing values ​​that do not correspond to the effective pressing area (e.g., an area outside the area EA1 or EA2) based at least on the first sensing information, and the touch sensing values ​​corresponding to the effective pressing area (e.g., EA1 or EA2) contribute less to the calculation. In this way, the calculation can be determined or configured to be more accurate, wherein Fig.10 and 11 As shown, with respect to the above-mentioned problem, the effective pressing area can be used as a correction in the calculation.

[0159] Furthermore, in the embodiment of step S30 , in order to reduce the contribution of the touch sensing value corresponding to the effective pressing area (eg, EA1 or EA2 ) to the calculation, various suitable methods may be provided.

[0160] In an embodiment, for the contribution of the touch sensing value corresponding to the effective pressing area (e.g., EA1 or EA2) to the calculation in step S30, any touch sensing value corresponding to the effective pressing area may be completely excluded from the calculation. Fig.12 , showing an example of generation of third sensing information corresponding to pressing. Fig.10 In the case of Fig.12 In the example shown, it is assumed that the user presses one finger on the screen of the display panel 9. Fig.10 In the case of generating the third sensing information, a plurality of touch sensing values ​​not corresponding to the effective pressing area based on at least the first sensing information may be performed (such as in Fig.12 The first weight sum is generated by calculating the first weight sum of the sub-image 301A represented by the sub-image 301A, wherein the sub-image 301A is generated by excluding (e.g., completely or partially) the touch sensing values ​​corresponding to the effective pressing area (e.g., EA1) (e.g., Fig.12 The blank area 301B surrounded by the sub-image 301A is shown in FIG. Fig.11 In the case where the user presses a palm on the screen of the display panel 9, the generation of the third sensing information may be performed based on at least a plurality of touch sensing values ​​not corresponding to the effective pressing area of ​​the first sensing information (such as Fig.13 The sub-image 302A is represented by the sub-image 302A in FIG. 1 , wherein the sub-image 302A is generated by excluding (e.g., in whole or in part) the touch sensing values ​​corresponding to the effective pressing area (e.g., EA2) (e.g., in FIG. 1 ). Fig.13 302B surrounded by the sub-image 302A).

[0161] Further, as in Fig.12 and Fig.13 As shown in FIG. 3 , the sub-image 301A corresponding to a larger force (e.g., through a finger) is larger than the sub-image 302A corresponding to a smaller force (e.g., through a palm), so that it can be determined that the force represented by the third sensing information corresponding to the sub-image 301A is larger than the force represented by the third sensing information corresponding to the sub-image 302A, thereby obtaining the conclusion that Fig.12 and Fig.13 The two examples provide reasonable and consistent predictions of force information.

[0162] In another embodiment, for the contribution of the touch sensing values ​​corresponding to the effective pressing area (e.g., EA1 or EA2) to the calculation in step S30, the touch sensing values ​​corresponding to the effective pressing area are not completely excluded in the calculation. For example, when the calculation does not completely exclude the touch sensing values ​​corresponding to the effective pressing area, the calculation for generating the third sensing information may be based on at least the first sensing information that is not related to the effective pressing area (e.g., as in Fig.12 represented by sub-image 301A in, or Fig.13 The method is performed by summing a first weight of a plurality of touch sensing values ​​corresponding to the sub-image 302A in the image and based on a portion or some of the touch sensing values ​​corresponding to the effective pressing area (eg, EA1 or EA2).

[0163] In an embodiment, the third sensing information can be generated by performing the calculation of the second weighted sum of multiple fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing value not corresponding to the effective pressing area dominates the result of the calculation. In this way, more accurate force information represented by the third sensing information can be obtained.

[0164] In this regard, for example, the third sensing information may be generated based on a sum value (SV), as expressed by the following equation:

[0165]

[0166] Among them, the plurality of touch sensing values ​​(T_XEA) of the first sensing information not corresponding to the effective pressing area (eg, EA1) i ) can be obtained by ∑ i a i ·T_XEA i Indicates, and a plurality of fingerprint sensing values ​​(FP_EA) of the second sensing information j ) can be represented by ∑ j b j FP_EA j For the weight a of the first weight sum i can be used in a suitable manner (for example, with the weight b of the second weight sum j , a larger value or a positive value than the first weight sum) is set so that the contribution of the first weight sum to the sum value SV is greater than that of the second weight sum, thereby satisfying that the touch sensing value not corresponding to the effective pressing area dominates the result of the calculation. i Compared to the weight b for the second weight sum jIt can be set by a smaller value or a negative value so that the contribution of the second weight sum to the total value SV is smaller.

[0167] Of course, the implementation of the present invention is not limited to the above examples. In some embodiments, the third sensing information can be generated based on the sum value SV as expressed by another equation below:

[0168] SV=f(T_XEAi)+g(FP_EA j ), (Equation 5)

[0169] where f() and g() represent any suitable functions. For example, f() can be chosen so that f(T_XEA i ) contributes more to the total value SV than g(FP_EA j ), thereby satisfying the touch sensing value that does not correspond to the effective pressing area and affects the calculation.

[0170] In some embodiments, the third sensing information representing the force information can be determined, for example, by a calculation derived by a statistical (e.g., linear or nonlinear) method or a machine learning (e.g., a neural network, a convolutional neural network, etc.) method, as appropriate, based on the touch sensing value of the first sensing information and the fingerprint sensing value of the second sensing information as input values, and the corresponding force information (e.g., represented by a numerical value) as an output value. For example, the calculation can be implemented by obtaining a trained module by machine learning.

[0171] In actual implementation, any of the above embodiments regarding the calculation in step S30 can be implemented by table lookup when appropriate to simplify the calculation complexity of the electronic module (eg, 10, 10A, 10B or 10C).

[0172] In some embodiments of computing devices using electronic modules as exemplified in the present invention, additional force corrections may be implemented to reduce the effect of gravity on the determination of force information due to mechanical bending or batch production variations of the computing device. In this regard, additional force corrections may be performed after assembly of the computing device to achieve first-order or multi-order force information determination, even information in grams, for example, by additional table lookups or calculations.

[0173] In some embodiments, the effective pressing area represented by the second sensing information can be obtained in an effective manner by using the touch sensing value of the touch sensing area. Fig.14 As shown in , step S20 of obtaining the second sensing information may include steps S210 and S220.

[0174] As shown in step S210 , a touch-related area is determined based on the first sensing information and a first threshold.

[0175] As shown in step S220, the second sensing information is determined based on the touch-related area, the fingerprint sensing data and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or larger than the touch-related area.

[0176] In an embodiment of step S210, the touch-related area is determined wherein each touch sensing value of the touch-related area exceeds or is equal to the first threshold. For example, the control unit (e.g., 112 or 122) may be configured to check whether the touch sensing value described by the first sensing information in the touch sensing area (e.g., 300) exceeds or is equal to the first threshold, and the control unit (e.g., 112 or 122) may be configured to determine the touch sensing value exceeding or equal to the first threshold as the touch-related area (e.g., as in Fig.13 Thus, the touch-related area (eg, 310) may be smaller than the touch-sensing area (eg, 300) in the data block, thereby reducing the amount of data required in subsequent processing (eg, step S30).

[0177] In an embodiment of step S220, the second sensing information is determined in which each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold. Fig.13 For example, the control unit (e.g., 112 or 122) may be configured to determine a corresponding area (e.g., 410) in the fingerprint sensing area 400 by using the touch-related area 310. The fingerprint sensing module (e.g., 12) may be configured to perform a fingerprint scan (i.e., an image scan) on an object (e.g., a finger) relative to the corresponding area (e.g., 410), and check whether the fingerprint sensing value described by the fingerprint sensing data in the corresponding area (e.g., 410) exceeds or is equal to the second threshold. The control unit (e.g., 112 or 122) may be configured to determine a fingerprint sensing value that exceeds or is equal to the second threshold as the valid touch area (e.g., in Fig.13 EA1 shown in Figure 1). From Fig.13 It can be seen that the first threshold can be set to a suitable value so that the touch-related area 310 is close to the actual area touched by the object (e.g., finger) to make the corresponding area (e.g., 410) smaller and facilitate the fingerprint sensing module (e.g., 12) to determine the effective touch area (e.g., EA1) in a more effective or efficient manner.

[0178] In addition, the second sensing information representing the effective pressing area (eg, EA1) can be used to identify a possible touch position corresponding to the pressing of the object (eg, finger). Fig.15 For example, a portion 311 of the touch sensing value exceeding the first threshold has a corresponding portion 411 relative to the fingerprint sensing area 400, and the corresponding portion 411 is not within the effective pressing area EA1, which means that the portion 311 of the touch sensing value is caused by deformation due to force. Therefore, the portion 311 of the touch sensing value should be used to determine the force information.

[0179] The example of step S20 and one or more of the above-mentioned embodiments may be utilized in the example of step S30 or any embodiment for determining the force information.

[0180] Based on Figure 1 In the actual application scenario of a computing device 1 such as a smart phone or a tablet computer, the computing device 1 can be based on the Figure 4 The third sensing information is generated by a method to determine force information corresponding to the pressure of an object (e.g., one or more fingers of a user or a stylus). For example, the computing device 1 (e.g., a smart phone, a tablet computer, etc.) can be configured to perform one of the operations (e.g., user operations or functions in a game) associated with the generation of the third sensing information by the electronic module 10, the third sensing information representing different values ​​of force. For example, in a game application with gun shooting, a user can focus on a target and shoot bullets on the screen by touching the screen with a greater force, or focus on a target by touching the screen only with a smaller force. In another example, in a piano playing application that displays a piano keyboard, a user can play music with a touch-sensitive effect; that is, the greater the force with which the user strikes the key, the louder the note associated with the key will be played. Of course, the implementation of the present invention is not limited to these examples.

[0181] In an embodiment, fingerprint sensing is performed after the user has completed identity identification (i.e., the user has been identified), or even possibly independently of the user's identity. In other words, the fingerprint sensing data can be used in contribution to the generation of force information. For example, the fingerprint sensing data obtained in step S20 can be used in step S30 to implement force sensing internally in the electronic module (e.g., 10, 10A, 10B or 10C), and the processing unit does not require the electronic module (e.g., 10, 10A, 10B or 10C) to provide output fingerprint data for user identity identification. This can occur after the computing device 1 has been unlocked so that the user is authorized to operate the computing device 1.

[0182] exist Figure 5In some embodiments of the method, step S20 may be configured to be performed only when at least one criterion is met. In one embodiment, the method further includes requesting the fingerprint sensing data from the fingerprint sensing module 12 when the touch sensing data represents a touch event. For example, when the touch sensing data represents a touch event, the control unit 112 may send a request for the fingerprint sensing data to the fingerprint sensing module 12. On the contrary, when the touch sensing data does not represent a touch event, the control unit 112 does not send the request to the fingerprint sensing module 12. In this way, the fingerprint sensing data can be used to enhance the generation of the force information more effectively or more efficiently. In some embodiments, the method further includes determining touch sensing data representing the touch event when the touch sensing data exceeds a threshold. For example, assuming that the touch sensing data represents a plurality of touch sensing values ​​associated with a position in the area where the touch is detected, the touch sensing data may be represented by an average value of the touch sensing values, and when the average value of the touch sensing values ​​exceeds a corresponding threshold, the control unit 112 may determine the touch sensing data representing the touch event. In another example, when one or more touch sensing values ​​exceed a threshold, the control unit 112 may determine touch sensing data representing a touch event.

[0183] In this way, based on Figure 1 In the actual application scenario of a computing device 1 such as a smart phone, the Fig. 9 By using the process shown in , force information associated with multiple touch points by a user (or multiple users) can be accurately determined based on the touch sensing data and the fingerprint sensing data.

[0184] Further, the electronic module (eg, in a single chip form) capable of facilitating force sensing as described above may be integrated with a display driver module. Figure 3A or 3B, the electronic module can be based on Figure 1 or Figure 2And further comprises a display driver module 19, the display driver module 19 is coupled to the touch sensing module 11 and the fingerprint sensing module 12 inside the electronic module; and the display driver module 19 is used to couple to the display module 91 of the display panel 9, so as to drive the display module 91 to display an image. In an embodiment of the display panel 9 with a fingerprint sensor 95, the fingerprint sensor 95 is an optical fingerprint scanner. For example, when a touch event is detected by the touch sensing module 11, the touch sensing module 11 can notify the display driver module 19 of the area where the touch event occurs. In some embodiments, the display driver module 19 can drive the display module 19 to emit light or an image pattern for exposure of the area. At the same time, the touch sensing module 11 can notify the fingerprint sensing module 12 to start the fingerprint sensing component (for example, an optical sensing component) corresponding to the area for fingerprint detection, so as to obtain fingerprint sensing data. The fingerprint sensing data can be based on the above-mentioned embodiment as shown in one of the embodiments. Figure 4 The method is utilized inside the electronic module. Of course, the implementation of the present invention is not limited to the above examples. It should also be noted that different implementations may integrate or separate different modules into one or more circuits. For example, the display driver module and the touch sensing module may be integrated into a circuit, and the fingerprint sensing module may be implemented as another circuit, which may be further completely or partially separated or integrated from the circuits of the display driver module and the touch sensing module.

[0185] In some embodiments, a computing device (or computing apparatus) is provided, which includes a display panel, a first device, a second device, and a third device. The display panel is provided with a touch sensor and a fingerprint sensor (for example, as in Figure 1 ). The first device, coupled to the touch sensor, is used to obtain the first sensing information, the first sensing information representing the distribution of capacitance values ​​on the touch sensing area caused by the pressing of the object on the display panel. The second device, coupled to the fingerprint sensor, is used to obtain the second sensing information, the second sensing information representing the effective pressing area caused by the pressing of the object. The third device, coupled to the first device and the second device, is used to generate third sensing information based on the first sensing information and the second sensing information, the third sensing information representing the force corresponding to the pressing of the object. The computing device (e.g., 1) is capable of selectively performing at least one operation based on the third sensing information.

[0186] In an embodiment of the computing device, the capacitance value distribution (for example, as in Figure 6 , 7, 8A or 8B) represents that multiple locations within the touch sensing area have respective capacitance values, which are generated due to the deformation of the touch sensor caused by the pressing of the object.

[0187] In an embodiment of the computing device, the second device is configured to average the fingerprint sensing values ​​in each block of a plurality of blocks on the fingerprint sensing area to obtain second sensing information, wherein the second sensing information represents an effective pressing area caused by pressing of the object.

[0188] In an embodiment of the computing device, the second device determines a touch-related area based on the first sensing information and a first threshold; and the second device determines the second sensing information based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area (for example, as in Fig.15 as shown in the relevant embodiments).

[0189] In an embodiment of the computing device, the second device determines the touch-related area, wherein each touch sensing value of the touch-related area exceeds or is equal to the first threshold.

[0190] In an embodiment of the computing device, the second device determines the second sensing information, wherein each fingerprint sensing value in the effective pressing area exceeds or is equal to the second threshold.

[0191] In some embodiments of the computing device, the third device (e.g., as in Figure 1 or Figure 2 112 or 122) shown in , performs calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information, and reduces the contribution of the touch sensing value corresponding to the effective pressing area in the calculation.

[0192] In an embodiment of the computing device, the calculation completely excludes any touch sensing value corresponding to the effective pressing area, so as to generate the third sensing information (for example, Fig.12 or Fig.13 as shown in the relevant embodiments).

[0193] In an embodiment of the computing device, the calculation does not completely exclude the touch sensing value corresponding to the effective pressing area (for example, as shown in the embodiment related to step S30 ).

[0194] In an embodiment of the computing device, the third device generates third sensing information by performing the calculation of a second weighted sum of multiple fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing value not corresponding to the effective pressing area dominates the result of the calculation (for example, as shown in the embodiment related to step S30).

[0195] In an embodiment of the computing device, the computing device further comprises: a single chip and a processing unit (e.g., as in Figure 1 5 shown in ). The single chip (for example, as shown in Figure 3A 10B) includes the first device, the second device and the third device. The processing unit (as shown in Figure 1 5), coupled to the display panel (for example, as shown in Figure 1 9) and the single chip (eg, as shown in Figure 3A 10B) shown in , the processing unit is configured to selectively perform at least one operation among a plurality of operations based on the third sensing information.

[0196] In an embodiment of the computing device, the computing device further comprises a single chip (e.g., Figure 1 or 2), the single chip includes the first device and the second device, wherein the third device is a processing unit (e.g., 5) coupled between the display panel (e.g., 9) and the single chip, and the processing unit is configured to selectively perform at least one of a plurality of operations based on the third sensing information.

[0197] In this embodiment, it may not be necessary to redesign the mechanical structure and electronic circuit of the panel module to include an additional force sensor, thereby reducing the volume and manufacturing cost of the computing device and hardware.

[0198] Although the present invention has been described through specific embodiments, those skilled in the art may make various modifications, combinations and changes thereto without departing from the scope and spirit of the present invention as set forth in the claims.

[0199] The above descriptions are merely embodiments of the present invention and are not intended to limit the protection scope of the present invention.

Claims

1. A method for force sensing using information from a display panel having a touch sensor and a fingerprint sensor, characterized in that, The method comprises: Based on touch sensing data from a touch sensing circuit coupled to the touch sensor, first sensing information is obtained, the first sensing information representing a capacitance value distribution on a touch sensing area due to pressing of an object on the display panel, and based on fingerprint sensing data from a fingerprint sensing circuit coupled to the fingerprint sensor, second sensing information is obtained, the second sensing information representing an effective pressing area due to pressing of the object; and generating third sensing information based on the first sensing information and the second sensing information, wherein the third sensing information represents a force corresponding to the pressing of the object, The third sensing information is generated by performing a calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information, and the touch sensing values ​​corresponding to the effective pressing area have a lower contribution to the calculation.

2. The method according to claim 1, characterized in that The force is positively correlated with the area difference between the touch sensing area and the effective pressing area.

3. The method according to claim 1, characterized in that The effective pressing area is within the touch sensing area and is smaller than the touch sensing area.

4. The method according to claim 1, characterized in that: The capacitance value distribution represents that a plurality of positions in the touch sensing area have respective capacitance values, and the capacitance values ​​can be obtained based on the touch sensing data, and the touch sensing data is generated by deformation of the touch sensor due to the pressing of the object.

5. The method according to claim 1, characterized in that The second sensing information representing the effective pressing area due to the pressing of the object is obtained by averaging the fingerprint sensing values ​​in each of the plurality of blocks on the fingerprint sensing area based on the fingerprint sensing data.

6. The method according to claim 1, characterized in that Obtaining the second sensing information includes: determining a touch-related area based on the first sensing information and a first threshold; and The second sensing information is determined based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area.

7. The method according to claim 6, characterized in that The touch-related area is determined, wherein each touch sensing value of the touch-related area exceeds or is equal to the first threshold.

8. The method according to claim 7, characterized in that The second sensing information is determined, wherein each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold.

9. The method according to claim 1, characterized in that: The calculation completely excludes any touch sensing values ​​corresponding to the effective pressing area.

10. The method according to claim 1, characterized in that The calculation does not completely exclude the touch sensing value corresponding to the effective pressing area.

11. The method according to claim 10, characterized in that The third sensing information is generated by performing the calculation of a second weighted sum of a plurality of fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing values ​​not corresponding to the effective pressing area dominate the result of the calculation.

12. An electronic module capable of facilitating force sensing, the electronic module being used to couple to a display panel having a touch sensor and a fingerprint sensor, characterized in that: The electronic module includes: a touch sensing circuit for coupling to the touch sensor and outputting touch sensing data; a fingerprint sensing circuit, configured to be coupled to the touch sensor and output fingerprint sensing data; and at least one control unit, coupled to the touch sensing circuit and the fingerprint sensing circuit, the at least one control unit being configured to perform a plurality of operations for force sensing, the operations comprising: obtaining first sensing information based on the touch sensing data, the first sensing information representing a distribution of capacitance values ​​on a touch sensing area due to a press of an object on the display panel; obtaining second sensing information based on the fingerprint sensing data, the second sensing information representing an effective press area due to a press of the object; and generating third sensing information based on the first sensing information and the second sensing information, wherein the third sensing information represents a force corresponding to the pressing of the object, The at least one control unit performs calculation of a first weighted sum of multiple touch sensing values ​​that do not correspond to the effective pressing area based at least on the first sensing information to generate the third sensing information, and the touch sensing values ​​corresponding to the effective pressing area have a lower contribution to the calculation.

13. The electronic module according to claim 12, characterized in that: The force is positively correlated with the area difference between the touch sensing area and the effective pressing area.

14. The electronic module according to claim 12, characterized in that: The effective pressing area is within the touch sensing area and is smaller than the touch sensing area.

15. The electronic module according to claim 12, characterized in that: The capacitance value distribution represents that a plurality of positions in the touch sensing area have respective capacitance values, and the capacitance values ​​can be obtained based on the touch sensing data, and the touch sensing data is generated by deformation of the touch sensor due to the pressing of the object.

16. The electronic module according to claim 12, characterized in that: The at least one control unit is configured to average the fingerprint sensing values ​​in each of the plurality of blocks on the fingerprint sensing area based on the fingerprint sensing data to obtain second sensing information representing an effective pressing area generated due to pressing of the object.

17. The electronic module according to claim 12, characterized in that: The at least one control unit is configured to determine a touch-related area based on the first sensing information and a first threshold, and to determine the second sensing information based on the touch-related area, the fingerprint sensing data, and a second threshold, wherein the effective pressing area represented in the second sensing information is equal to or smaller than the touch-related area.

18. The electronic module according to claim 17, characterized in that: The at least one control unit is configured to determine the touch-related area, wherein each touch sensing value in the touch-related area exceeds or is equal to the first threshold.

19. The electronic module according to claim 18, characterized in that The at least one control unit is configured to determine the second sensing information, wherein each fingerprint sensing value of the effective pressing area exceeds or is equal to the second threshold.

20. The electronic module according to claim 12, characterized in that: The calculation completely excludes any touch sensing values ​​corresponding to the effective pressing area.

21. The electronic module according to claim 12, characterized in that: The calculation does not completely exclude the touch sensing value corresponding to the effective pressing area.

22. The electronic module according to claim 21, characterized in that The at least one control unit performs the calculation of a second weighted sum of a plurality of fingerprint sensing values ​​further based on the second sensing information, wherein the touch sensing values ​​not corresponding to the effective pressing area dominate the result of the calculation.

23. The electronic module according to claim 12, characterized in that: The electronic module is implemented in a single chip.

24. A computing device, characterized in that: The computing device comprises: Display panel with touch sensor and fingerprint sensor; A first device, coupled to the touch sensor, for obtaining first sensing information, wherein the first sensing information represents a distribution of capacitance values ​​on a touch sensing area caused by a pressing of an object on the display panel; a second device, coupled to the fingerprint sensor, for obtaining second sensing information, wherein the second sensing information represents an effective pressing area caused by the pressing of the object; and a third device, coupled to the first device and the second device, for generating third sensing information based on the first sensing information and the second sensing information, wherein the third sensing information represents a force corresponding to the pressing of the object; wherein the computing device is capable of selectively performing at least one operation based on the third sensing information, The first sensing information includes a plurality of touch sensing values, and a portion of the touch sensing values ​​corresponds to the effective pressing area and the portion is completely excluded from calculation to generate the third sensing information.

25. The computing device according to claim 24, characterized in that The force is positively correlated with the area difference between the touch sensing area and the effective pressing area.

26. The computing device according to claim 24, characterized in that The first sensing information includes a plurality of touch sensing values, and a portion of the touch sensing values ​​corresponds to the effective pressing area and the calculation does not completely exclude the portion.

27. The computing device according to claim 24, characterized in that The computing device further comprises: a single chip, the single chip comprising the first device, the second device, and the third device; and A processing unit is coupled between the display panel and the single chip, and the processing unit is configured to selectively execute the at least one operation based on the third sensing information.

28. The computing device according to claim 24, characterized in that The computing device further comprises: a single chip, the single chip comprising the first device and the second device, The third device is a processing unit coupled between the display panel and the single chip, and the processing unit is configured to selectively execute the at least one operation based on the third sensing information.