Electronic device

Abstract

An electronic device that implements fingerprint authentication upon a sliding motion of a display. The electronic device is provided with a display and an optical fingerprint sensor. The display is provided with a display surface that is arrayed with light emitting pixels in a first direction and a direction intersecting the first direction. The optical fingerprint sensor is provided with a photographing element that is arrayed with light receiving elements in the first direction and a second direction on the opposite side of the display surface of the display in a third direction intersecting the first direction and the second direction, the light receiving elements respectively transferring electric charges obtained by photoelectric conversion at the same timing.

Description

Technical Field

[0001] This disclosure relates to electronic devices. Background Technology

[0002] Optical fingerprint sensors require the finger to remain still for a predetermined time to acquire a fingerprint, which is inconvenient from a user interface perspective. Furthermore, in many cases, on devices logged in via fingerprint authentication, subsequent operations can be performed without fingerprint authentication. For example, often after a single fingerprint-based personal authentication, one can make purchases via a browser using a credit card, which also raises security concerns.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: U.S. Patent Application Publication No. 2012 / 0258773 Summary of the Invention

[0006] This disclosure provides an electronic device that enables fingerprint authentication during a swiping motion on a display.

[0007] According to one embodiment, an electronic device includes a display and an optical fingerprint sensor. The display has a display surface on which light-emitting pixels are arranged in an array in a first direction and in directions intersecting the first direction. The optical fingerprint sensor includes an imaging element, which has light-receiving elements arranged in an array in the first and second directions on the side opposite to the display surface in a third direction intersecting the first and second directions. The light-receiving elements transmit charge obtained by photoelectric conversion at the same timing.

[0008] The imaging element may also have a pixel memory that temporarily stores the light-receiving information of each of the light-receiving elements, and the transmission of information from the light-receiving elements to the pixel memory may be performed at the same time.

[0009] The transfer of information from the light-receiving element to the pixel memory can also be performed by applying a transfer control signal to a transfer transistor at the same timing, the transfer transistor transferring the charge accumulated in the light-receiving element to the pixel memory.

[0010] The electronic device may also include an information processing unit that performs information processing on the fingerprint information read by the imaging element.

[0011] The information processing unit can also read the fingerprint information during actions that include at least a finger sliding motion.

[0012] The electronic device may also include a touch panel for sensing contact information with the display, and the speed of the swiping motion may be estimated in the touch panel.

[0013] The information processing unit can also estimate the speed of the sliding motion based on the information read from the imaging element.

[0014] It can also output an indication to slow down the rate of the sliding motion, based on the required authentication accuracy.

[0015] The speed of the sliding motion can also be indicated by outputting the display.

[0016] The instructions can also be displayed as a rate guide on the monitor.

[0017] It can also output the case where the rate is too fast if the rate is faster than the predetermined rate.

[0018] The output when the rate is too fast can also be at least one of the output to the display, the sound output, and the vibration output.

[0019] The information processing unit can also shorten the exposure time of the optical fingerprint sensor when the rate is faster than the predetermined rate.

[0020] The information processing unit can also generate authentication information based on the fingerprint information captured at different times.

[0021] The light-emitting pixels can also output light of different wavelengths on one side of the display surface of the light-receiving element, and the light-receiving element acquires the fingerprint information based on the reflected light of different wavelengths.

[0022] A polarizing filter may also be provided between the light-receiving element and the display surface, or the light-receiving element may sense the light polarized by the polarizing filter.

[0023] A filter for acquiring the state of hemoglobin may also be provided between the light-receiving element and the display surface, or the information processing unit may acquire the information of the hemoglobin for biological authentication.

[0024] The information processing unit can also perform biometric authentication based on information about the shape change of a finger in contact with the display surface over time.

[0025] The light-receiving element can also detect the sliding motion of multiple fingers.

[0026] The information processing unit can also perform fingerprint authentication using a combination of multiple fingers during the sliding motion of multiple fingers.

[0027] Depending on the required authentication precision, the combination of multiple fingers can also be different.

[0028] The information processing unit can also detect finger information and accumulate fingerprint information during the authentication process or when authentication is not possible.

[0029] The information processing unit can also accumulate changes in the finger to improve authentication accuracy.

[0030] The information processing unit can also acquire and accumulate the fingerprint information from fingers other than those not registered.

[0031] The light-receiving element can also be configured such that the number of elements in the direction intersecting the direction of the sliding action is greater than the number of elements in the direction of the sliding action.

[0032] The light-receiving element can also be configured such that the number of elements in the direction intersecting the direction of the sliding action is greater than twice the number of elements in the direction of the sliding action.

[0033] Alternatively, a guide can be displayed on the monitor to perform the sliding action in a direction that intersects with the direction in which the light-receiving element is set.

[0034] It may also have an interface, which displays an area with the light-receiving element on the display and configures product information and a purchase button. The product information can be slid from the product information to the purchase button through the area, thereby enabling the purchase of the product. Alternatively, the purchase information can be sent to the server based on the fingerprint authentication result.

[0035] Dynamic objects can also be displayed on the display in such a way that they include an area containing the light-receiving element.

[0036] The object can also change shape when touched by the user's finger.

[0037] The object can also change its light-emitting state when touched by a user's finger.

[0038] The luminescence state can also be changed to suit the acquisition of the fingerprint information.

[0039] The object can also change dynamically based on the acquisition status of the fingerprint information or the personal authentication status after the user's finger passes through it.

[0040] It may also include a tilt detection unit that detects the tilt of the display in the horizontal direction, and performs authentication of the fingerprint information based on the tilt detected by the tilt detection unit. Attached Figure Description

[0041] Figure 1 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0042] Figure 2 This is a schematic cross-sectional view of an electronic device according to one embodiment.

[0043] Figure 3 This is a block diagram illustrating an example of the configuration of an electronic device according to one embodiment.

[0044] Figure 4A This is a schematic cross-sectional view of an electronic device according to one embodiment.

[0045] Figure 4B This is a schematic cross-sectional view of an electronic device according to one embodiment.

[0046] Figure 5A This is a schematic diagram illustrating the light-receiving pixels according to one embodiment.

[0047] Figure 5B This is a diagram schematically illustrating the connection of light-receiving pixels according to one embodiment.

[0048] Figure 6 This is a flowchart illustrating the personal authentication process involved in one embodiment.

[0049] Figure 7 This is a block diagram illustrating an example of the configuration of an electronic device according to one embodiment.

[0050] Figure 8 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0051] Figure 9 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0052] Figure 10 This is a block diagram illustrating an example of the configuration of an electronic device according to one embodiment.

[0053] Figure 11 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0054] Figure 12 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0055] Figure 13 This is a diagram illustrating an example of the user interface of an electronic device according to one embodiment.

[0056] Figure 14 This is a block diagram illustrating an example of the configuration of an electronic device according to one embodiment.

[0057] Figure 15 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0058] Figure 16 This is a schematic diagram illustrating an electronic device according to one embodiment.

[0059] Explanation of reference numerals in the attached figures:

[0060] 1: Electronic equipment; 2: Display unit; 3: Camera module; 4: Display panel; 5: Circular polarizer; 6: Touch panel; 7: Cover glass; 8: Imaging unit; 8a: Photoelectric conversion unit; 8b: Pixel memory; 9: Optical system; 10: Information processing unit; 100: A / D conversion unit; 101: Clamping unit; 102: Color differentiation output unit; 103: Defect correction unit; 104: Linear matrix unit; 105: Frame memory; 106: Authentication unit; 107: Output unit; 108: Speed ​​detection unit; 109: Correction amount calculation unit; 11: Storage unit. Detailed Implementation

[0061] The following description, with reference to the accompanying drawings, will illustrate the implementation of the electronic device. The description will focus on the main components of the electronic device, but additional components or functions not shown or described may exist within the device. The following description does not exclude unshown or undescribed components or functions. Furthermore, for illustrative purposes, dimensions, shapes, aspect ratios, etc., may sometimes be changed, but they will have appropriate dimensions, shapes, aspect ratios, etc., when installed.

[0062] Furthermore, in the following description, the acquired signals are recorded as image information or shooting information, but this image information or shooting information is a broad concept, which also includes still images, moving images, or one frame of an image. In addition, "greater than" and "less than" can also be respectively translated as "above" and "below".

[0063] (First Implementation)

[0064] Figure 1 This is a schematic diagram illustrating an electronic device according to one embodiment. Additionally, Figure 2 This is a schematic cross-sectional view of an electronic device according to one embodiment. The electronic device 1 is any device with display and photography functions, such as a smartphone, mobile phone, tablet computer, or personal computer.

[0065] Electronic device 1 includes a display unit 2 and a camera module 3. From Figure 1As seen in the left image, the display screen 1a extends to roughly the size of the electronic device 1, and the width of the bezel 1b surrounding the display screen 1a can be set to, for example, a few millimeters or less. In electronic devices 1, a fingerprint authentication unit is often located within the bezel 1b; however, in this embodiment, as shown by the dotted line, a camera module 3 is located within the display screen 1a. Figure 1 As shown in the left figure, by placing the camera module 3, which takes photos for fingerprint authentication, on the inner surface of the display screen 1a, the width of the bezel 1b can be reduced.

[0066] In addition, Figure 1 In this diagram, the camera module 3 is positioned on the inner surface of the display screen 1a, approximately near the center. However, its position is not limited to this diagram; it can be positioned anywhere on the inner surface of the display screen 1a. For example, the camera module 3 can be positioned near the periphery of the display screen 1a, or it can be positioned further below the center compared to the diagram on the right. Furthermore, although it is positioned in one location in the diagram, it can be positioned in multiple locations. Additionally, in this diagram, the display unit 2 and the camera module 3 are provided on one side of the electronic device 1, but this is not a limitation. For example, the display unit 2 and the camera module 3 can be provided on both sides of the electronic device 1.

[0067] Display unit 2 is a display optical system, and is a structure in which display panel 4, circular polarizing plate 5, touch panel 6 and cover glass 7 are stacked. In addition, their arrangement is not particularly limited, and they can be appropriately replaced, or more than two of the same configurations can exist, and other configurations can also be included.

[0068] The display panel 4 may also be provided in an array, for example, as an OLED (Organic Light Emitting Diode), liquid crystal, MicroLED, or light-emitting elements based on other display principles. For example, the light-emitting elements are arranged in an array within a plane including a first direction and a second direction. The display panel 4, such as an OLED, is composed of multiple layers. The display panel 4 often includes components with low transmittance, such as color filter layers. As described below, the components with low transmittance in the display panel 4 may also form through-holes corresponding to the placement location of the camera module 3. If subject light passing through the through-hole is incident on the camera module 3, the image quality captured by the camera module 3 can be improved.

[0069] The circular polarizer 5 is installed to reduce glare or improve the visual clarity of the display screen 1a even in bright environments. A touch sensor is assembled in the touch panel 6. Touch sensors can be of various types, including capacitive, resistive, and pressure-sensitive types, but any type can be used. Alternatively, the touch panel 6 and the display panel 4 can be integrated. A cover glass 7 is provided to protect the display panel 4, etc. These components can also be bonded together using an adhesive with minimal optical impact.

[0070] The camera module 3 includes a shooting unit 8 and an optical system 9. The optical system 9 is positioned on the light-incident surface side of the shooting unit 8, close to the display unit 2, so that the light passing through the display unit 2 is focused onto the shooting unit 8. The optical system 9 may also include one or more lenses. For example, the shooting unit 8 functions as an optical fingerprint sensor to acquire the user's fingerprint information.

[0071] The imaging unit 8, which functions as an optical fingerprint sensor, includes multiple photoelectric conversion units. Each photoelectric conversion unit is equipped with a lens. These lenses allow the photoelectric conversion units constituting each pixel to receive light appropriately emitted by the optical system 9 onto the imaging unit 8. The photoelectric conversion units perform photoelectric conversion on the light incident via the display unit 2. The photoelectric conversion units can be CMOS (Complementary Metal-Oxide-Semiconductor Field-Effect Transistor) sensors or CCD (Charge Coupled Device) sensors. The photoelectric conversion units are arranged in an array, for example, in a plane including a first direction and a second direction. Alternatively, the photoelectric conversion units can be arranged in an array along both the first and second directions. For example, the light-receiving elements of the photoelectric conversion units are configured as an array of light-receiving pixels aligned with the same direction as the light-emitting pixels of the display unit.

[0072] Furthermore, the photoelectric conversion unit can include a photodiode or an organic photoelectric conversion film. Multiple photoelectric conversion units can be arranged in any manner. For example, the arrangement of multiple photoelectric conversion units can be a Bayer arrangement, a row-and-row arrangement, a grid arrangement, a stripe arrangement, or other arrangements.

[0073] In this disclosure, the output value of the photoelectric conversion unit, or the value based on the output value after a predetermined conversion, is referred to as the light-receiving pixel value.

[0074] In this embodiment, the user's fingerprint information is acquired at a time when the finger passes through the area where the camera module 3 of the display exists, and the fingerprint information is used to perform personal authentication. For example, as Figure 1As shown in the left image, when product information and a purchase button are displayed on the screen, the camera module 3 acquires fingerprint information from a finger that has been swiping, tapping, or wiping when the product image is dragged onto the purchase button. This acquisition is performed using a global shutter, as described below. Furthermore, the camera module 3 performs this global shutter action at an appropriate exposure time.

[0075] Figure 3 This is a block diagram illustrating an example of the electronic device 1 according to this embodiment. (Omitted) Figure 1 , Figure 2 The display unit 2 is located inside the device 1. The electronic device 1 includes an information processing unit 10 and a storage unit 11 that perform information processing by outputting information from the camera module 3 and the touch panel 6 described above.

[0076] To achieve the aforementioned global shutter action, the imaging unit 8 of the camera module 3 includes a photoelectric conversion unit 8a (photoelectric conversion element) and a pixel memory 8b corresponding to each photoelectric conversion unit 8a. Information sensed by the photoelectric conversion unit 8a is transmitted to the corresponding pixel memory 8b at the same timing. Furthermore, the light-receiving result is output from the pixel memory 8b to the information processing unit 10 at any timing.

[0077] The information processing unit 10 may be configured with, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or a FPGA (Field Programmable Gate Array). This information processing unit 10 may also be divided into blocks that perform various actions, as described below, or each action may be specifically implemented through software-based information processing by the CPU.

[0078] The storage unit 11 may include various types of memory and storage devices. Information such as programs and data required for the operation of the electronic device 1 may also be stored non-temporarily in the storage unit 11. Additionally, the memory may include a temporary storage area such as a cache area.

[0079] The information processing unit 10 includes: an A / D conversion unit 100, a clamping unit 101, a color differentiation output unit 102, a defect correction unit 103, a linear matrix unit 104, a frame memory 105, an authentication unit 106, and an output unit 107. The frame memory 105 may also be part of the storage unit 11 and not provided in the information processing unit 10.

[0080] The A / D converter 100 receives and converts the data received and converted by the photoelectric converter 8a, i.e., the analog signal stored in the pixel memory 8b at a predetermined time, and converts it into a digital signal for each pixel. The A / D converter 100 then outputs the converted digital signal as image data.

[0081] The clamping unit 101 may, for example, define a black level, and output the image data output from the A / D conversion unit 100 by subtracting the defined black level. The clamping unit 101 may also set a ground level for each photoelectric conversion element provided in the pixel. In this case, grounding correction of the signal value is performed based on the ground level set for each photoelectric conversion element.

[0082] The color differentiation output unit 102 outputs subpixel image data by color when, for example, analog signals are acquired by color differentiation in the photoelectric conversion unit 8a. The photoelectric conversion unit 8a has color filters such as R (red), G (green), and B (blue). Each photoelectric conversion unit 8a outputs sensing information by color through these filters. Alternatively, the photoelectric conversion unit 8a may not have color filters, and it may acquire analog signals by color using an organic photoelectric conversion film.

[0083] By adjusting the grounding level based on these filters via the clamping unit 101, the color differentiation output unit 102 outputs the signal output by the clamping unit 101 according to color differentiation. The analog signal acquired in the photoelectric conversion unit 8a does not include color information. Therefore, the color differentiation output unit 102 can also store data related to the colors configured according to each light-receiving element in the photoelectric conversion unit 8a, and output according to each color based on this data.

[0084] Furthermore, the photoelectric conversion unit 8a may also include, for example, elements that receive near-infrared light, plasma filters, or filters that acquire complex spectra. In these cases, representing information as a single color is difficult, but if the color differentiation output unit 102 can differentiate by wavelength of light, it can also associate this information with wavelength information and process it. In such cases, wavelength information can also be processed by the color differentiation output unit 102 as information with a range.

[0085] The defect correction unit 103 performs defect correction in the image data. Defects in the image data are caused by, for example, pixel loss or information loss due to defects in photoelectric conversion elements provided within pixels, or information loss due to light saturation in the optical system 9. The defect correction unit 103 may also perform defect correction processing by interpolating based on, for example, information from surrounding pixels or information from surrounding pixels with the same color information.

[0086] The linear matrix unit 104 performs color reproduction by performing matrix operations on color information. The linear matrix unit 104 obtains the desired spectral dispersion by performing operations on multiple wavelengths, for example. In this embodiment, the linear matrix unit 104 can also perform outputs suitable for skin color detection, for example. Furthermore, the linear matrix unit 104 can perform operations to obtain biometric authentication information such as veins, resulting in an output suitable for detection in the wavelength range from yellow to red.

[0087] The frame memory 105 can also be provided as part of the storage unit 11, for example, as described above. After the above-described processing, the frame memory 105 temporarily stores the information acquired by the pixel memory 8b and output from the photoelectric conversion unit 8a at the same timing. For example, the information processing unit 10 performs various of the above-described processing on the fingerprint information acquired at the same timing and saves it in the frame memory 105 as a block of image data that is finally reproduced in color by the linear matrix unit 104.

[0088] The authentication unit 106 uses fingerprint information stored in the frame memory 105 to perform personal authentication. For example, this personal authentication is performed by extracting feature points from the fingerprint data and comparing the extracted feature points with the authentication information stored in the storage unit 11. Furthermore, the authentication unit 106 can also perform biometric authentication to prevent impersonation based not only on RGB information, but also on, for example, near-infrared light reception information or information obtained via a plasma filter. For example, biometric authentication can be performed by checking whether the wavelength corresponding to hemoglobin in the data stored in the frame memory 105 is correctly acquired. These can also be achieved by statistical processing to determine feature points, and as another example, authentication can be performed using a network model trained by machine learning.

[0089] For example, the authentication unit 106 stores pre-acquired fingerprint information of a specific user in the storage unit 11. Personal authentication is performed based on this stored user information. When authenticating multiple users, it is possible to appropriately store the fingerprint information of multiple users. Furthermore, the stored fingerprint information can also be encrypted with images of each fingerprint or information about feature points obtained from each fingerprint.

[0090] Output unit 107 outputs the authentication result based on authentication unit 106. For example, in the case of shopping based on e-commerce, the result can be sent to a server or the like for checkout processing. In this way, output unit 107 can also send the authentication result wirelessly. Alternatively, as another example, the authentication result, such as fingerprint authentication, can be used in electronic device 1 to replace authentication based on password, etc.

[0091] Figure 4A More detailed explanation Figure 1The diagram shows the relationship between the camera module 3 and the display panel 4. As described above, the camera module 3 includes, for example, an imaging unit 8 and an optical system 9. The optical system 9 is positioned on the light incident surface side of the imaging unit 8, i.e., the side closest to the display unit 2. Light that has passed through the display surface of the display unit 2 propagates to the imaging unit 8 through the optical system 9.

[0092] For example, the shooting unit 8 can also be equipped with Figure 3 The photoelectric conversion unit 8a, such as a photodiode, and the pixel memory 8b are shown. Light that is conducted through the optical system 9 by light collection, refraction, diffusion, etc., is received by the photoelectric conversion unit 8a provided in the imaging unit 8 and output as an analog signal. The photoelectric conversion unit 8a may also have color filters such as Bayer filters, or stacked color filters, provided on the incident surface side of each imaging element. Alternatively, as described above, it may not have color filters and may be provided with other filters or other components such as organic photoelectric conversion films. In addition, regarding the output, although not shown, other components and circuits required for output light reception and analog signals are provided.

[0093] The optical system 9 can also be configured as including a through-hole in a component with low transmittance. The through-hole could include, for example, an opening in the display panel 4, particularly in a substrate a with low transmittance within the display panel 4. The optical system 9 could include, for example, an opening in the display panel 4 and a lens positioned closer to the imaging unit 8 than the opening. Through this lens and opening, the optical system 9 can also be defined as optical characteristics such as the number of openings Na (numerical aperture), F-number, etc., in the camera module 3.

[0094] Furthermore, while the opening and lens are shown as examples, the configuration of optical system 9 is not necessarily limited to these combinations. Additionally, in the figures, one or more lenses are provided for an opening, but this is not a limitation. For example, as shown... Figure 4B As shown, multiple openings can also be provided for a lens in the optical system 9. Light-emitting elements, such as those of the display panel 4, can be provided in areas where no openings exist, with the openings provided between these light-emitting elements. With this configuration, the camera module 3 can be installed without disrupting the display.

[0095] Figure 5A This is a schematic cross-sectional view illustrating a light-receiving pixel according to one embodiment. The light-receiving pixel includes, for example, the photoelectric conversion unit 8a and pixel memory 8b described above. Furthermore, in this disclosure, only an example is shown, therefore detailed descriptions are omitted. Additionally, the light-receiving pixel in this disclosure is not limited to having… Figure 5A and Figure 5B The light-receiving pixels shown can be used as the basis for the global shutter action.

[0096] The photoelectric conversion unit 8a receives light in the light-receiving area 8R depicted above it in the figure, which is provided in the optical system 9 and the lens of the light-receiving pixel. This light-receiving unit is separated from the light-receiving area 8R of the adjacent pixel by the blocking part 8d. As shown in the figure, the blocking part 8d is configured to prevent light from passing through the surface of the photoelectric conversion unit 8a above the pixel memory 8b, i.e., the surface where the light-receiving area 8R is located.

[0097] The shielding portion 8d is made of metal, for example, and is separated from the photoelectric conversion portion 8a and the pixel memory 8b by an insulating portion 8e. The insulating portion 8e is, for example, an oxide film.

[0098] When the photoelectric conversion unit 8a receives light in the light-receiving area 8R, it accumulates charge according to the intensity of the received light. The accumulated charge is transferred to the pixel memory 8b by applying a voltage to the electrode 8c at a predetermined time. Before the transfer of charge received from the pixel memory 8b at the next time interval, an analog signal is output to the A / D conversion unit 100 via other transistors or the like. The transfer to the A / D conversion unit 100 may also occur at different times.

[0099] The transfer of charge from the photoelectric conversion unit 8a to the pixel memory 8b is performed at the same timing by multiple photoelectric conversion units 8a present in the light-receiving pixel array. The same timing does not mean exactly the same instant; there may be deviations to the extent that there is no shutter distortion. Therefore, the imaging unit 8 performs a global shutter operation on the light received by the multiple photoelectric conversion units 8a, thereby outputting an analog signal based on the light received at the same timing. Furthermore, the information processing unit 10 can acquire an image based on the light received at the same timing. That is, the imaging unit 8 can act as an optical fingerprint sensor to acquire the user's fingerprint information at the same timing.

[0100] Figure 5B It is schematically shown in multiple pixels. Figure 5A The figure shows a top view of the connection between the photoelectric conversion unit 8a and the pixel memory 8b. As an example, as shown in the figure, in... Figure 5A The middle electrode 8c can also be the gate of a transmission transistor. Figure 5B In this paper, wiring other than the control line between the photoelectric conversion unit 8a and the pixel memory 8b is omitted, but wiring from the pixel memory 8b to the information processing unit 10 can of course be appropriately provided.

[0101] The connection between the photoelectric conversion unit 8a of each light-receiving pixel and the pixel memory 8b is controlled by a transfer transistor. After transferring charge to the pixel memory 8b and discharging the charge contained in the pixel memory 8b, a voltage is applied to the respective electrode 8c of each light-receiving pixel for the same duration at the same timing. This voltage, for example, in the case where the transfer transistor is an n-type MOSFET, is sufficiently higher than the threshold voltage for the flow of charge from the drain to the source. By applying voltage at the same timing, the charge acquired by photoelectric conversion at the same timing is transferred to each pixel memory 8b. As a result, analog information (charge) of the image acquired at the same timing is temporarily accumulated in each pixel memory 8b.

[0102] The output from the pixel memory 8b may also be transmitted to the information processing unit 10 at different times. For example, the transmission from the pixel memory 8b to the information processing unit 10 is performed at predetermined times for each line.

[0103] Furthermore, as an example, the pixel memory 8b is configured to store charge states, but it is not limited to this. Other examples of the pixel memory 8b include storing voltage states, digital values, or other states related to the intensity of light received.

[0104] In addition, Figure 5A , Figure 5B In this embodiment, the pixel memory 8b is disposed on the same substrate as the light-receiving element, but is not limited thereto. Alternatively, the pixel memory 8b may also be disposed within a chip stacked and connected to the imaging element. In this case, the transmission transistors may also be disposed in any layer or between layers. These layers may also be connected via, for example, vias, microbumps, micropads, plasma bonding, etc. These connections may also be stacked using, for example, CoC (Chip on Chip), CoW (Chip on Wafer), or WoW (Wafer on Wafer).

[0105] comprehensive Figures 1 to 5A , Figure 5B Alternatively, the imaging unit 8 may have an imaging element positioned in a third direction. This imaging element has light-receiving elements arranged in an array in both a first and a second direction on the side opposite to the display surface of the display unit 2, which serves as the display. Furthermore, these light-receiving elements each perform a global shutter operation to read out the light-receiving information at the same timing. Reading out the light-receiving information at the same timing could mean that information indicating the charge to be converted by photoelectric conversion in the light-receiving element (photoelectric conversion unit) is transmitted to a memory provided in each light-receiving element at the same timing. Alternatively, it could be different. Figure 5A , Figure 5BAs shown, the light-receiving pixels employ other methods to achieve the same action as the global shutter.

[0106] As described above, the information processing unit 10 performs signal processing on the fingerprint information read by the imaging unit 8, i.e., the optical fingerprint sensor. Furthermore, the user can obtain fingerprint information from the optical fingerprint sensor by sliding their finger across the display surface of the display unit 2 (performing a sliding motion), thus performing personal authentication, etc. The following description focuses on sliding motions, but is not limited to sliding; any action such as tapping or wiping across the touch panel is acceptable.

[0107] Figure 6 This is a flowchart of a personal authentication process involved in one implementation method.

[0108] First, electronic device 1 determines whether to start personal authentication (S100). If it is not the timed start of personal authentication (S100: No), it continues in standby mode. The start of authentication can also be, for example... Figure 3 As shown by the dashed line, the sensing information from the touch panel 6 is used. Alternatively, once the sensing information from the touch panel 6 reaches the range of the camera module 3, the process can proceed to fingerprint information acquisition. As a preceding stage, or for authentication purposes, the electronic device 1 is notified that authentication processing is being performed on a website, etc., and the electronic device 1 enters a standby state.

[0109] When the authentication start state is reached (S100: Yes), the camera unit 8 begins to receive light for personal authentication (S102). For example, by activating the camera module 3 during fingerprint authentication and switching to a standby state for acquiring fingerprint information, light is received at necessary intervals.

[0110] When performing fingerprint authentication, the photoelectric conversion unit 8a of the imaging unit 8 transmits the intensity of the light received to the pixel memory 8b at the same timing as described above (S102). For example, the transmission is performed by applying a voltage for transmission to multiple electrodes 8c at the same timing.

[0111] Next, the A / D conversion unit 100 performs A / D conversion on the data transmitted to the pixel memory 8b at the same timing (S106). Here, the analog signal acquired by the photoelectric conversion unit 8a and output by the pixel memory 8b is converted into a digital signal.

[0112] Next, the information processing unit 10 performs signal processing other than A / D conversion of the converted digital signal, and image processing (S108). This processing includes, for example, clamping, color differentiation output, defect correction, color reproduction processing, and storage to the frame memory 105, as described above. Furthermore, the above is just one example; in addition to these, appropriate processing may be performed further.

[0113] Next, the authentication unit 106 performs authentication using an image including the acquired fingerprint information (S110). As described above, authentication can also be performed by extracting feature points and comparing them, or by using a trained neural network. Furthermore, the electronic device 1 may also include a chip capable of performing the aforementioned information processing and neural network processing within the same substrate as the camera module 3.

[0114] The output unit 107 outputs the authentication result to the necessary location (S112). For example, when performing personal authentication during online shopping, the output unit 107 can notify the browser that has opened the website or the application that accepts the authentication result of the personal authentication result. In this way, the output unit 107 can perform the output at the required location.

[0115] For example, if authentication fails, the output unit 107 can notify the camera unit 8 of the authentication failure and the need to perform the authentication process again. Alternatively, the output unit 107 can send a shooting instruction to the camera unit 8 again.

[0116] The authentication unit 106 can also perform authentication processing using multiple fingerprint information when acquiring images in consecutive frames. For example, if authentication is successful using one of the multiple fingerprint information, the output unit 107 can be notified of the successful authentication at that time.

[0117] The authentication unit 106 can also adjust the success and failure thresholds based on authentication accuracy when seeking high-precision authentication. For example, to achieve high authentication accuracy, the accuracy can be improved by increasing the authentication threshold, such as the fingerprint consistency threshold. Improving authentication accuracy can also involve increasing, for example, the consistency threshold of feature points. Furthermore, when using a neural network model, the consistency threshold can be increased. Situations requiring high-precision authentication include, for example, when wanting to buy a high-priced item or access information with high machine density.

[0118] Alternatively, if the imaging unit 8 is equipped with a near-infrared filter, plasma filter, or the like, it can be used in conjunction with personal authentication to determine whether the person is a living organism. This authentication, for example, is based on information obtained from near-infrared light to determine whether the wavelength of light transmitted through hemoglobin, such as vein light, is present. Alternatively, when performing this authentication, infrared light or the like can be emitted from the display unit 2, and the reflection state can be obtained for determination.

[0119] Beyond these, for example, the display panel 4 can emit light of various colors, and the biological entity can be identified based on the reflection of that emitted light. In this case, filters that acquire various colors as described above can be included to obtain so-called multispectral and hyperspectral information for analysis. For example, it is possible to make a judgment based on information representing a biological entity, especially human skin or reflections from within the skin.

[0120] Biometric authentication can utilize not only the information acquired by the imaging unit 8, but also sensing information from the touch panel 6. For example, the authentication unit 106 can determine whether the sensing area on the touch panel 6 is elastic to authenticate the biometric entity. The imaging unit 8 can also be used to determine whether the image of a finger acquired by the imaging unit 8 is elastic over time. Elasticity can also be determined based on, for example, changes in the shape or size of the area occupied by the finger. Alternatively, the touch panel 6 can be equipped with a pressure sensor, and elasticity can be determined based on the sensing information from that pressure sensor.

[0121] Alternatively, authentication accuracy can be improved by acquiring fingerprint information at multiple time intervals. For example, the imaging unit 8 can acquire fingerprint information at multiple time intervals, and this fingerprint information can be accumulated in the frame memory 105. Furthermore, the accumulated fingerprint information can be used to perform authentication. For example, image defects generated during image acquisition can be corrected by referring to fingerprint information from time to time. More simply, a weighted average of the fingerprint information acquired at each time interval can be calculated after adjusting the position and angle of the fingerprint. Furthermore, the fingerprint information obtained by this averaging calculation can be used as the object of authentication.

[0122] The imaging unit 8 may also include other filters. For example, by setting a polarizing filter on the light-receiving pixel, it is possible to acquire fingerprint information after polarization. By acquiring the polarized information, the convexity and concavity of the fingerprint can be acquired more clearly. The information processing unit 10 can also synthesize such polarized information and extract feature points.

[0123] Furthermore, even when the imaging unit 8 operates the global filter, motion blur may sometimes occur due to shutter speed. To address this motion blur, the information processing unit 10 may also include an image correction unit that performs degradation correction on the images stored in the frame memory 105.

[0124] As described above, according to this embodiment, highly accurate personal authentication can be achieved by using a camera module located under the display. Furthermore, by authenticating the human body, impersonation can be prevented. Additionally, in the imaging unit 8, images can be acquired with a global shutter action, thus enabling reliable fingerprint authentication even for shutter shake and other image-dependent issues.

[0125] (Second Implementation)

[0126] In the first embodiment described above, the user's swiping speed was not taken into account. In the second embodiment, the speed of the user's finger is determined, and fingerprint information is corrected based on that speed.

[0127] Figure 7This is a block diagram illustrating an example of the electronic device 1 according to this embodiment. Electronic device 1, besides... Figure 3 In addition to the components described herein, it also includes a speed detection unit 108.

[0128] The speed detection unit 108, for example, acquires the speed of the user's finger movement based on the sensing information of the touch panel 6, at the timing when the camera module 3 acquires the fingerprint image. That is, when the user makes a swiping motion, while the camera module 3 acquires fingerprint information from the light-receiving information, the touch panel 6 acquires finger speed information from the sensing information. For example, the speed detection unit 108 acquires the finger speed based on the amount of movement of the area of ​​the user's finger sensed by the touch panel 6 within a predetermined time.

[0129] For example, when a user's finger touches the touchscreen during authentication, the speed detection unit 108 acquires speed sensing information from the touch panel 6. The speed detection unit 108 also acquires sensing information of the touch panel 6 at a certain moment and sensing information of the touch panel 6 after a predetermined time, for example, when the user's finger moves.

[0130] The sensing information from the touch panel 6 can also be obtained from the center point of that finger area, such as the position of the center of gravity, when a finger area has been sensed. The position of the center of gravity can be obtained by calculating, for example, the sum of the first and second directions of the point in the sensing area. Furthermore, the speed detection unit 108 detects the sliding speed of the user's finger by calculating the position of the center of gravity at a certain moment and a predetermined moment later.

[0131] If the finger speed is faster than the predetermined speed, the speed detection unit 108 can, for example,... Figure 8 As shown, the output prompts the user to slow down the sliding speed on the display unit 2. Alternatively, as another example, the speed detection unit 108 may notify the output unit 107 that the speed is too fast, and output this notification to the display unit 2 via the output unit 107. Furthermore, in this case, the output unit 107 is not limited to displaying on the display unit 2; it could also be, for example, notifying the user via sound through a speaker, or vibrating a vibrator.

[0132] Alternatively, the output unit 107 can determine the rate based on the desired authentication accuracy. In such cases, for example, where high authentication accuracy is required, the output unit 107 lowers the threshold for the speed detected by the speed detection unit 108, and can output a state of slow sliding at a slower rate to the user.

[0133] Figure 9Other examples of output are shown. Output unit 107 can also display a gradual increase in the number of triangles displayed within a predetermined time period, for example, from the left image to the right image. The rate at which the triangles increase can also be the rate at which the imaging unit 8 acquires high-precision fingerprint information. Thus, output unit 107 can also output a rate indicator (rate guide) displayed on the screen. This indicator is not limited to... Figure 9 Such a triangle can also be a diagram pointing to, for example, an arrow. It is not limited to this. As long as the interface is used to indicate the speed to the user, it can be any graphic or shape.

[0134] Furthermore, the speed detection unit 108 detects the rate based on the sensing information of the touch panel 6, but is not limited to this. For example, the rate can also be detected based on the shooting information acquired by the imaging unit 8. The speed detection unit 108, for example, refers to the frame memory 105 and detects the rate of the user's finger based on the timing from when an image of the user's finger is acquired to when an image of the user's finger can no longer be acquired, and based on the time spent between these timings and the size of the fingerprint acquisition area. It is not limited to this; the rate can also be detected by estimating the extent of finger movement after one or more frames from a certain moment.

[0135] As described above, according to this embodiment, by detecting the sliding speed of the user's finger, the accuracy of fingerprint acquisition or the accuracy of fingerprint authentication can be improved. Furthermore, if necessary, the user can be urged to output the fingerprint at a rate suitable for fingerprint acquisition.

[0136] Furthermore, the speed detected by the speed detection unit 108 can be used for purposes other than this. For example, the camera module 3 can also acquire fingerprint information with an appropriate exposure time based on this speed information. The camera module 3 can also shorten the exposure time, for example, in the case of a slow speed, and the longer the exposure time, the faster the speed.

[0137] (Third Implementation)

[0138] The second embodiment may further include a speed-based correction process.

[0139] Figure 10 This is a block diagram illustrating an example of the electronic device 1 according to this embodiment. Electronic device 1, besides... Figure 7 In addition to the components described herein, it also includes a correction calculation unit 109.

[0140] The correction calculation unit 109 acquires a correction amount based on the speed detected by the speed detection unit 108. Image processing can also be performed on the fingerprint information stored in the frame memory 105 based on this correction amount. This image processing is based on, for example, motion blur image processing. Alternatively, a point spread function (PSF) can be generated based on the finger speed and shutter speed, and inverse filter processing (deconvolution filter processing) can be performed on the fingerprint information.

[0141] By performing such processing, the accuracy of authentication can be further improved.

[0142] (Fourth Implementation)

[0143] In the above example, authentication is performed using one finger. Authentication can also be performed, for example, using the index finger. Furthermore, authentication using multiple fingers is also possible.

[0144] For example, the imaging unit 8 can acquire information not only from the index finger but also from the middle finger during the same sliding timing, performing personal authentication based on two fingers. Furthermore, it can use three to five fingers, or even separate fingers such as the index and ring fingers, for authentication. Obviously, the index finger may not be included in these combinations of multiple fingers.

[0145] The above settings are based on the same timing, but different timings are also possible. For example, after performing the first authentication with the index finger, the second authentication can be performed with the middle finger, and so on. In this case, information from both the right and left hands can also be used.

[0146] When acquiring information from multiple fingers at the same time interval, authentication can be performed for each finger individually, or the information from swiping multiple fingers can be acquired as a single authentication message.

[0147] As mentioned above, authentication can be performed at any time using any number or combination of fingers. By increasing the authentication information in this way, false detections and impersonation can be prevented with even higher precision. Multi-finger-based authentication can also be modified based on the required level of authentication accuracy. For example, multiple fingers or specific combinations of fingers can be used when high authentication accuracy is required.

[0148] (Fifth Implementation)

[0149] The electronic device 1 in the aforementioned embodiments is, for example, a smartphone. In devices with such a touch panel, in addition to the timing of fingerprint authentication, fingers may sometimes pass over the camera module 3.

[0150] Therefore, the electronic device 1 according to this embodiment acquires information about such a finger at arbitrary time intervals via the imaging unit 8. Furthermore, the fingerprint information can be stored in the storage unit 11 if the user is authenticated as a logged-in user via the authentication unit 106. Alternatively, the acquired fingerprint information can be stored in the storage unit 11 upon successful authentication.

[0151] Furthermore, the information processing unit 10 can also update the authentication information based on the information accumulated at any given time interval. Alternatively, when fingerprint information is acquired as image information, for example, the information accumulated in the same direction and at the same location can be calibrated, and a weighted average can be calculated to use as the authentication information, assigning greater weight to newer information.

[0152] Alternatively, when personal authentication is performed using a predetermined finger, information about other fingers used by the user can be acquired at regular intervals and stored in storage unit 11. This approach improves authentication accuracy even when multiple fingers are used.

[0153] As described above, according to this embodiment, the information used for authentication can be updated. By updating, authentication information applicable even to subtle changes in fingers (fingerprints) due to aging, changes caused by the environment, etc., can be acquired. As a result, high accuracy of personal authentication can be maintained.

[0154] (Sixth Implementation Method)

[0155] Figure 11 as well as Figure 12 This diagram schematically illustrates the electronic device 1 according to this embodiment. In this embodiment, the camera module 3 is configured to be longer in a direction perpendicular to the sliding direction. For example, as... Figure 11 As shown, when sliding from top to bottom or from bottom to top (second direction), a camera module 3 is used in the area where the shooting section 8, which is wide in the horizontal direction (first direction), exists. Figure 12 As shown, when sliding from left to right or from right to left (first direction), a camera module 3 is used in the area where the shooting part 8, which has a width in the longitudinal direction (second direction), exists.

[0156] For example, the shooting unit 8 through Figure 11 , Figure 12 The area shown by the dashed line contains a photoelectric conversion unit 8a (light-receiving pixel), enabling such installation. The size of this light-receiving pixel can also be, for example, more than twice the number of pixels on the wider side compared to the narrower side. For example, in... Figure 11 Alternatively, the number of light-receiving pixels set along the first direction can be more than twice the number of light-receiving pixels set along the second direction.

[0157] Furthermore, it can also be done in a way that spans the area where the light-receiving pixel is located, such as... Figure 12 The image shows a guide indicating the direction of the swipe. As another example, the area for fingerprint authentication via the swipe action could also be displayed on the screen. Figure 11 , Figure 12 The area is indicated by the dotted line. This display also shows the user where to swipe to perform personal authentication.

[0158] (Seventh Implementation)

[0159] The sliding area can also be easily understood by the object displayed on the display unit 2 of the electronic device 1. This object can also change dynamically.

[0160] Figure 13 This diagram illustrates an example of a GUI (Graphical User Interface) displayed on a display unit 2 of an electronic device 1. The display unit 2 shows a circular object, such as a water droplet as shown in the left diagram. Furthermore, as shown in the right diagram, when the user slides across the circular object, the object can be scattered.

[0161] Additionally, to more clearly show the sliding area while waiting to slide, the object in the left image can, for example, shake or move within a predetermined area. Its shape and size can also change based on the user's finger touch. Furthermore, it can fade out after the slide.

[0162] The display of objects is not limited to Figure 13 Such water droplets, for example, can also be displayed as flowing objects like a river. By setting the display in this way, the direction of the swipe can also be shown to the user. Not limited to these examples, for example, displaying predetermined symbols or dynamic objects, etc., the display should be easy for the user to understand.

[0163] Alternatively, the light-emitting state of the light-emitting element can change when the user's finger touches the area. Changing the light-emitting state can also alter the wavelength of the reflected light in the display panel 4 where the user's finger touches, making it easier to acquire fingerprint information. For example, by setting the emitted light wavelength to a yellow-red range, it's easier to discern the reflection from the user's skin tone or veins. Conversely, by emitting light with a wavelength close to the complementary color, such as cyan, the wavelength of the reflected light can be made suitable for fingerprint information acquisition.

[0164] Furthermore, the object can dynamically change based on the fingerprint information acquisition status. For example, if fingerprint information is acquired normally, the object's color may change, its shape may change, or it may fade out. If fingerprint information cannot be acquired normally, the object may be displayed in its initial state, or it may change to a color or shape different from the normally acquired state. Additionally, as in the embodiments described above, a rate guide or a location guide may be displayed. The information processing unit 10 may also include a fingerprint information acquisition determination unit. The acquisition of fingerprint information may also be based on whether a predetermined number of dents or convexities are sensed within, for example, a predetermined area.

[0165] This can also be based on authentication information. For example, the object can be changed as described above in cases of successful and failed personal authentication. Of course, changes can occur at two levels based on the acquisition status of fingerprint information and the success or failure of authentication.

[0166] As described above, according to this embodiment, the area where the imaging portion 8 exists in the camera module 3 can be expanded in a direction intersecting the sliding direction. By providing an optical fingerprint sensor in this way, an optical fingerprint sensor that is stable against positional deviations can be installed.

[0167] (Eighth Implementation Method)

[0168] Figure 14 This is a block diagram illustrating an example of the electronic device 1 according to this embodiment. The electronic device 1 also includes a tilt detection unit 12.

[0169] The tilt detection unit 12 includes, for example, a gyroscope or an accelerometer. The tilt detection unit 12 detects the tilt of the electronic device 1 at the time when fingerprint information is acquired. Tilt refers to the tilt of the display relative to, for example, gravity (vertical direction) or the horizontal direction.

[0170] Figure 15 and Figure 16 This is a diagram illustrating an example of the tilt of electronic device 1. For example, as shown... Figure 15 As shown, the first and second directions in electronic device 1 are represented as the sum of the components of the gravity direction and the horizontal direction intersecting it. When the absolute value of the gravity direction component of the first direction is less than the absolute value of the gravity direction component of the second direction, the authentication unit 106 preferentially uses matching data such as the finger's orientation as shown in the attached diagram on the right to perform fingerprint authentication.

[0171] On the other hand, such as Figure 16As shown, when the absolute value of the gravity direction component in the first direction in the electronic device 1 is greater than the absolute value of the gravity direction component in the second direction, the authentication unit 106 preferentially uses the matching data, such as the finger pointing to the left in the attached diagram shown on the right, to perform fingerprint authentication.

[0172] Alternatively, the magnitude and sign of the direction of gravity in each direction can be used. In this case, for example, if one of the four sides is at the bottom, fingerprint authentication can be performed preferentially using data that rotates the fingerprint every 90 degrees for matching.

[0173] As described above, according to this embodiment, by selecting or prioritizing the data used for matching based on the tilt of gravity, the accuracy or speed of matching can be improved. As an example, intervals of 180 degrees or 90 degrees have been described, but the angle intervals can also be smaller.

[0174] Furthermore, while it is set to select a matching pattern, it is not limited to this. For example, personal authentication processing can also be performed by rotating and correcting fingerprint information based on the results of tilt detection.

[0175] Furthermore, this technology can be configured as follows.

[0176] (1) An electronic device comprising:

[0177] Displays; and

[0178] Optical fingerprint sensor,

[0179] The display has a display surface, on which light-emitting pixels are arranged in an array in a first direction and a second direction intersecting the first direction.

[0180] The optical fingerprint sensor includes an imaging element, which has light-receiving elements arranged in an array on the side opposite to the display surface of the display in a third direction intersecting the first and second directions. The light-receiving elements transmit charges obtained by photoelectric conversion at the same timing.

[0181] (2) The electronic device according to (1), wherein,

[0182] The imaging element has a pixel memory that temporarily stores the light-receiving information of each of the light-receiving elements, and the transmission of information from the light-receiving elements to the pixel memory is performed at the same time interval.

[0183] (3) The electronic device according to (2), wherein,

[0184] The transfer of information from the light-receiving element to the pixel memory is performed by applying a transfer control signal to a transfer transistor at the same timing, the transfer transistor transferring the charge accumulated in the light-receiving element to the pixel memory.

[0185] (4) The electronic device according to (1) to (3), wherein,

[0186] The electronic device also includes an information processing unit, which performs information processing on the fingerprint information read by the imaging element.

[0187] (5) The electronic device according to (4), wherein,

[0188] The information processing unit reads the fingerprint information during actions that include at least a finger sliding motion.

[0189] (6) The electronic device according to (5), wherein,

[0190] The electronic device also includes a touch panel that senses contact information with the display.

[0191] The speed of the sliding motion is estimated in the touch panel.

[0192] (7) The electronic device according to (5) or (6), wherein,

[0193] The information processing unit estimates the speed of the sliding motion based on the information read from the imaging element.

[0194] (8) The electronic device according to any one of (5) to (7), wherein,

[0195] Based on the required authentication accuracy, an indication is output to slow down the rate of the sliding motion.

[0196] (9) The electronic device according to (8), wherein,

[0197] The display outputs an indication of the rate of the sliding motion.

[0198] (10) The electronic device according to (8) or (9), wherein,

[0199] The instruction is displayed on the monitor as a rate guide.

[0200] (11) The electronic device according to any one of (8) to (10), wherein,

[0201] When the rate is faster than the predetermined rate, an output is made indicating that the rate is too fast.

[0202] (12) The electronic device according to (11), wherein,

[0203] The output of the condition of excessive speed is at least one of the output to the display, the output via sound, and the output via vibration.

[0204] (13) The electronic device according to (6), wherein,

[0205] When the rate is faster than a predetermined rate, the information processing unit shortens the exposure time of the optical fingerprint sensor.

[0206] (14) The electronic device according to any one of (5) to (13), wherein,

[0207] The information processing unit generates authentication information based on the fingerprint information captured at different times.

[0208] (15) The electronic device according to (14), wherein,

[0209] The light-emitting pixels output light of different wavelengths on one side of the display surface of the light-receiving element.

[0210] The light-receiving element acquires the fingerprint information based on reflected light of different wavelengths.

[0211] (16) The electronic device according to (14) or (15), wherein,

[0212] A polarizing filter is provided between the light-receiving element and the display surface.

[0213] The light-receiving element senses the light after it has been polarized by the polarizing filter.

[0214] (17) The electronic device according to any one of (14) to (16), wherein,

[0215] A filter for capturing the state of hemoglobin is provided between the light-receiving element and the display surface.

[0216] The information processing unit acquires the information of the hemoglobin to perform biological authentication.

[0217] (18) The electronic device according to any one of (14) to (17), wherein,

[0218] The information processing unit performs biometric authentication based on information about the shape changes of a finger that is in contact with the display surface over time.

[0219] (19) The electronic device according to any one of (5) to (18), wherein,

[0220] The light-receiving element detects the sliding motion of the multiple fingers.

[0221] (20) The electronic device according to (19), wherein,

[0222] The information processing unit uses a combination of multiple fingers to perform fingerprint authentication during the sliding motion of multiple fingers.

[0223] (21) The electronic device according to (20), wherein,

[0224] Depending on the required authentication accuracy, the combination of multiple fingers may vary.

[0225] (22) The electronic device according to any one of (4) to (21), wherein,

[0226] The information processing unit detects finger information and accumulates the fingerprint information during the authentication process or when authentication cannot be performed.

[0227] (23) The electronic device according to (22), wherein,

[0228] The information processing unit accumulates changes in the finger to improve authentication accuracy.

[0229] (24) The electronic device according to (22) or (23), wherein,

[0230] The information processing unit acquires and accumulates the fingerprint information from fingers other than those not registered.

[0231] (25) The electronic device according to any one of (5) to (24), wherein,

[0232] The light-receiving element is configured such that the number of elements in the direction intersecting the direction of the sliding motion is greater than the number of elements in the direction of the sliding motion.

[0233] (26) The electronic device according to (25), wherein,

[0234] The number of light-receiving elements in the direction intersecting the direction of the sliding motion is greater than twice the number of elements in the direction of the sliding motion.

[0235] (27) The electronic device according to (25) or (26), wherein,

[0236] The display shows a guide for performing the sliding action in a direction that intersects with the direction in which the light-receiving element is set.

[0237] (28) The electronic device according to any one of (5) to (27), wherein,

[0238] The electronic device has an interface that displays an area with the light-receiving element on the screen, and includes product information and a purchase button. The user can slide the product information from the display area to the purchase button to make a purchase.

[0239] Based on the authentication result using the fingerprint information, the purchase information is sent to the server.

[0240] (29) The electronic device according to any one of (1) to (27), wherein,

[0241] The display shows dynamic objects in a manner that includes the area set by the light-receiving element.

[0242] (30) The electronic device according to (29), wherein,

[0243] The object changes shape when touched by the user's finger.

[0244] (31) The electronic device according to (29), wherein,

[0245] The object's light emission state changes when touched by the user's finger.

[0246] (32) The electronic device according to (31), wherein,

[0247] The luminescence state changes in a manner suitable for acquiring the fingerprint information.

[0248] (33) The electronic device according to any one of (29) to (32), wherein,

[0249] The object changes dynamically based on the acquisition status of the fingerprint information or the personal authentication status after the user's finger passes through it.

[0250] (34) The electronic device according to any one of (1) to (33), wherein,

[0251] It also includes a tilt detection unit for detecting the horizontal tilt of the display.

[0252] The fingerprint information is authenticated based on the tilt detected by the tilt detection unit.

[0253] The methods disclosed herein are not limited to the embodiments described above, but also include various modifications that can be conceived by those skilled in the art, and the effects of this disclosure are not limited to the above content. That is, various additions, changes, and partial deletions can be made without departing from the concept and spirit of this disclosure derived from the content defined in the claims and their equivalents.

Claims

1. An electronic device, characterized in that, have: Displays; and Optical fingerprint sensor, The display has a display surface, on which light-emitting pixels are arranged in an array in a first direction and a second direction intersecting the first direction. The optical fingerprint sensor includes an imaging element. The imaging element has light-receiving elements arranged in an array along the first and second directions on the side opposite to the display surface of the display in a third direction intersecting the first and second directions. Each light-receiving element transmits charge obtained through photoelectric conversion at the same timing interval. The electronic device also includes an information processing unit, which performs information processing on the fingerprint information read by the imaging element. The information processing unit reads the fingerprint information when an action, including at least a finger swiping motion, is performed. The electronic device has an interface that displays an area with the light-receiving element on the screen, and includes product information and a purchase button. The user can slide the product information from the display area to the purchase button to make a purchase. Based on the authentication result using the fingerprint information, purchase information is sent to the server.

2. The electronic device according to claim 1, wherein, The imaging element has a pixel memory that temporarily stores the light-receiving information of each of the light-receiving elements, and the transmission of information from the light-receiving elements to the pixel memory is performed at the same time interval.

3. The electronic device according to claim 1, wherein, The electronic device also includes a touch panel for sensing contact information with the display. The speed of the sliding motion is estimated in the touch panel.

4. The electronic device according to claim 1, wherein, The information processing unit estimates the speed of the sliding motion based on the information read from the imaging element.

5. The electronic device according to claim 3, wherein, Based on the required authentication accuracy, an indication is output to slow down the sliding motion.

6. The electronic device according to claim 5, wherein, The display outputs an indication of the speed of the sliding motion.

7. The electronic device according to claim 6, wherein, When the sliding motion is too fast, the output is at least one of the following: output to the display, sound output, and vibration output.

8. The electronic device according to claim 3, wherein, When the speed of the sliding motion is faster than a predetermined speed, the information processing unit shortens the exposure time of the optical fingerprint sensor.

9. The electronic device according to claim 1, wherein, The information processing unit generates authentication information based on the fingerprint information captured at different times.

10. The electronic device according to claim 9, wherein, The light-emitting pixels output light of different wavelengths on one side of the display surface of the light-receiving element. The light-receiving element acquires the fingerprint information based on reflected light of different wavelengths.

11. The electronic device according to claim 9, wherein, A polarizing filter is provided between the light-receiving element and the display surface. The light-receiving element senses the light after it has been polarized by the polarizing filter.

12. The electronic device according to claim 9, wherein, A filter for capturing the state of hemoglobin is provided between the light-receiving element and the display surface. The information processing unit acquires the information of the hemoglobin to perform biological authentication.

13. The electronic device according to claim 9, wherein, The information processing unit performs biometric authentication based on information about the shape change of the finger in contact with the display surface over time.

14. The electronic device according to claim 1, wherein, The light-receiving element detects the sliding motion of the multiple fingers.

15. The electronic device according to claim 14, wherein, When multiple fingers perform the sliding action, the information processing unit performs authentication based on the fingerprint information using a combination of multiple fingers.

16. The electronic device according to claim 1, wherein, The information processing unit detects finger information and accumulates the fingerprint information during the authentication process or when authentication is not performed.

17. The electronic device according to claim 1, wherein, The light-receiving element is configured such that the number of elements in the direction intersecting the direction of the sliding action is greater than the number of elements in the direction of the sliding action.

Images