Imaging method and imaging device of fingerprint module

Abstract

A fingerprint module imaging method and imaging device, comprising: a light source assembly including multiple light sources; a sensing surface; and an image sensor; wherein the projection ranges of the multiple reflected lights on the image sensor at least partially overlap. The imaging method comprises: obtaining an image to be processed; separating the image to be processed; and obtaining a fingerprint image based on the separation results. Separating the image to be processed avoids averaging the light generated by multiple display pixels, effectively improving the quality of the obtained fingerprint image. Furthermore, processing the image to be processed in the overlapping area overcomes the limitations of the spacing between light sources, effectively improving the measurement efficiency of the entire fingerprint image. Furthermore, this fingerprint module imaging method and imaging device can utilize the module structure to encrypt the image, thereby enhancing the security of the fingerprint image.

Description

Technical Field

[0001] The present invention relates to the field of fingerprint imaging, and in particular to an imaging method and an imaging device for a fingerprint module. Background Art

[0002] Existing fingerprint recognition technology uses a fingerprint imaging module to capture a person's fingerprint image, then compares it with the existing fingerprint image information in the fingerprint recognition system to achieve identity verification. Due to its ease of use and the uniqueness of human fingerprints, fingerprint recognition technology has been widely used in various fields, such as security inspections such as public security bureaus and customs, building access control systems, and consumer products such as personal computers and mobile phones.

[0003] Among the fingerprint imaging modules used in fingerprint recognition technology, one is to collect the fingerprint image of the human body through an optical fingerprint imaging module: that is, the incident light is generated by a light source; the incident light is projected onto the surface of the finger, and is reflected by the finger to form reflected light with fingerprint information; the image sensor receives the reflected light and obtains the fingerprint image.

[0004] Fingerprint imaging modules are increasingly being used in mobile devices such as laptops, tablets, and mobile phones, enabling automatic unlocking and functionality. Fingerprint recognition in mobile phones is particularly widespread.

[0005] However, the presence of the fingerprint imaging module often hinders the improvement of the screen-to-body ratio of mobile devices, especially when the fingerprint imaging module is set on the front of the mobile phone. The setting of the fingerprint imaging module often affects the increase of the display device area.

[0006] To achieve both fingerprint unlocking and a higher screen-to-body ratio, under-display fingerprint modules integrate the fingerprint imaging module with the display module, utilizing the light generated by the display module for fingerprint imaging. However, this type of fingerprint module cannot simultaneously achieve both fingerprint image quality and measurement area efficiency. Summary of the Invention

[0007] The problem solved by the present invention is how to give consideration to both fingerprint image quality and effective imaging area.

[0008] To solve the above problems, the present invention provides a fingerprint image imaging method, comprising:

[0009] The fingerprint module includes: a light source component, which includes multiple light sources; a sensing surface, on which the light generated by the multiple light sources forms multiple corresponding reflected lights, each of which carries fingerprint information; an image sensor, which collects the multiple reflected lights to obtain a collected image; the projection ranges of the multiple reflected lights on the image sensor at least partially overlap; the imaging method includes: obtaining an image to be processed, which is the overlapping part of the images formed by the multiple reflected lights on the image sensor in the collected image; separating the image to be processed; and obtaining a fingerprint image based on the separation result.

[0010] Optionally, the light source assembly includes a first light source and a second light source; the light generated by the first light source forms a first reflected light carrying fingerprint information on the sensing surface, and the second light source forms a second reflected light carrying fingerprint information on the sensing surface; the image sensor collects the first reflected light and the second reflected light to obtain the collected image; in the step of obtaining the image to be processed, the image to be processed is the overlapping part of the image formed by the first reflected light and the image formed by the second reflected light in the collected image.

[0011] Optionally, the projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is a first known image, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is a second known image; the step of separating the image to be processed includes: taking one of the area corresponding to the first known image and the area corresponding to the second known image as the first zone, and the image corresponding to the first zone as the first determined image; according to the first zone, along the direction toward the image to be processed, dividing the area corresponding to the image to be processed into adjacent second zones, third zones, ..., Nth zones, wherein the shapes of the second zone, the third zone, ..., the Nth zone are the same as those of the first zone; according to the mth determined image, separating the image to be processed in the m+1th zone to obtain the m+1th determined image, wherein m is an integer in the range of 1 to N-1.

[0012] Optionally, the step of obtaining the m+1th determined image includes: obtaining the first layer of the m+1th area according to the mth determined image based on the relative positions of the first light source, the second light source, the sensing surface and the image sensor; obtaining the m+1th determined image according to the difference between the to-be-processed image of the m+1th area and the first layer of the m+1th area.

[0013] Optionally, in the step of obtaining the fingerprint image, the fingerprint image is obtained based on the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone and the first known image or the second known image.

[0014] Optionally, in the step of obtaining the fingerprint image, the fingerprint image is obtained based on the first determination image, the second determination image, the third determination image, ..., the Nth determination image.

[0015] Optionally, the step of separating the image to be processed includes: separating the image to be processed according to a preset separation model.

[0016] Optionally, the separation model is a pre-trained deep learning model.

[0017] Optionally, the light source includes a plurality of point light sources, and the projection ranges of the reflected lights formed by the plurality of point light sources in one light source on the sensing surface do not overlap on the image sensor.

[0018] Optionally, the multiple point light sources are arranged in an array.

[0019] Optionally, the light source assembly is a liquid crystal display, an active matrix organic light emitting diode display, or a light emitting diode display.

[0020] Correspondingly, the present invention also provides an imaging device for a fingerprint module,

[0021] The fingerprint module includes: a light source component, which includes multiple light sources; a sensing surface, on which the light generated by the multiple light sources forms multiple corresponding reflected lights, each of which carries fingerprint information; an image sensor, which collects the multiple reflected lights to obtain a collected image; the projection range of each of the multiple reflected lights on the image sensor at least partially overlaps; the imaging device includes: a processing module, which is suitable for obtaining an image to be processed, and the image to be processed is the overlapping part of the image formed by each of the multiple reflected lights on the image sensor in the collected image; the care module is also suitable for separating the image to be processed and obtaining a fingerprint image based on the separation result.

[0022] Optionally, the light source assembly includes a first light source and a second light source; the light generated by the first light source forms a first reflected light carrying fingerprint information on the sensing surface, and the second light source forms a second reflected light carrying fingerprint information on the sensing surface; the image sensor collects the first reflected light and the second reflected light to obtain the collected image; the image to be processed obtained by the processing module is the overlapping part of the image formed by the first reflected light and the image formed by the second reflected light in the collected image.

[0023] Optionally, the projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is a first known image, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is a second known image. The processing module includes: a partitioning unit, wherein the partitioning unit is suitable for taking one of the area corresponding to the first known image and the area corresponding to the second known image as the first area, and the image corresponding to the first area as the first determined image; the partitioning unit is also suitable for dividing the area corresponding to the image to be processed into adjacent second areas, third areas, ..., N areas along the direction toward the image to be processed according to the first area, wherein the shapes of the second area, the third area, ..., the N area are the same as the shape of the first area; a separation unit, wherein the separation unit separates the image to be processed in the m+1th area according to the mth determined image to obtain the m+1th determined image, wherein m is an integer in the range of 1 to N-1.

[0024] Optionally, the separation unit includes: a converter, which is suitable for obtaining the first layer of the m+1th area according to the mth determined image based on the relative positions of the first light source, the second light source, the sensing surface and the image sensor; and a separator, which is suitable for obtaining the m+1th determined image based on the difference between the image to be processed in the m+1th area and the first layer of the m+1th area.

[0025] Optionally, the processing module further includes: an imaging unit, which is suitable for obtaining the fingerprint image based on the first layer of the second zone, the first layer of the third zone,..., the first layer of the Nth zone and the first known image or the second known image.

[0026] Optionally, the processing module further includes: an imaging unit, which is suitable for obtaining the fingerprint image based on the first determined image, the second determined image, the third determined image, ..., the Nth determined image.

[0027] Optionally, a separation model is preset in the processing module, and the processing module separates the image to be processed according to the separation model.

[0028] Optionally, the separation model is a pre-trained deep learning model.

[0029] Optionally, the light source includes a plurality of point light sources, and the projection ranges of the reflected lights formed by the plurality of point light sources in one light source on the sensing surface do not overlap on the image sensor.

[0030] Optionally, the multiple point light sources are arranged in an array.

[0031] Optionally, the light source assembly is a liquid crystal display, an active matrix organic light emitting diode display, or a light emitting diode display.

[0032] Accordingly, the present invention also provides a training method for the separation model used in the present invention, which specifically includes:

[0033] A training fingerprint sample is provided, and the training fingerprint sample is used for imaging for training; the training fingerprint sample is imaged using light generated by a light source assembly to obtain a fused fingerprint image; the fused fingerprint image is separated using a separation model to obtain a plurality of separated fingerprint fragments corresponding to the plurality of light sources in the light source assembly; the training fingerprint is imaged using light generated by a single light source in the light source assembly to obtain independent fingerprint fragments corresponding to the light source of the generated light; the separated fingerprint fragments are compared with the corresponding independent fingerprint fragments to determine whether the obtained separated fingerprint fragments are correctly separated; when the separation is determined to be incorrect, the separation model is optimized, and the fused fingerprint image is separated again using the optimized separation model; when the separation is determined to be correct, one training step is completed.

[0034] Optionally, in the step of determining whether the obtained separated fingerprint fragments are correctly separated, the determination is performed according to a preset loss function.

[0035] Optionally, the fingerprint module includes a light source component, the light source component includes multiple light sources, the light source includes multiple point light sources, and all the point light sources in the light source component are traversed by the multiple light sources.

[0036] Optionally, in the step of separating the fused fingerprint image using a separation model, the initial separation model is a separation model established according to the structure of the fingerprint module.

[0037] Accordingly, the present invention also provides a training device for the separation model used in the present invention, comprising:

[0038] A fingerprint sample library, wherein the fingerprint sample library includes training fingerprint samples, wherein the training fingerprint samples are used for imaging for training; a fusion image unit, wherein the fusion image unit is suitable for controlling the light source component to generate light and performing fingerprint imaging on the training fingerprint samples using the light generated by the light source component to obtain a fused fingerprint image; an image separation unit, wherein a separation model is pre-set in the image separation unit, wherein the image separation unit is suitable for using the separation model to separate the fused fingerprint image and obtain a plurality of separated fingerprint fragments corresponding to the plurality of light sources in the light source component; an independent fragment unit, wherein the independent fragment unit is suitable for controlling the light sources in the light source component to generate light one by one and performing fingerprint imaging on the training fingerprint samples using the light generated by the light source component to obtain a fused fingerprint image; The light generated by a single light source in the light source assembly performs fingerprint imaging on the training fingerprint sample, thereby obtaining independent fingerprint fragments corresponding to the light source of the generated light; a comparison and judgment unit, wherein the comparison and judgment unit is suitable for comparing the separated fingerprint fragments with the corresponding independent fingerprint fragments to determine whether the obtained separated fingerprint fragments are correctly separated; when the comparison and judgment unit determines that the separation is correct, one training is completed; a model optimization unit, wherein the model optimization unit is suitable for optimizing the separation model in the image separation unit when the comparison and judgment unit determines that the separation is incorrect; the image separation unit is also suitable for using the optimized separation model to separate the fused fingerprint image again.

[0039] Optionally, it is characterized in that the comparison and judgment unit is further suitable for judging whether the obtained separated fingerprint fragments are correctly separated according to a preset loss function.

[0040] Optionally, it is characterized in that the initial separation model is a separation model established according to the structure of the fingerprint module.

[0041] Optionally, it is characterized in that the fingerprint module includes a light source component, the light source component includes multiple light sources, the light source includes multiple point light sources, and all the point light sources in the light source component are traversed by the multiple light sources.

[0042] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0043] The technical solution of the present invention separates the image to be processed to obtain multiple fingerprint images formed independently on the image sensor by reflected light. This separation avoids averaging the light generated by multiple display pixels, effectively improving the quality of the resulting fingerprint image. Furthermore, the processing of overlapping images can overcome the limitations of the spacing between light sources, effectively increasing the efficiency of measuring the entire fingerprint image area. Furthermore, this fingerprint module imaging method and imaging device can utilize the module structure to encrypt the image, thereby enhancing the security of the fingerprint image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the cross-sectional structure of a fingerprint module;

[0045] Figure 2 1 is a flow chart of an embodiment of an imaging method for a fingerprint module of the present invention;

[0046] Figure 3 yes Figure 2 A schematic structural diagram of a fingerprint module used in an embodiment of the fingerprint module imaging method is shown;

[0047] Figure 4 yes Figure 3 Schematic diagram of the distribution of the projection range of the first reflected light and the second reflected light in the fingerprint module;

[0048] Figure 5 yes Figure 2 Schematic diagram of the equivalent optical path of an embodiment of the fingerprint module imaging method shown;

[0049] Figure 6 yes Figure 2 Schematic diagram of the flow of steps S120 and S130 in the embodiment of the fingerprint model imaging method shown;

[0050] Figure 7 1 is a flow chart of a training method for a separation model used in another embodiment of the fingerprint model imaging method of the present invention;

[0051] Figure 8 yes Figure 7 A schematic diagram of the light spot structure of the light source assembly used in the embodiment of the fingerprint module imaging method is shown;

[0052] Figure 9 yes Figure 7 Schematic diagram of the light spot structure of the first light source, the second light source, the third light source and the fourth light source in the training method adopted in the embodiment of the fingerprint model imaging method;

[0053] Figure 10 yes Figure 2The functional block diagram of the processing module 140 in the embodiment of the fingerprint module imaging device is shown;

[0054] Figure 11 It is a functional block diagram of the training device of the separation model of the present invention. DETAILED DESCRIPTION

[0055] As can be seen from the background technology, the fingerprint module imaging method needs to be improved. Now, the reasons for the need for improvement are analyzed in combination with the structure of the fingerprint module and its imaging method:

[0056] The display module contains multiple display pixels, each capable of generating light for fingerprint imaging. One imaging method involves averaging the light generated by multiple display pixels, essentially treating them as a single surface light source, to obtain a fingerprint image. However, this imaging method suffers from high noise levels and severe crosstalk, resulting in poor fingerprint image quality.

[0057] Another imaging method is to regard one or more display pixels in the display module as a point light source for imaging to suppress noise signals and improve image quality.

[0058] refer to Figure 1 , which shows a schematic diagram of the cross-sectional structure of a fingerprint module.

[0059] The fingerprint module includes: a light source 11, an image sensor 12, and a sensing surface 13. When a finger is pressed on the sensing surface 13, the light generated by the light source 11 forms reflected light carrying fingerprint information on the sensing surface 13.

[0060] The critical angle at which the light emitted by the light source 11 is totally reflected on the sensing surface 13 is θ c At this time, the projection of the light emitted by the light source 11 on the sensing surface 13 is r. When the incident angle θ of the light emitted by the light source 11 on the sensing surface 13 is greater than θ c When a finger is pressed on the sensing surface 13, total reflection can occur on the sensing surface 13. When the finger is pressed on the sensing surface 13, the maximum radius of the contact area between the finger and the sensing surface 13 that can be detected by the light source 11 is R (R is usually related to the light intensity of the light source 11 and the distance between the light source 11 and the sensing surface 13). The area of ​​the maximum fingerprint area that can be detected by the light emitted by the light source 11 is 4πR 2When the projection of the light emitted by the light source 11 on the sensing surface 13 is less than r, on the one hand, when the light intensity of the light source 11 itself is large, the light source 11 forms a bright spot with a certain area on the corresponding receiving area of ​​the image sensor 12. On the other hand, since the light emitted by the light source 11 cannot be totally reflected on the sensing surface 13 at this time, compared with the area where the projection of the light emitted by the light source 11 on the sensing surface 13 is greater than or equal to r, the fingerprint information contained in the image formed after the light is received by the image sensor 12 in the area with the incident point on the sensing surface 13 and within a radius of r with the projection of the light source 11 as the center is much weaker. Therefore, the effective contact area of ​​the fingerprint information that the light source 11 can detect is π(R 2 -r 2 ), the efficiency of the measured area in the fingerprint image obtained by the fingerprint module is: π(R 2 -r 2 ) / 4πR 2 =[1-(r / R) 2 ] / 4, it can be seen that the limit of the measurement area efficiency of the fingerprint module is 25%, that is, the measurement area efficiency of the fingerprint module with a single point light source will not exceed 25%.

[0061] Since the measurement area efficiency limit of a single point light source fingerprint module is only 25%, in the solution of the under-screen fingerprint module that uses display pixels as point light sources, multiple arrayed point light sources are generally used to form a light source array to increase the amount of information obtained in the fingerprint image. However, in this structure, the distance between the point light sources in the light source array cannot be too close, otherwise the images formed by adjacent point light sources are likely to overlap, thereby affecting the acquisition of the fingerprint image. Under the premise of the same fingerprint image size, the limitation of the spacing between the point light sources and the limit of the measurement area efficiency of a single point light source fingerprint module will affect the measurement area efficiency of the entire fingerprint image.

[0062] To solve the technical problem, the present invention provides an imaging method for a fingerprint module, wherein the fingerprint module comprises: a light source component, wherein the light source component comprises multiple light sources; a sensing surface, wherein the light generated by the multiple light sources forms multiple corresponding reflected lights on the sensing surface, and each of the reflected lights carries fingerprint information; an image sensor, wherein the image sensor collects the multiple reflected lights to obtain a collected image; the projection ranges of the multiple reflected lights on the image sensor at least partially overlap; the imaging method comprises: obtaining an image to be processed, wherein the image to be processed is the overlapping portion of the images formed by the multiple reflected lights on the image sensor in the collected image; separating the image to be processed; and obtaining a fingerprint image based on the separation result.

[0063] The technical solution of the present invention obtains fingerprint images by a separation approach. On the one hand, it avoids averaging the light generated by multiple display pixels, which can effectively improve the quality of the obtained fingerprint image; on the other hand, the processing of the to-be-processed images in the overlapping area can break through the limitation of the distance between light sources, which can effectively improve the measurement area efficiency of the entire fingerprint image.

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0065] refer to Figure 2 and Figure 3 ,in Figure 2 A schematic diagram showing a flow chart of an embodiment of an imaging method for a fingerprint module of the present invention; Figure 3 yes Figure 2 FIG. 1 is a schematic structural diagram of a fingerprint module used in an embodiment of a fingerprint module imaging method.

[0066] The fingerprint module includes: a light source assembly 110, which includes multiple light sources; a sensing surface 120, on which the light generated by the multiple light sources forms multiple corresponding reflected lights, each of which carries fingerprint information; an image sensor 130, which collects the multiple reflected lights to obtain a captured image; the projection range of each of the multiple reflected lights on the image sensor 130 at least partially overlaps.

[0067] In some embodiments, the light source may be a point matrix light source, each point matrix light source including a plurality of discrete point light sources.

[0068] like Figure 3 As shown, in this embodiment, the light source assembly 110 includes a first light source 111a and a second light source 111b; the light generated by the first light source 111a forms a first reflected light carrying fingerprint information on the sensing surface 120, and the second light source 111b forms a second reflected light carrying fingerprint information on the sensing surface 120; the image sensor 130 collects the first reflected light and the second reflected light to obtain a collected image.

[0069] In some embodiments of the present invention, the light source includes a plurality of point light sources, and the reflected light formed by the plurality of point light sources in one light source on the sensing surface has no overlapping projection range on the image sensor. The plurality of point light sources are arranged in an array. Figure 3As shown, in this embodiment, the light source assembly is an active matrix organic light-emitting diode (OLED) display, and the point light sources are display pixels of the active matrix organic light-emitting diode display. The first light source 111a includes some of the display pixels; the second light source 111b includes some of the display pixels. In other embodiments of the present invention, the light source can also be a liquid crystal display or a light-emitting diode display.

[0070] The spacing between the multiple light sources is small, so the projection ranges of the multiple reflected lights on the image sensor at least partially overlap. Figure 3 As shown, the distance between the first light source 111a and the second light source 111b is small, so the projection range of the first reflected light on the image sensor and the projection range of the second reflected light on the image sensor at least partially overlap, that is, the distance between the point light source of the first light source 111a and the point light source of the second light source 111b is small, that is, Figure 4 As shown, in the collected image, there is an overlapping area 110c between the image 112a formed by the first reflected light and the image 112b formed by the second reflected light.

[0071] In one of the light sources, the spacing between the multiple point light sources is large enough so that the projection range of the reflected light formed by each of the point light sources in the same light source on the image sensor does not overlap, that is, the image formed by a single light source on the image sensor does not have an image overlapping area.

[0072] Specifically, such as Figure 4 As shown, in the captured image, the spacing between all the point light sources in the first light source 111a is large enough so that there is no overlapping area between the images 112a formed by the first reflected light; the spacing between all the point light sources in the second light source 111b is large enough so that there is no overlapping area between the images 112b formed by the second reflected light.

[0073] like Figure 2 As shown, the imaging method includes: executing step S110 to obtain an image to be processed, where the image to be processed is the overlapping portion of the image formed by the multiple reflected lights on the image sensor in the captured image; executing step S120 to separate the image to be processed; and then executing step S130 to obtain a fingerprint image based on the separation result.

[0074] The fingerprint image is obtained by separating the pixels, which, on the one hand, avoids averaging the light generated by multiple display pixels, and can effectively improve the quality of the obtained fingerprint image. On the other hand, the processing of the image to be processed in the overlapping area can break through the limitation of the distance between the light sources, and can effectively improve the measurement area efficiency of the entire fingerprint image.

[0075] In this embodiment, Figure 4 As shown, the image to be processed is the overlapping portion 110c of the image 112a formed by the first reflected light and the image 112b formed by the second reflected light in the collected image.

[0076] It should be noted that, in this embodiment, the projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is the first known image 113a, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is the second known image 113b.

[0077] Combined with reference Figure 5 , Figure 5 Shown Figure 2 The equivalent optical path diagram of an embodiment of the fingerprint module imaging method is shown. It should be noted that, to facilitate explanation and clearly convey the design concept of the technical solution of the present invention, the equivalent optical path diagram is used as an auxiliary explanation. In the equivalent optical path diagram, the fingerprint image captured by the image sensor is mirrored along the sensing surface to facilitate understanding of the overlapping relationship between the projection ranges of reflected light from different light sources on the image sensor.

[0078] Combined with reference Figure 2 and Figure 5 The imaging method includes: first performing step S110 to obtain an image to be processed. In this embodiment, the image to be processed is a portion of the captured image where the image formed by the first reflected light and the image formed by the second reflected light overlap.

[0079] like Figure 5As shown, in this embodiment, the light generated by the first light source 114a and the second light source 114b are respectively projected onto the fingerprint 121 of the sensing surface to form a first reflected light 115a and a second reflected light 115b; the image sensor collects the first reflected light 115a and the second reflected light 115b to obtain a collected image, and the collected image includes a to-be-processed image 116, a first known image 117a and a second known image 117b, respectively. The first known image 117a is a portion of the image formed by the first reflected light 115a that does not overlap with the image formed by the second reflected light 115b, and the second known image 117b is a portion of the image formed by the second reflected light 115b that does not overlap with the image formed by the first reflected light 115a.

[0080] After obtaining the image to be processed 116 , step S120 is executed to separate the image to be processed.

[0081] Combined with reference Figure 6 , Figure 6 Shown Figure 2 FIG. 1 is a flow chart of steps S120 and S130 in an embodiment of a fingerprint module imaging method.

[0082] Specifically, in step S120, the step of separating the image to be processed includes:

[0083] First, step S121 is executed, where one of the area corresponding to the first known image 117a and the area corresponding to the second known image 117b is used as the first area, and the image corresponding to the first area is used as the first determined image.

[0084] Specifically, in this embodiment, the area corresponding to the first known image 117a is used as the first zone, and accordingly, the first determined image is the first known image 117a. In other embodiments of the present invention, the area corresponding to the second known image may also be used as the first zone, and accordingly, the second determined image is the second known image.

[0085] After obtaining the first zone and the first determined image, step 122 is executed to divide the area corresponding to the image to be processed into adjacent second zones, third zones, ..., and Nth zones along the direction toward the image to be processed, based on the first zone, wherein the shapes of the second zone, the third zone, ..., and the Nth zone are the same as the shape of the first zone.

[0086] In this embodiment, the area corresponding to the first known image 117a is taken as the first zone, so the overlapping area corresponding to the image to be processed 116 is divided into the second zone, the third zone, ..., the Nth zone, and the second zone, the third zone, ..., the Nth zone are arranged adjacent to each other in sequence along the direction Z pointing from the first zone to the overlapping area, and the shapes of the second zone, the third zone, ..., the Nth zone are the same as that of the first zone.

[0087] It should be noted that the second region, the third region, ..., and the Nth region being adjacent means that the adjacent boundaries among the second region, the third region, ..., and the Nth region are common boundaries.

[0088] In other embodiments of the present invention, when the area corresponding to the second known image 117b is used as the first area, the direction of the second area, the third area, ..., the Nth area along the first area toward the overlapping area is opposite to the Z direction shown in the figure.

[0089] After the image to be processed is partitioned, step S123 is executed to separate the image to be processed in the m+1th region according to the mth determination image to obtain the m+1th determination image, where m is an integer in the range of 1 to N-1.

[0090] Specifically, the step of separating the image to be processed in the m+1th area to obtain the m+1th determination image includes: first, based on the first light source 114a, the second light source 114b, the sensing surface (i.e. Figure 5 The fingerprint 121 on the sensing surface and the relative position of the image sensor are determined based on the mth determined image to obtain the first layer of the m+1th region. The m+1th determined image is obtained based on the difference between the image to be processed in the m+1th region and the first layer of the m+1th region. The first layer of the m+1th region is the image formed by the second reflected light 115b alone in the m+1th region.

[0091] Since the relative positions of the first light source 114a, the second light source 114b, the sensing surface, and the image sensor are fixed, if the partial fingerprint image (i.e., image) obtained by one of the first and second light sources 114a, 114b is known, then based on the imaging principle of rectilinear light propagation, the image of the portion of the fingerprint (i.e., object) corresponding to that partial fingerprint image on the image sensor can be obtained under the illumination of the other light source. In other words, if the partial image obtained by one of the first and second reflected lights is known, then based on the imaging principle of rectilinear light propagation, the partial image obtained by the other reflected light corresponding to that partial fingerprint image can be obtained. Since a fingerprint is a single image, the image corresponding to each light source is single, and light source translation will cause image translation. The image formed by the second light source 114b can be viewed as an upward translation of the image of the first light source 114a. Therefore, in the overlapping image area, to resolve the image formed by each light source, it is sufficient to know the partial image of either light source (i.e., area 1) where no overlap occurs.

[0092] For example, Figure 5 As shown, since the first determined image in the first zone, i.e., the first known image, is the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light, in other words, the first determined image is the image formed by the first reflected light alone on the image sensor. This portion of the fingerprint image does not overlap and is known with certainty. Therefore, based on the first determined image and in conjunction with the imaging principle of rectilinear light propagation, the image formed by the portion of the fingerprint corresponding to the first determined image on the image sensor under illumination by the second light source 114b can be analytically obtained (i.e., accurately obtained). Furthermore, the position on the image sensor where the image of the portion of the fingerprint corresponding to the first determined image is formed under illumination by the second light source 114b corresponds to the second zone. Therefore, the image formed by the portion of the fingerprint corresponding to the first determined image under illumination by the second light source 114b is the fingerprint image formed by the second light source 114b in the second zone, i.e., the fingerprint image formed by the second reflected light in the second zone, i.e., the first layer of the second zone.

[0093] On the other hand, the image to be processed corresponding to the second area is a linear superposition of the fingerprint image formed by the first reflected light alone in the second area and the fingerprint image formed by the second reflected light alone in the second area. That is to say, one layer in the image to be processed in the second area is the first layer of the second area; therefore, the difference between the image to be processed in the second area and the first layer of the second area is the fingerprint image formed by the first reflected light alone in the second area.

[0094] The first determination image is a first known image formed by the first reflected light alone on the image sensor. Correspondingly, the fingerprint image formed by the first reflected light alone in the second area on the image sensor is a second determination image.

[0095] Similarly, after obtaining the second definitive image, the fingerprint image formed by the second reflected light alone in the third region, i.e., the first layer of the third region, can be analytically obtained based on the second definitive image, and thus the third definitive image can be obtained. After obtaining the third definitive image, the fingerprint image formed by the second reflected light alone in the fourth region, i.e., the first layer of the fourth region, can be analytically obtained based on the third definitive image. And so on and so forth. After obtaining the (N-1)th definitive image, the first layer of the second reflected light alone in the Nth region can be analytically obtained based on the (N-1)th definitive image, and thus the Nth definitive image can be obtained. It can be seen that the (m+1)th definitive image is the image formed by the first reflected light alone in the (m+1)th region on the image sensor.

[0096] At this point, the image to be processed in the m+1th area is divided into the m+1th determined image formed by the first reflected light alone in the m+1th area and the first layer of the m+1th area formed by the second reflected light alone in the m+1th area; that is, the image to be processed in the m+1th area is separated.

[0097] After the image to be processed in the m+1th area is separated, continue to refer to Figure 2 , execute step S130 to obtain a fingerprint image according to the separation result.

[0098] Combined with reference Figure 6 In the step S130 of obtaining a fingerprint image according to the separation result, step S131 is executed to obtain a fingerprint image according to the first determination image, the second determination image, the third determination image, ..., the Nth determination image.

[0099] Specifically, the m+1th determined image is the image formed by the first reflected light alone in the m+1th area. Therefore, by splicing the first determined image, the second determined image, the third determined image, ..., the Nth determined image, the fingerprint image formed by the first reflected light alone can be obtained, that is, the fingerprint image.

[0100] Since the first layer of the m+1th area is the image formed by the second reflected light alone in the m+1th area, and the second known image is the image formed by the second reflected light alone, in some other embodiments of the present invention, the step of obtaining the fingerprint image can also be obtained based on the first layer of the second area, the first layer of the third area, ..., the first layer of the Nth area and the second known image 117b.

[0101] In this embodiment, the area corresponding to the first known image 117a is used as the first zone, and the first determined image is the first known image 117a. Therefore, the first determined image, the second determined image, the third determined image, ..., and the Nth determined image are all images formed solely by the first reflected light, and the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the second known image 117b are all images formed solely by the second reflected light. Therefore, the fingerprint image can be obtained by splicing the first determined image, the second determined image, the third determined image, ..., and the Nth determined image, or by splicing the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the second known image 117b.

[0102] In other embodiments of the present invention, the area corresponding to the second known image is used as the first zone, and the first determined image is used as the second known image. Therefore, the first determined image, the second determined image, the third determined image, ..., and the Nth determined image are all images formed solely by the second reflected light, and the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the first known image are all images formed solely by the first reflected light. In this case, the fingerprint image can be obtained by splicing the first determined image, the second determined image, the third determined image, ..., and the Nth determined image, or by splicing the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the first known image.

[0103] It should be noted that, in this embodiment, the light source assembly only includes the first light source and the second light source, so it is only necessary to separate the image to be processed in which the image formed by the first reflected light and the image formed by the second reflected light overlap to obtain the fingerprint image.

[0104] In other embodiments of the present invention, the light source assembly may further include two or more light sources, and the two or more light sources respectively form a plurality of reflected lights carrying fingerprint information on the sensing surface; the image sensor collects the plurality of reflected lights to obtain a collected image; the projection ranges of the plurality of reflected lights on the image sensor at least partially overlap, and a method similar to the aforementioned embodiment of the present invention can still be used to separate the image to be processed, and obtain a fingerprint image based on the separation result.

[0105] Since the fingerprint module does not contain nonlinear optical elements, the overlapping parts of the image obtained by the image sensor are all linear combinations of images formed by reflected light from different light sources. They can be separated by removing the image formed by a single reflected light from the image of the overlapping part, thereby obtaining a high-quality fingerprint image.

[0106] It should also be noted that, in this embodiment, the light source includes a plurality of point light sources arranged in an array; and the separation of the image to be processed is closely related to the specific structure of the fingerprint module, that is, the acquisition of the fingerprint image is related to the structure of the fingerprint module. This setting is equivalent to encrypting the fingerprint image through the specific structure of the fingerprint module, which can effectively improve the security of the fingerprint image.

[0107] The method of separating the image to be processed to obtain an accurate fingerprint image by analyzing it based on the imaging principle of rectilinear light propagation is only an example. In other embodiments of the present invention, a pre-trained deep learning model can also be used to separate the image to be processed. Specifically, the step of separating the image to be processed includes: separating the image to be processed according to a preset separation model.

[0108] In some embodiments of the present invention, the separation model may be a pre-trained deep learning model.

[0109] Combined with reference Figure 7 , which shows a flow chart of a training method for a separation model adopted in another embodiment of the fingerprint model imaging method of the present invention.

[0110] First, step S001 is executed to provide a fingerprint module.

[0111] In this embodiment, the light source assembly of the fingerprint module includes four light sources, namely a first light source, a second light source, a third light source and a fourth light source; the light generated by the four light sources forms four corresponding reflected lights on the sensing surface, namely a first reflected light, a second reflected light, a third reflected light and a fourth reflected light; the first light source, the second light source, the third light source and the fourth light source are spaced relatively small, so that the projection range of the first reflected light, the second reflected light, the third reflected light and the fourth reflected light on the image sensor has an overlapping part.

[0112] Combined with reference Figure 8 and Figure 9 ,in Figure 8 yes Figure 7 The schematic diagram of the light spot structure of the light source assembly used in the embodiment of the fingerprint module imaging method is shown. Figure 9 Shown respectively Figure 7Schematic diagram of the light spot structure of the first light source, the second light source, the third light source and the fourth light source in the light source assembly used in the embodiment of the fingerprint module imaging method.

[0113] Similarly, in this embodiment, the four light sources in the light source assembly are all dot matrix light sources. Each dot matrix light source includes multiple discrete point light sources. The spacing between adjacent dot matrix light sources is small, and the images formed on the image sensor by the reflected light generated by adjacent dot matrix light sources have overlapping areas. The distance between the multiple discrete point light sources in a dot matrix light source is large, and the images formed on the image sensor by the reflected light generated by each point light source do not overlap. In some embodiments, the dot matrix light source can include multiple discrete display pixel regions in a display panel, each display pixel region including one or more display pixels, and the multiple display pixel regions are separated by non-luminous pixels.

[0114] It should be noted that, in order to ensure the accuracy of the training results, all point light sources in the light source assembly are traversed by the multiple dot matrix light sources, that is, any point light source in the light source assembly is included in at least one dot matrix light source. In other embodiments, the light source assembly may also include other numbers of dot matrix light sources.

[0115] Next, step S002 is performed to provide a training fingerprint sample, which is used for imaging for training.

[0116] Next, step S003 is executed to perform fingerprint imaging on the training fingerprint sample using the light generated by the light source assembly to obtain a fused fingerprint image.

[0117] Specifically, the training fingerprint sample is imaged using light simultaneously generated by all light sources in the light source assembly, thereby obtaining a fused fingerprint image having overlapping areas. In this embodiment, the training fingerprint sample is imaged using light simultaneously generated by four light sources in the light source assembly to obtain the fused fingerprint image.

[0118] Thereafter, step S004 is executed to separate the fused fingerprint image using a separation model to obtain a plurality of separated fingerprint fragments corresponding one-to-one to the plurality of light sources in the light source assembly.

[0119] Specifically, the light source component of the fingerprint module includes four light sources, namely, a first light source, a second light source, a third light source and a fourth light source; therefore, after separating the fused fingerprint, the first separated fingerprint fragment corresponding to the first light source, the second separated fingerprint fragment corresponding to the second light source, the third separated fingerprint fragment corresponding to the third light source and the fourth separated fingerprint fragment corresponding to the fourth light source are obtained respectively.

[0120] It should be noted that the initial separation model can be established based on the structure of the fingerprint module.

[0121] Step S005 is executed to perform fingerprint imaging on the training fingerprint using the light generated by a single light source in the light source assembly, thereby obtaining independent fingerprint fragments corresponding to the light source that generates the light.

[0122] Specifically, the light source component of the fingerprint module includes multiple light sources, and each time one of the multiple light sources is lit up to image the training fingerprint sample to obtain an independent fingerprint fragment corresponding to the lit light source.

[0123] In this embodiment, the light source assembly of the fingerprint module includes four light sources, namely a first light source, a second light source, a third light source and a fourth light source; only one of the four light sources is lit each time to image the training fingerprint sample, and a first independent fingerprint fragment corresponding to the first light source, a second independent fingerprint fragment corresponding to the second light source, a third independent fingerprint fragment corresponding to the third light source, and a fourth independent fingerprint fragment corresponding to the fourth light source are obtained respectively.

[0124] After the separated fingerprint fragments and the independent fingerprint fragments are obtained, step S006 is executed to compare the separated fingerprint fragments with the corresponding independent fingerprint fragments to determine whether the separated fingerprint fragments are correctly separated.

[0125] Specifically, the difference between the separated fingerprint fragments and the corresponding independent fingerprint fragments is determined based on a preset loss function. If the separation is determined to be incorrect, step S007 is executed to adjust the parameters of the separation model to optimize the separation model. After obtaining the optimized separation model, step S004 is repeated, and the fused fingerprint image is separated using the optimized separation model to re-obtain multiple separated fingerprint fragments corresponding to the multiple light sources in the light source assembly. If the separation is determined to be correct in step S006, one training cycle is completed.

[0126] The separation model is trained by using different training fingerprint samples in the training set or fingerprint sample library to improve the separation model. It should be noted that in some embodiments of the present invention, in actual training, techniques such as gradient penalty and spectral normalization can be used to improve training efficiency and training accuracy.

[0127] It should be noted that in this embodiment, the number of light sources in the light source assembly is four. However, this is merely an example. In other embodiments of the present invention, the number of light sources included in the light source assembly can be in the range of two to ten, thereby improving the measurement area efficiency of the entire fingerprint image.

[0128] The number of light sources in the light source assembly is equal to the number of independent fingerprint fragments obtained. The greater the number of independent fingerprint fragments obtained, the more complete the fingerprint range corresponding to the multiple separate fingerprint fragments. Therefore, the number of light sources in the light source assembly can represent the efficiency of the fingerprint module in extracting fingerprint information within a fixed area. However, the greater the number of light sources in the light source assembly, the more information the fingerprint image obtained will have, and the more complete and accurate the fingerprint image will be. However, an increase in the number of light sources, that is, an increase in the number of independent fingerprint fragments, will result in a rapid increase in the amount of computation. Therefore, in some embodiments, the number of light sources in the light source assembly is 4 to 6, for example 5, to obtain sufficient eigenvalues ​​for fingerprint comparison.

[0129] Correspondingly, the present invention also provides an imaging device for a fingerprint module.

[0130] refer to Figure 3 , shows a structural schematic diagram of the imaging device of the fingerprint imaging module.

[0131] The fingerprint module includes: a light source assembly 110, which includes multiple light sources; a sensing surface 120, on which the light generated by the multiple light sources forms multiple corresponding reflected lights, each of which carries fingerprint information; an image sensor 130, which collects the multiple reflected lights to obtain a captured image; the projection range of each of the multiple reflected lights on the image sensor 130 at least partially overlaps.

[0132] like Figure 3 As shown, in this embodiment, the light source assembly 110 includes a first light source 111a and a second light source 111b; the light generated by the first light source 111a forms a first reflected light carrying fingerprint information on the sensing surface 120, and the second light source 111b forms a second reflected light carrying fingerprint information on the sensing surface 120; the image sensor 130 collects the first reflected light and the second reflected light to obtain a collected image.

[0133] In some embodiments of the present invention, the light source includes a plurality of point light sources, and the reflected light formed by the plurality of point light sources in one light source on the sensing surface has no overlapping projection range on the image sensor. The plurality of point light sources are arranged in an array. Figure 3 As shown, in this embodiment, the light source assembly is an active matrix organic light-emitting diode (OLED) display, and the point light sources are display pixels of the active matrix organic light-emitting diode display. The first light source 111a includes some of the display pixels; the second light source 111b includes some of the display pixels. In other embodiments of the present invention, the light source can also be a liquid crystal display (LCD) or an LED display.

[0134] The spacing between the multiple light sources is small, so the projection ranges of the multiple reflected lights on the image sensor at least partially overlap. Figure 3 As shown, the distance between the first light source 111a and the second light source 111b is small, so the projection range of the first reflected light on the image sensor and the projection range of the second reflected light on the image sensor at least partially overlap, that is, the distance between the point light source of the first light source 111a and the point light source of the second light source 111b is small, that is, Figure 4 As shown, in the collected image, there is an overlapping area 110c between the image 112a formed by the first reflected light and the image 112b formed by the second reflected light.

[0135] In one of the light sources, the spacing between the multiple point light sources is large enough so that the projection range of the reflected light formed by each of the point light sources in the same light source on the image sensor does not overlap, that is, the image formed by a single light source on the image sensor does not have an image overlapping area.

[0136] Specifically, such as Figure 4 As shown, in the captured image, the spacing between all the point light sources in the first light source 111a is large enough so that there is no overlapping area between the images 112a formed by the first reflected light; the spacing between all the point light sources in the second light source 111b is large enough so that there is no overlapping area between the images 112b formed by the second reflected light.

[0137] like Figure 3 As shown, the imaging device includes: a processing module 140, which is suitable for obtaining an image to be processed, wherein the image to be processed is the overlapping part of the image formed by the multiple reflected lights on the image sensor in the collected image; the care module 140 is also suitable for separating the image to be processed and obtaining a fingerprint image based on the separation result.

[0138] The fingerprint image is obtained by separating the pixels, which, on the one hand, avoids averaging the light generated by multiple display pixels, and can effectively improve the quality of the obtained fingerprint image. On the other hand, the processing of the image to be processed in the overlapping area can break through the limitation of the distance between the light sources, and can effectively improve the measurement area efficiency of the entire fingerprint image.

[0139] In this embodiment, Figure 4 As shown, the image to be processed is the overlapping portion 110c of the image 112a formed by the first reflected light and the image 112b formed by the second reflected light in the collected image.

[0140] It should be noted that, in this embodiment, the projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is the first known image 113a, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is the second known image 113b.

[0141] Combined with reference Figure 5 , showing Figure 3 It should be noted that, in order to facilitate the description and clearly express the design concept of the technical solution of the present invention, the equivalent optical path schematic diagram is used for auxiliary explanation.

[0142] The processing module 140 is adapted to obtain an image to be processed. In this embodiment, the image to be processed is a portion of the collected image where the image formed by the first reflected light and the image formed by the second reflected light overlap.

[0143] like Figure 5 As shown, in this embodiment, the light generated by the first light source 114a and the second light source 114b are respectively projected onto the fingerprint 121 of the sensing surface to form a first reflected light 115a and a second reflected light 115b; the image sensor collects the first reflected light 115a and the second reflected light 115b to obtain a collected image, and the collected image includes a to-be-processed image 116, a first known image 117a and a second known image 117b, respectively. The first known image 117a is a portion of the image formed by the first reflected light 115a that does not overlap with the image formed by the second reflected light 115b, and the second known image 117b is a portion of the image formed by the second reflected light 115b that does not overlap with the image formed by the first reflected light 115a.

[0144] Combined with reference Figure 10 , Figure 10 Shown Figure 2 FIG. 1 is a functional block diagram of the processing module 140 in the embodiment of the fingerprint module imaging device.

[0145] Specifically, the processing module 140 includes: a partitioning unit 141, which is suitable for using one of the area corresponding to the first known image and the area corresponding to the second known image as the first area, and the image corresponding to the first area as the first determined image.

[0146] Specifically, in this embodiment, the area corresponding to the first known image 117a is used as the first zone, and accordingly, the first determined image is the first known image 117a. In other embodiments of the present invention, the area corresponding to the second known image may also be used as the first zone, and accordingly, the second determined image is the second known image.

[0147] In addition, the partitioning unit 141 is also suitable for dividing the area corresponding to the image to be processed into adjacent second areas, third areas,..., and Nth areas according to the first area along the direction toward the image to be processed, wherein the shapes of the second area, the third area,..., and the Nth area are the same as the shape of the first area.

[0148] In this embodiment, the area corresponding to the first known image 117a is taken as the first zone, so the overlapping area corresponding to the image to be processed 116 is divided into the second zone, the third zone, ..., the Nth zone, and the second zone, the third zone, ..., the Nth zone are arranged adjacent to each other in sequence along the direction Z pointing from the first zone to the overlapping area, and the shapes of the second zone, the third zone, ..., the Nth zone are the same as that of the first zone.

[0149] It should be noted that the second region, the third region, ..., and the Nth region being adjacent means that the adjacent boundaries among the second region, the third region, ..., and the Nth region are common boundaries.

[0150] In other embodiments of the present invention, when the area corresponding to the second known image 117b is used as the first area, the direction of the second area, the third area, ..., the Nth area along the first area toward the overlapping area is opposite to the Z direction shown in the figure.

[0151] The processing module 140 further includes a separation unit 142 , which separates the image to be processed in the (m+1)th region according to the (m)th determination image to obtain the (m+1)th determination image, where m is an integer ranging from 1 to N−1.

[0152] Specifically, the separation unit 142 includes: a converter 142a, adapted to obtain a first layer of the m+1th region based on the mth determined image based on the relative positions of the first light source, the second light source, the sensing surface, and the image sensor; and a separator 142b, adapted to obtain an m+1th determined image based on the difference between the image to be processed in the m+1th region and the first layer of the m+1th region. The first layer of the m+1th region is an image formed by the second reflected light 115b alone in the m+1th region.

[0153] Since the relative positions of the first light source 114a, the second light source 114b, the sensing surface, and the image sensor are fixed, if the partial fingerprint image (i.e., image) obtained by one of the first and second light sources 114a, 114b is known, then based on the imaging principle of rectilinear light propagation, the image of the portion of the fingerprint (i.e., object) corresponding to that partial fingerprint image on the image sensor can be obtained under the illumination of the other light source. In other words, if the partial image obtained by one of the first and second reflected lights is known, then based on the imaging principle of rectilinear light propagation, the partial image obtained by the other reflected light corresponding to that partial fingerprint image can be obtained. Since a fingerprint is a single image, the image corresponding to each light source is single, and light source translation will cause image translation. The image formed by the second light source 114b can be viewed as an upward translation of the image of the first light source 114a. Therefore, in the overlapping image area, to resolve the image formed by each light source, it is sufficient to know the partial image of either light source (i.e., area 1) where no overlap occurs.

[0154] For example, Figure 5 As shown, since the first determined image in the first zone, i.e., the first known image, is the portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light, in other words, the first determined image is the image formed by the first reflected light alone on the image sensor. This portion of the fingerprint image does not overlap and is known with certainty. Therefore, based on the first determined image and in conjunction with the imaging principle of rectilinear light propagation, the image formed by the portion of the fingerprint corresponding to the first determined image on the image sensor under illumination by the second light source 114b can be analytically obtained (i.e., accurately obtained). Furthermore, the position on the image sensor where the image of the portion of the fingerprint corresponding to the first determined image is formed under illumination by the second light source 114b corresponds to the second zone. Therefore, the image formed by the portion of the fingerprint corresponding to the first determined image under illumination by the second light source 114b is the fingerprint image formed by the second light source 114b in the second zone, i.e., the fingerprint image formed by the second reflected light in the second zone, i.e., the first layer of the second zone.

[0155] On the other hand, the image to be processed corresponding to the second area is a linear superposition of the fingerprint image formed by the first reflected light alone in the second area and the fingerprint image formed by the second reflected light alone in the second area. That is to say, one layer in the image to be processed in the second area is the first layer of the second area; therefore, the difference between the image to be processed in the second area and the first layer of the second area is the fingerprint image formed by the first reflected light alone in the second area.

[0156] The first determination image is a first known image formed by the first reflected light alone on the image sensor. Correspondingly, the fingerprint image formed by the first reflected light alone in the second area on the image sensor is a second determination image.

[0157] Similarly, after obtaining the second definitive image, the fingerprint image formed by the second reflected light alone in the third region, i.e., the first layer of the third region, can be analytically obtained based on the second definitive image, and thus the third definitive image can be obtained. After obtaining the third definitive image, the fingerprint image formed by the second reflected light alone in the fourth region, i.e., the first layer of the fourth region, can be analytically obtained based on the third definitive image. And so on and so forth. After obtaining the (N-1)th definitive image, the first layer of the second reflected light alone in the Nth region can be analytically obtained based on the (N-1)th definitive image, and thus the Nth definitive image can be obtained. It can be seen that the (m+1)th definitive image is the image formed by the first reflected light alone in the (m+1)th region on the image sensor.

[0158] At this point, the image to be processed in the m+1th area is divided into the m+1th determined image formed by the first reflected light alone in the m+1th area and the first layer of the m+1th area formed by the second reflected light alone in the m+1th area; that is, the separator 142b realizes the separation of the image to be processed in the m+1th area.

[0159] After the separator 142b separates the image to be processed in the m+1th region, the processing module 140 is further adapted to obtain a fingerprint image based on the separation result. Therefore, in this embodiment, the processing module 140 further includes: an imaging unit 143, which is adapted to obtain the fingerprint image based on the first determination image, the second determination image, the third determination image, ..., the Nth determination image.

[0160] Specifically, the m+1th determined image is the image formed by the first reflected light alone in the m+1th area. Therefore, by splicing the first determined image, the second determined image, the third determined image, ..., the Nth determined image, the fingerprint image formed by the first reflected light alone can be obtained, that is, the fingerprint image.

[0161] Since the first layer of the m+1th area is the image formed by the second reflected light alone in the m+1th area, and the second known image is the image formed by the second reflected light alone, in other embodiments of the present invention, the imaging unit 143 is suitable for obtaining the fingerprint image based on the first layer of the second area, the first layer of the third area,..., the first layer of the Nth area and the first known image or the second known image.

[0162] In this embodiment, the area corresponding to the first known image 117a is used as the first zone, and the first determined image is the first known image 117a. Therefore, the first determined image, the second determined image, the third determined image, ..., and the Nth determined image are all images formed solely by the first reflected light, and the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the second known image 117b are all images formed solely by the second reflected light. Therefore, the fingerprint image can be obtained by splicing the first determined image, the second determined image, the third determined image, ..., and the Nth determined image, or by splicing the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone, and the second known image 117b.

[0163] It should be noted that, in this embodiment, the light source assembly only includes the first light source and the second light source, so it is only necessary to separate the image to be processed in which the image formed by the first reflected light and the image formed by the second reflected light overlap to obtain the fingerprint image.

[0164] In other embodiments of the present invention, the light source assembly may further include two or more light sources, and the two or more light sources respectively form a plurality of reflected lights carrying fingerprint information on the sensing surface; the image sensor collects the plurality of reflected lights to obtain a collected image; the projection ranges of the plurality of reflected lights on the image sensor at least partially overlap, and a method similar to the aforementioned embodiment of the present invention can still be used to separate the image to be processed, and obtain a fingerprint image based on the separation result.

[0165] Since the fingerprint module does not contain nonlinear optical elements, the overlapping parts of the image obtained by the image sensor are all linear combinations of images formed by reflected light from different light sources. They can be separated by removing the image formed by a single reflected light from the image of the overlapping part, thereby obtaining a high-quality fingerprint image.

[0166] It should also be noted that, in this embodiment, the light source includes a plurality of point light sources arranged in an array; and the separation of the image to be processed is closely related to the specific structure of the fingerprint module, that is, the acquisition of the fingerprint image is related to the structure of the fingerprint module. This setting is equivalent to encrypting the fingerprint image through the specific structure of the fingerprint module, which can effectively improve the security of the fingerprint image.

[0167] The method of separating the image to be processed to obtain an accurate fingerprint image by analyzing it based on the imaging principle of rectilinear light propagation is only an example. In other embodiments of the present invention, a pre-trained deep learning model can also be used to separate the image to be processed. Specifically, the processing module has a pre-set separation model, and the processing module separates the image to be processed based on the separation model.

[0168] In some embodiments of the present invention, the separation model is a pre-trained deep learning model. Specifically, the separation model is trained by the separation model training device of the present invention.

[0169] refer to Figure 11 , shows a functional block diagram of the training device for the separation model of the present invention.

[0170] The training device is suitable for training the separation model of the fingerprint module of the present invention.

[0171] In this embodiment, the fingerprint module's light source assembly includes four light sources, namely, a first light source, a second light source, a third light source, and a fourth light source. The light generated by these four light sources forms four corresponding reflected lights on the sensing surface, namely, a first reflected light, a second reflected light, a third reflected light, and a fourth reflected light. The first light source, the second light source, the third light source, and the fourth light source are spaced relatively close together, so that the projection ranges of the first reflected light, the second reflected light, the third reflected light, and the fourth reflected light on the image sensor overlap. In other embodiments, the fingerprint module's light source assembly may include other light sources.

[0172] Combined with reference Figure 8 and Figure 9 ,in Figure 8 yes Figure 7 The schematic diagram of the light spot structure of the light source assembly used in the embodiment of the fingerprint module imaging method is shown. Figure 9 Shown respectively Figure 7 Schematic diagram of the light spot structure of the first light source, the second light source, the third light source and the fourth light source in the light source assembly used in the embodiment of the fingerprint module imaging method.

[0173] Similarly, in this embodiment, the four light sources in the light source assembly each include multiple point light sources, the spacing between adjacent light sources is small, and the images formed on the image sensor by the reflected light formed by the adjacent light sources have overlapping areas; the distance between the multiple point light sources in a light source is large, and the images formed on the image sensor by the reflected light formed by each point light source do not have overlapping areas.

[0174] It should be noted that, in order to ensure the accuracy of the training results, all point light sources in the light source assembly are traversed by the multiple light sources, that is, any point light source in the light source assembly is included in at least one light source.

[0175] The training device includes: a fingerprint sample library 151, which includes training fingerprint samples, and the training fingerprint samples are used for imaging for training; a fusion image unit 152, which is suitable for controlling the light source component to generate light and using the light generated by the light source component to perform fingerprint imaging on the training fingerprint samples to obtain a fused fingerprint image.

[0176] The fusion image unit 152 is connected to both the fingerprint module (not shown) and the fingerprint sample library 151. It controls all light sources in the light source assembly to generate light simultaneously. This generated light is used to image the training fingerprint sample, thereby obtaining the fused fingerprint image. In this embodiment, the fusion image unit 152 controls four light sources in the light source assembly to generate light simultaneously. This generated light is used to image the training fingerprint sample to obtain the fused fingerprint image.

[0177] The image separation unit 153 is pre-configured with a separation model. The image separation unit 153 is adapted to use the separation model to separate the fused fingerprint image to obtain a plurality of separated fingerprint fragments corresponding to the plurality of light sources in the light source assembly. In some embodiments, the separation model may include a separation network.

[0178] The image separation unit 153 is connected to the fusion image unit 152 and obtains the fusion fingerprint image from the fusion image unit 152. A separation model is preset in the image separation unit 153, so the image separation unit 153 separates the fusion fingerprint image according to the separation model to obtain multiple separated fingerprint fragments.

[0179] In this embodiment, the light source component of the fingerprint module includes four light sources, namely a first light source, a second light source, a third light source and a fourth light source; therefore, after the image separation unit 153 separates the fused fingerprint, it obtains first separated fingerprint fragments corresponding to the first light source, second separated fingerprint fragments corresponding to the second light source, third separated fingerprint fragments corresponding to the third light source, and fourth separated fingerprint fragments corresponding to the fourth light source.

[0180] It should be noted that the initial separation model is a separation model established according to the structure of the fingerprint module, that is, the initial state of the separation model is obtained based on the structure of the fingerprint module.

[0181] The independent fragmentation unit 154 controls the light sources in the light source assembly one by one to generate light and uses the light generated by a single light source in the light source assembly to perform fingerprint imaging on the training fingerprint sample, thereby obtaining independent fingerprint fragments corresponding to the light source generating the light.

[0182] The independent fragmentation unit 154 is also connected to the fingerprint module and can control the light sources in the light source assembly to emit light respectively; the independent fragmentation unit 154 controls the light sources in the light source assembly one by one to generate light and uses the light generated by a single light source in the light source assembly to perform fingerprint imaging on the training fingerprint sample, thereby obtaining independent fingerprint fragments corresponding to the light source of the generated light.

[0183] Specifically, the light source component of the fingerprint module includes multiple light sources, and the independent fragmentation unit 154 lights up one of the multiple light sources each time to image the training fingerprint sample to obtain an independent fingerprint fragment corresponding to the lit light source.

[0184] In this embodiment, the light source component of the fingerprint module includes four light sources, namely a first light source, a second light source, a third light source and a fourth light source; the independent fragmentation unit 154 only lights up one of the four light sources each time to image the training fingerprint sample, and obtains a first independent fingerprint fragment corresponding to the first light source, a second independent fingerprint fragment corresponding to the second light source, a third independent fingerprint fragment corresponding to the third light source, and a fourth independent fingerprint fragment corresponding to the fourth light source.

[0185] The comparison and judgment unit 155 is adapted to compare the separated fingerprint fragments with corresponding independent fingerprint fragments to judge whether the obtained separated fingerprint fragments are correctly separated; when the comparison and judgment unit judges that the separation is correct, one training is completed.

[0186] The comparison and judgment unit 155 is connected to both the image separation unit 153 and the independent fragment unit 154, and obtains the fused fingerprint image and the multiple independent fingerprint fragments from the image separation unit 153 and the independent fragment unit 154 respectively; the comparison and judgment unit compares the fused fingerprint image and the multiple independent fingerprint fragments, and makes a judgment based on a preset loss function. When it is judged that the separation is correct, a training is completed.

[0187] The training device also includes: a model optimization unit 156. When the comparison and judgment unit 155 determines that the separation is incorrect, the model optimization unit 156 is suitable for adjusting the parameters of the separation model in the image separation unit 153 to optimize the separation model; the image separation unit 153 is also suitable for using the optimized separation model to separate the fused fingerprint image again, and re-obtain multiple separated fingerprint fragments for comparison and judgment until the separation is determined to be correct.

[0188] The separation model is trained by using different training fingerprint samples in the fingerprint sample library to improve the separation model. It should be noted that in some embodiments of the present invention, in actual training, techniques such as gradient penalty and spectral normalization can be used to improve training efficiency and training accuracy.

[0189] In summary, the technical solution of the present invention separates the image to be processed to obtain multiple fingerprint images formed independently on the image sensor by reflected light. This separation avoids averaging the light generated by multiple display pixels, effectively improving the quality of the obtained fingerprint image. Furthermore, the processing of the overlapping image to be processed overcomes the limitations of the spacing between light sources, effectively improving the measurement efficiency of the entire fingerprint image. Furthermore, this fingerprint module imaging method and imaging device can utilize the module structure to encrypt the image, thereby enhancing the security of the fingerprint image.

[0190] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A fingerprint module imaging method, characterized in that: The fingerprint module includes: a light source assembly, the light source assembly comprising a plurality of light sources; a sensing surface, wherein the light generated by the plurality of light sources forms a plurality of corresponding reflected lights on the sensing surface, each of the reflected lights carrying fingerprint information; An image sensor, wherein the image sensor collects the multiple reflected lights to obtain a captured image; The projection ranges of the plurality of reflected lights on the image sensor at least partially overlap; The imaging method comprises: Obtaining an image to be processed, wherein the image to be processed is an overlapping portion of images formed by the plurality of reflected lights on the image sensor in the collected image; Separating the image to be processed; Obtaining a fingerprint image according to the separation result; The step of separating the image to be processed includes: based on the relative positions of the multiple light sources, the sensing surface and the image sensor, obtaining an image formed by each light source according to a portion of the image where no light source is superimposed.

2. The imaging method according to claim 1, wherein The light source assembly includes a first light source and a second light source; The light generated by the first light source forms a first reflected light carrying fingerprint information on the sensing surface, and the light generated by the second light source forms a second reflected light carrying fingerprint information on the sensing surface; the image sensor collects the first reflected light and the second reflected light to obtain the collected image; In the step of obtaining the image to be processed, the image to be processed is an overlapping portion of the image formed by the first reflected light and the image formed by the second reflected light in the collected image.

3. The imaging method according to claim 2, wherein: The projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; The portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is a first known image, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is a second known image; The step of separating the image to be processed comprises: Taking one of the area corresponding to the first known image and the area corresponding to the second known image as a first area, and the image corresponding to the first area as a first determined image; According to the first zone, along a direction toward the image to be processed, the area corresponding to the image to be processed is divided into adjacent second zones, third zones, ..., Nth zones; The image to be processed in the m+1th region is separated according to the mth determination image to obtain the m+1th determination image, where m is an integer in the range of 1 to N-1.

4. The imaging method according to claim 3, wherein: The step of obtaining the (m+1)th determined image includes: Based on the relative positions of the first light source, the second light source, the sensing surface, and the image sensor, obtaining a first layer in the (m+1)th region according to the (m)th determined image; An m+1th determined image is obtained according to the image to be processed in the m+1th area and the first layer in the m+1th area.

5. The imaging method according to claim 4, wherein: Obtaining a fingerprint image according to the separation result includes: obtaining the fingerprint image according to the first layer of the second area, the first layer of the third area, ..., the first layer of the Nth area and the first known image or the second known image.

6. The imaging method according to claim 3, wherein: Obtaining the fingerprint image according to the separation result includes: obtaining the fingerprint image according to the first determination image, the second determination image, the third determination image, ..., the Nth determination image.

7. The imaging method according to claim 1, wherein The number of light sources included in the light source assembly ranges from 2 to 10.

8. The imaging method according to claim 1, wherein The light source includes a plurality of point light sources, and the projection ranges of reflected lights formed by the plurality of point light sources in one light source on the sensing surface do not overlap on the image sensor.

9. The imaging method according to claim 8, wherein The multiple point light sources are arranged in an array.

10. The imaging method according to claim 1 or 8, wherein: The light source assembly includes a liquid crystal display, an active matrix organic light emitting diode display or a light emitting diode display.

11. An imaging device for a fingerprint module, characterized in that: The fingerprint module includes: a light source assembly, the light source assembly comprising a plurality of light sources; a sensing surface, wherein the light generated by the plurality of light sources forms a plurality of corresponding reflected lights on the sensing surface, each of the reflected lights carrying fingerprint information; An image sensor, wherein the image sensor collects the multiple reflected lights to obtain a captured image; The projection ranges of the plurality of reflected lights on the image sensor at least partially overlap; The imaging device comprises: a processing module, wherein the processing module is adapted to obtain an image to be processed, wherein the image to be processed is an overlapping portion of images formed on the image sensor by the plurality of reflected lights in the collected image; the processing module is further adapted to separate the image to be processed and obtain a fingerprint image based on the separation result; Wherein, the processing module is also suitable for: based on the relative positions of the multiple light sources, the sensing surface and the image sensor, and according to the partial image of any light source that is not superimposed, obtaining the image formed by each light source to separate the image to be processed.

12. The imaging device according to claim 11, wherein The light source assembly includes a first light source and a second light source; The light generated by the first light source forms a first reflected light carrying fingerprint information on the sensing surface, and the light generated by the second light source forms a second reflected light carrying fingerprint information on the sensing surface; the image sensor collects the first reflected light and the second reflected light to obtain the collected image; The image to be processed obtained by the processing module is an overlapping portion of the image formed by the first reflected light and the image formed by the second reflected light in the collected image.

13. The imaging device according to claim 12, wherein The projection range of the first reflected light on the image sensor partially overlaps with the projection range of the second reflected light on the image sensor; The portion of the image formed by the first reflected light that does not overlap with the image formed by the second reflected light is a first known image, and the portion of the image formed by the second reflected light that does not overlap with the image formed by the first reflected light is a second known image. The processing module includes: a partitioning unit, the partitioning unit being adapted to use one of the area corresponding to the first known image and the area corresponding to the second known image as a first area, and the image corresponding to the first area as a first determined image; the partitioning unit being adapted to divide the area corresponding to the image to be processed into adjacent second areas, third areas, ..., and Nth areas based on the first area and along a direction toward the image to be processed; A separation unit is configured to separate the image to be processed in the m+1th region according to the mth determination image to obtain the m+1th determination image, where m is an integer ranging from 1 to N-1.

14. The imaging device according to claim 13, wherein The separation unit comprises: a converter adapted to obtain a first layer of an m+1th region according to an m-th determined image based on relative positions of the first light source, the second light source, the sensing surface, and the image sensor; A separator is adapted to obtain the m+1th determined image based on the image to be processed in the m+1th area and the first layer in the m+1th area.

15. The imaging device according to claim 14, wherein The processing module further includes: an imaging unit adapted to obtain the fingerprint image based on the first layer of the second zone, the first layer of the third zone, ..., the first layer of the Nth zone and the first known image or the second known image.

16. The imaging device according to claim 13, wherein The processing module further includes: an imaging unit, which is adapted to obtain the fingerprint image according to the first determination image, the second determination image, the third determination image, ..., the Nth determination image.

17. The imaging device according to claim 11, wherein The number of light sources included in the light source assembly ranges from 2 to 10.

18. The imaging device according to claim 11, wherein The light source includes a plurality of point light sources, and the projection ranges of reflected lights formed by the plurality of point light sources in one light source on the sensing surface do not overlap on the image sensor.

19. The imaging device according to claim 18, wherein The multiple point light sources are arranged in an array.

20. The imaging device according to claim 11 or 18, wherein The light source assembly includes a liquid crystal display, an active matrix organic light emitting diode display or a light emitting diode display.

Images