Foreign matter detection method, program product, storage medium and electronic equipment

Abstract

The invention relates to the technical field of fingerprint identification, and provides a foreign matter detection method, a program product, a storage medium and electronic equipment. The foreign matter detection method comprises the following steps: acquiring a to-be-detected image acquired by an under-screen fingerprint module; performing contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image; target geometric figure features are detected according to the contour pixels, the foreign matter detection result of the to-be-detected image is determined according to the detection condition, and the target geometric figure features represent features presented by the foreign matter in the to-be-detected image. According to the method, the contour of the foreign matter can present specific geometric figure features in the to-be-detected image, and the geometric figure features do not exist in the normal image, so that the foreign matter detection result can be obtained by extracting the contour in the to-be-detected image and then detecting the target geometric figure features according to the contour pixels. Moreover, the contour is a part with significant pixel value change in the image, so that the result of foreign matter detection based on the contour is relatively accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of fingerprint recognition, and in particular, to a foreign object detection method, a program product, a storage medium, and an electronic device. Background Art

[0002] In recent years, the under-screen fingerprint recognition technology has been widely applied to mobile devices. This technology mainly relies on a fingerprint module disposed under the device screen to collect fingerprint images. If there are foreign objects at the screen position where the fingerprint module is located, the quality of the collected fingerprint images will be affected, resulting in a decrease in the fingerprint recognition accuracy. In addition, during the test phase of the under-screen fingerprint module, image acquisition is also performed (although the images collected at this time may not be the fingerprint images of users), and a series of index calculations are performed based on the collected images. If there are foreign objects at the screen position at this time, the calculation of these indexes will be inaccurate, and ultimately, the recognition accuracy may also be low after the fingerprint module is put into use. Currently, there is no good detection method for these foreign objects. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a foreign object detection method, a program product, a storage medium, and an electronic device to improve the above technical problems.

[0004] To achieve the above purpose, the present application provides the following technical solutions:

[0005] In a first aspect, the embodiments of the present application provide a foreign object detection method, including: obtaining a to-be-detected image collected by a fingerprint module disposed under a screen; performing contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image; detecting target geometric graphic features according to the contour pixels, and determining a foreign object detection result of the to-be-detected image according to the detection situation; wherein the target geometric graphic features represent the features presented by a foreign object in the to-be-detected image.

[0006] The inventors have found through research that the contours of foreign objects will present some specific geometric graphics in the to-be-detected image, such as geometric graphics like straight lines and arcs. These geometric features do not exist in normal images (whether collected during the test or application phase of the fingerprint module). Therefore, by extracting the contour (or a part of the contour) in the to-be-detected image and then detecting the target geometric graphic features according to the contour pixels, a foreign object detection result can be obtained. Moreover, since the contour is a part of the image where the pixel values change significantly, it is easy to obtain a relatively accurate extraction result. Therefore, detecting foreign objects based on the contour information can obtain a relatively accurate detection result.

[0007] In an implementation of the first aspect, the contour extraction of the image to be detected to obtain the contour pixels in the image to be detected includes: calculating the gradient values of the pixels in the image to be detected row by row, and for the calculation results of each row, taking at least one pixel with the largest corresponding gradient value as the contour pixel in the pixels of this row; and / or calculating the gradient values of the pixels in the image to be detected column by column, and for the calculation results of each column, taking at least one pixel with the largest corresponding gradient value as the contour pixel in the pixels of this column.

[0008] In the image to be detected, the pixel value changes relatively fast at the contour of the foreign object, and the gradient value at a certain pixel can exactly reflect the speed of change of the pixel value in the image to be detected at this position. Therefore, pixels with relatively large corresponding gradient values in the image to be detected can be selected as contour pixels.

[0009] In the above implementation, the gradient value can be calculated only in the row direction or the column direction to improve the contour extraction efficiency, but the gradient value can also be calculated in both the row direction and the column direction to improve the integrity of the contour extraction; for each row or column, only one pixel with the largest gradient value can be taken as the contour pixel in this row or column to improve the contour extraction efficiency, but multiple pixels with the largest gradient value can also be taken as the contour pixel in this row or column to improve the integrity of the contour extraction.

[0010] In an implementation of the first aspect, the target geometric figure is a straight line. The detection of the target geometric figure features based on the contour pixels and the determination of the foreign object detection result of the image to be detected according to the detection situation include: traversing all the contour pixels, determining the value combinations of a set of parameters corresponding to a straight line passing through each contour pixel, the parameters including a first parameter and a second parameter, and the values of the parameters being quantized; counting the occurrence times of each value combination of the parameters obtained during the traversal to obtain a statistical result; determining the foreign object detection result of the image to be detected according to the relationship between the statistical result and a first threshold; wherein, if the occurrence times of a value combination of a parameter in the statistical result is greater than the first threshold, this value combination of the parameter identifies a straight line detected from the contour pixels, and the occurrence times of this value combination of the parameter is the target geometric figure feature corresponding to this straight line.

[0011] Since a straight line equation can be uniquely determined by only two parameters, the first parameter and the second parameter constitute a parameter space. Each point in this parameter space, that is, a value combination of a parameter, corresponds to a straight line. On the contrary, each straight line corresponds to a point in the parameter space.

[0012] Since there are infinitely many points in the parameter space, which is not convenient for calculation, the values of the first parameter and the second parameter can be quantized, so that the parameter space degenerates into a set of a finite number of points. Within the quantization accuracy, some relatively close lines can correspond to the same point in the parameter space.

[0013] In the case of parameter quantization, the value combinations of the parameters corresponding to the lines passing through the contour pixels can be statistically counted, which is equivalent to counting the number of occurrences of each point in the parameter space. If a certain point appears n times (n is a non-negative integer), it means that the line corresponding to this point passes through n contour pixels at the same time. If n is greater than the first threshold, it indicates that the line corresponding to this point passes through a relatively large number of contour pixels at the same time, that is, these contour pixels form a relatively obvious line. In this way, through simple statistics, the lines existing in the contour pixels can be quickly and accurately detected, and then the final foreign object detection result can be determined.

[0014] In one implementation manner of the first aspect, the first parameter represents the inclination angle of the line segment passing through the origin and perpendicular to the line, the second parameter represents the length of the line segment passing through the origin and perpendicular to the line, the first parameter takes discrete values at equal intervals within a preset angle range, and the value of the second parameter is calculated according to the value of the first parameter and the coordinates of the contour pixels.

[0015] The above implementation manner gives a selection method for the first parameter and the second parameter. The parameter space in this selection method can be considered as a polar coordinate system.

[0016] In one implementation manner of the first aspect, determining the foreign object detection result of the image to be detected according to the relationship between the statistical result and the first threshold includes: determining the maximum number of occurrences of the value combination of the parameters in the statistical result; normalizing the maximum number of occurrences according to the size of the image to be detected to obtain the normalized maximum number of occurrences; if the normalized maximum number of occurrences is greater than the first threshold, it is determined that there is a foreign object with a linear feature in the image to be detected.

[0017] In the above implementation manner, since it is only necessary to determine whether there is a foreign object with a linear feature in the image to be detected, only the maximum number of occurrences of the value combination of the parameters in the statistical result can be used, and other data in the statistical result are not used, thereby improving the detection efficiency.

[0018] In addition, considering that the image to be detected may have different sizes, before comparing the maximum number of occurrences with the first threshold, the maximum number of occurrences can be normalized according to the size of the image to be detected first, which can improve the generality of the solution.

[0019] In an implementation of the first aspect, after obtaining the to-be-detected image collected below the screen and before performing contour extraction on the to-be-detected image, the method further includes: performing edge-preserving smoothing filtering on the to-be-detected image.

[0020] In the above implementation, smoothing filtering is performed on the to-be-detected image before contour extraction to reduce high-frequency noise therein, improve contour extraction accuracy, and thus improve the accuracy of foreign object detection. At the same time, this smoothing filtering preserves edges and will not smooth out some valuable edges between the foreign object and the background, which is also beneficial to improving contour extraction accuracy.

[0021] In an implementation of the first aspect, the foreign object is a white foam, and the to-be-detected image is an image collected by the fingerprint module when a black object presses the fingerprint collection area of the screen.

[0022] Since the black object has a strong ability to absorb light, the reflected light of the white foam (i.e., the image of the foreign object) is mainly retained in the to-be-detected image, and there is less interference light signal, so that the white foam is relatively easy to detect.

[0023] In a second aspect, an embodiment of the present application provides a foreign object detection device, including: an image acquisition module for acquiring a to-be-detected image collected by a fingerprint module disposed below the screen; a contour extraction module for performing contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image; a foreign object detection module for detecting the characteristics of a target geometric figure according to the contour pixels and determining a foreign object detection result of the to-be-detected image according to the detection situation; wherein, the target geometric figure characteristics represent the characteristics presented by the foreign object in the to-be-detected image.

[0024] In a third aspect, an embodiment of the present application provides a computer program product, including computer program instructions, which when read and run by a processor, execute the method provided by the first aspect or any implementation of the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, which when read and run by a processor, execute the method provided by the first aspect or any implementation of the first aspect.

[0026] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor, wherein computer program instructions are stored in the memory, which when read and run by the processor, execute the method provided by the first aspect or any implementation of the first aspect. Description of the Drawings

[0027] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following accompanying drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related accompanying drawings can also be obtained based on these drawings.

[0028] Figure 1 It is the working principle diagram of the under-screen fingerprint module;

[0029] Figure 2 It is the flowchart of the foreign object detection method provided by the embodiment of the present application;

[0030] Figure 3 It is the image to be detected in the foreign object detection method provided by the embodiment of the present application;

[0031] Figure 4 It is the approximate shape and position of the foam in the image to be detected;

[0032] Figure 5 It is at least part of the contour pixels in the image to be detected;

[0033] Figure 6 It is the image to be detected after median filtering;

[0034] Figure 7 It is the straight line passing through the contour pixel (x, y) with parameters (θ1, ρ1) and (θ2, ρ2) respectively;

[0035] Figure 8 It is the statistical table of the occurrence times of the parameter value combinations;

[0036] Figure 9 It is the schematic diagram of the second method for detecting a straight line provided by the embodiment of the present application;

[0037] Figure 10 It is the schematic diagram of the third method for detecting a straight line provided by the embodiment of the present application;

[0038] Figure 11 It is the functional module diagram of the foreign object detection device provided by the embodiment of the present application;

[0039] Figure 12 It is the structural diagram of the electronic device provided by the embodiment of the present application. Specific embodiments

[0040] Biometric technologies have been widely applied to various terminal devices or electronic devices. Biometric identification technologies include, but are not limited to, fingerprint recognition, palmprint recognition, vein recognition, iris recognition, face recognition, liveness recognition, anti-counterfeiting recognition, and other technologies. Among them, fingerprint recognition usually includes optical fingerprint recognition, capacitive fingerprint recognition, and ultrasonic fingerprint recognition. With the rise of the full-screen technology, the fingerprint recognition module can be set in a partial area or the entire area under the display screen, thus forming an under-display optical fingerprint recognition; or, part or all of the optical fingerprint recognition module can be integrated into the display screen of the electronic device, thus forming an in-display optical fingerprint recognition. The display screen can be an organic light-emitting diode (OLED) display screen, a liquid crystal display (LCD), or the like. The fingerprint recognition method usually includes steps such as fingerprint image acquisition, preprocessing, feature extraction, and feature matching. Some or all of the above steps can be implemented by traditional computer vision (CV) algorithms or by deep learning algorithms based on artificial intelligence (AI). Fingerprint recognition technology can be applied to portable or mobile terminals such as smartphones, tablets, and gaming devices, as well as other electronic devices such as smart door locks, cars, and bank automated teller machines, for fingerprint unlocking, fingerprint payment, fingerprint attendance, identity authentication, and the like.

[0041] Figure 1 is the working principle diagram of the under-screen fingerprint module. Refer to Figure 1 , the electronic device (such as a mobile phone, a tablet) includes a screen, and the fingerprint module is set under the screen (such as on the main board of the electronic device), so it can sometimes be simply referred to as the under-screen fingerprint module. The area on the screen corresponding to the fingerprint module ( Figure 1 the gray circle in) generates an optical signal when the target object presses. After being reflected by the target object, the optical signal passes through the screen and is received by the photosensitive element on the fingerprint module (the general propagation path of the optical signal is shown by the arrow), and is subjected to photoelectric conversion to generate an image of the target object. Some preliminary processing may be performed on the image inside the fingerprint module and then output. Further processing can be performed inside the electronic device based on the image collected by the fingerprint module.

[0042] After the electronic device is produced, the above target object can be the user's finger, and the image of the target object is the user's finger image, or the fingerprint image. The electronic device can determine whether the identity of the current user is legal by comparing the currently collected fingerprint image with the fingerprint image in the database. This process is the fingerprint recognition process.

[0043] At this time, if there is a foreign object at the screen where the fingerprint module is located, such as impurities or stains on the screen surface, the foreign object will also appear in the fingerprint image, resulting in a decrease in the quality of the fingerprint image, which in turn affects the accuracy of fingerprint recognition and may even lead to fingerprint recognition failure.

[0044] In addition, during the production process of the electronic device, for testing purposes, an in-display fingerprint module may also be used to collect images. For example, during the whole machine assembly process of a mobile phone manufacturer, there is generally an industrial model test link:

[0045] After the in-display fingerprint module is installed in the mobile phone and placed in the production line machine, the machine presses rubber heads of the same or different types on the screen area corresponding to the in-display fingerprint module in sequence. The fingerprint module collects the images of the rubber heads after the corresponding rubber heads are pressed. The mobile phone executes corresponding test items based on the images of the rubber heads. For example, calculate the exposure time, high-frequency components, detect bad pixels, detect occlusion, etc. under the first type of rubber head, calculate the signal volume, light leakage ratio, etc. under the second type of rubber head, calculate the signal-to-noise ratio, magnification ratio, etc. under the third type of rubber head, and evaluate whether there are problems with the basic functions such as imaging of the module according to the calculation results of these indicators and the preset standards.

[0046] During the industrial model test process, if there is a foreign object at the screen, it will also affect the accuracy of the test results, which may in turn lead to low fingerprint recognition accuracy after the mobile phone is put into use. For example, there is usually a foam filled between the in-display fingerprint module and the screen to protect the screen. The natural color of the foam is usually white. Under normal circumstances, the foam will be blackened, so that the reflection of the foam is very weak after filling and has little impact on the quality of the rubber head image; however, there are also abnormal incoming materials. In this case, the foam is not blackened, so that the reflection of the foam is stronger after filling, and an image of the foam will be formed in the rubber head image (for reference, see Figure 2 ) in the following text, which has a serious impact on the quality of the rubber head image and may lead to mismeasurement of several items.

[0047] Note that in the above example, the unblackened foam is regarded as a foreign object, while the blackened foam is not regarded as a foreign object. That is to say, if there is no other object at the screen where the fingerprint module is located, it naturally belongs to the situation of no foreign object. However, even if there is other object at the screen, as long as it has no impact or little impact on the quality of the image collected by the fingerprint module, such an object may not be regarded as a foreign object. Only an object that has an obvious negative impact on the image quality is regarded as a foreign object.

[0048] In the prior art, there is no good detection method for foreign objects on the screen. However, through long-term research, the inventor found that the outline of a foreign object will present some specific geometric shapes in the images collected by the fingerprint module. Based on this, embodiments of the present application provide a foreign object detection method, a program product, a storage medium, and an electronic device, which detect foreign objects on the screen by detecting the features of specific geometric shapes in the images collected by the under-screen fingerprint module, filling the gap in the prior art.

[0049] It should be understood that in addition to the technical solutions proposed in the embodiments of the present application, the above discovery and analysis process of the problems existing in the prior art also belong to the contribution made by the inventor to the present application, rather than the content already existing in the prior art.

[0050] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0051] The term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0052] Terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and cannot be understood as indicating or implying relative importance, nor can they be understood as requiring or implying any actual relationship or order between these entities or operations.

[0053] Figure 2 is a flowchart of the foreign object detection method provided by the embodiments of the present application. This method can be, but is not limited to, executed by Figure 12 the electronic device 300 shown. For the specific structure of the electronic device 300, reference can be made to the description in the following text about Figure 12 . Referring to Figure 2 , this method includes:

[0054] Step S110: Obtain a to-be-detected image collected by a fingerprint module disposed under the screen.

[0055] Figure 2The method in [description] can be applied to the application stage or the testing stage of the in-screen fingerprint module. In the application stage, the image to be detected can be a fingerprint image collected by the fingerprint module after a finger presses the screen. If a foreign object is detected, various measures can be taken. For example, a prompt message indicating that there is a foreign object on the screen or that fingerprint collection needs to be redone can be output. Another example is to remove the part of the fingerprint image with the foreign object and perform fingerprint recognition based on the remaining part. Another example is to use the entire fingerprint image for fingerprint recognition, but reduce the weight of the part with the foreign object, and so on.

[0056] In the testing stage, the image to be detected can be an object image collected by the fingerprint module after a test object (such as the rubber head mentioned above) presses the screen. If a foreign object is detected, various measures can be taken. For example, a prompt message indicating that there is a foreign object on the screen can be output, and the worker can process the foreign object (such as blackening the foam) and then retest. Another example is to continue the test, but reduce the credibility of the test result. Another example is to continue the test, but use a specific compensation algorithm to reduce the impact of the foreign object on the test result, and so on.

[0057] If the device that executes step S110 is the device equipped with the in-screen fingerprint module, then the image to be detected obtained in step S110 can be the image collected by the fingerprint module. If the device that executes step S110 is not the device equipped with the in-screen fingerprint module, then the image to be detected may be transferred to this device through network transmission, copying, etc. after the fingerprint module has finished collecting. At this time, the image to be detected can be directly read from the local storage medium of the device. For example, if a mobile phone is equipped with an in-screen fingerprint module, the (processor of the) mobile phone can obtain the image to be detected and perform foreign object detection. If a mobile phone is equipped with an in-screen fingerprint module, the mobile phone can also send the image to be detected to a remote server for foreign object detection to be performed on the server.

[0058] The image to be detected can be the original image directly output by the in-screen fingerprint module (which can be called a raw image), or it can also be an image obtained after further processing (such as encoding, denoising, brightness adjustment) of this original image.

[0059] Figure 3 This is the image to be detected in the foreign object detection method provided by the embodiments of this application. The foreign object to be detected here is the foam, or more precisely, the foam that has not been properly blackened. The bluish-green part in the figure is the background (referring to other objects that are not the foam, such as the rubber head), and the yellowish part may be the image of the foam formed due to the reflection of the white foam.

[0060] Note that the actual image to be detected may be a grayscale image. Here, for the convenience of better displaying the image content, the grayscale image has been colorized. Therefore Figure 3The image to be detected is not completely real. Similar situations exist in some of the following figures, and will not be explained one by one.

[0061] S120: Extract the contour of the image to be detected to obtain the contour pixels in the image to be detected.

[0062] The contour can be understood as the boundary of an object, and the contour pixels are the pixels that make up the contour of the object. The contour pixels are the result of contour extraction. According to the detection requirements, step S120 can extract part of the contour in the image to be detected, or can extract all the contours in the image to be detected. Contour extraction can be completed using specific algorithms. For example, the contour can be extracted using the method based on the Canny operator, the contour can be extracted using library functions (such as findContours) in an image processing library (such as OpenCV), the contour can be extracted using a neural network, etc. Examples of contour extraction will also be given later.

[0063] It should be noted that there may be no obvious contour in the image to be detected without foreign objects, but the algorithm in step S120 may still extract a contour. However, the contour extracted at this time does not correspond to the contour of the actual object, but is just a result of the algorithm running, and it is very likely that foreign objects cannot be detected based on such a contour in the subsequent steps.

[0064] Figure 4 is the approximate shape and position of the foam in the image to be detected. Refer to Figure 4 , the approximate shape of the foam is a square with round holes, and its position is in the part between the white frame lines, that is, the area filled with slashes. And the white frame lines are the approximate positions of the foam contour.

[0065] Figure 5 are at least some of the contour pixels in the image to be detected (shown as yellow-green dots). Comparing Figure 4 and Figure 5 it is not difficult to see that the contour pixels are generally located near the white frame lines in Figure 4 , which represents part of the foam contour. Regarding how to obtain the contour pixels in Figure 5 , it will be elaborated later.

[0066] S130: Detect the target geometric shape features based on the contour pixels, and determine the foreign object detection result of the image to be detected according to the detection situation; wherein, the target geometric shape features characterize the features presented by the foreign objects in the image to be detected.

[0067] The inventors' research found that the outline of a foreign object will present some specific geometric shapes in the image to be detected, such as straight lines, arcs, triangles and other geometric shapes, which are called target geometric shapes. These geometric shapes do not exist in normal images without foreign objects (whether in the test or application stage of the fingerprint module). Therefore, by extracting the outline (or a part of the outline) in the image to be detected and then detecting the features of the target geometric shape according to the outline pixels (which can be abbreviated as target geometric shape features), it can be determined whether the outline pixels contain the target geometric shape, and then the foreign object detection result can be obtained. And detecting the target geometric shape features according to the outline pixels can be calculated through the coordinates of the outline pixels. Some possible implementation methods will be given later.

[0068] For example, by comparing Figure 3 and Figure 4 it can be seen that at the outer outline of the foam ([[]] Figure 4 [[]] the square frame line), there is a linear feature (the target geometric shape feature corresponding to the straight line) in the image to be detected. At the inner outline of the foam ([[]] Figure 4 [[]] the circular frame line), there is a circular feature (the target geometric shape feature corresponding to the circle) in the image to be detected. For a normal image to be detected, typically, it is brighter in the middle and gradually darker towards the periphery, but the transition is very uniform and there will be no obvious linear or circular features (there may be fingerprints, but fingerprints are not straight lines or circles, and fingerprints are mainly texture features. Here, it mainly refers to brightness features). Therefore, by detecting the linear feature or circular feature according to the outline pixels, it can be determined whether there is an (unblackened) foam at the screen where the fingerprint module is located. Generally speaking, if the position distribution of some (or all) outline pixels presents as one (or more) straight lines, it can be considered that a linear feature is detected based on the outline pixels; if the position distribution of some (or all) outline pixels presents as one (or more) circles (including arcs), it can be considered that a circular feature is detected based on the outline pixels. The detection of other geometric shape features can be analyzed similarly.

[0069] Furthermore, since detecting linear features is relatively simple, the case of detecting linear features will be mainly taken as an example in the following text, but it does not exclude that circular features or other geometric shape features can be detected in other embodiments.

[0070] It should also be understood that Figure 3 and Figure 4 are only examples. If the foam is not the [[[]] Figure 4 [[]] shape, the geometric features presented by the foam in the image to be detected may be different, and the target geometric shape features to be detected need to be adjusted accordingly.

[0071] ​​​​​​The foreign object detection result can include not only whether a foreign object is detected, but also more information related to the foreign object, such as one or more pieces of information including the position of the foreign object, the number of foreign objects, the type of foreign object, etc. In particular, the information of some foreign objects may be obtained through prior knowledge rather than detection. For example, if it is known in advance that the only foreign object with linear features that may appear in the test stage is foam, then after detecting the linear features in the contour, the type of the foreign object can be directly determined as foam rather than other objects.

[0072] Furthermore, the foreign object detection result can also be information reflecting the situation of the object to be detected, which refers to an object that may have foreign objects. For example, the object to be detected can be the screen of an electronic device, and the foreign object is an impurity on the screen, then the foreign object detection result can be information indicating whether the screen is clean, not necessarily information directly describing the impurity itself; another example is that the object to be detected can also be the lens in the fingerprint module, and the foreign object is a lens stain, then the detection result can be information indicating whether the lens is clean, not necessarily information directly describing the stain itself.

[0073] Figure 2 The method in [reference] realizes foreign object detection by extracting the contour in the image to be detected and detecting the geometric shape features presented by the foreign objects in the contour, filling the gap in the prior art and being beneficial to improving the fingerprint recognition performance of the in-screen fingerprint module. And since the contour is the part where the pixel values in the image change significantly, it is easy to obtain a relatively accurate extraction result, so that relatively accurate detection results can be obtained by detecting foreign objects based on the contour information.

[0074] In some implementation manners, the foreign object is white foam, and the image to be detected is an image collected by the in-screen fingerprint module when a black object presses the fingerprint collection area of the screen. The black object here can be a black rubber head (for example, in the industrial mold test stage), but other materials of black objects are not excluded.

[0075] Since the black object has a strong ability to absorb light, the reflected light of the white foam (i.e., the image of the foreign object) is mainly retained in the image to be detected, and there is less other interfering light signals, so that the white foam is relatively easy to detect.

[0076] Of course, this does not mean that the images to be detected collected by other methods cannot be used for foreign object detection. For example, if the in-screen fingerprint module has been put into use, basically only fingerprint images will be collected, and at this time, foreign object detection will only be based on the fingerprint images. However, in the industrial mold test link of the in-screen fingerprint module, in principle, it is possible to detect foam with images of different rubber heads. At this time, it is possible to preferentially select the image of the black rubber head to detect foam.

[0077] In some implementations, between step S110 and step S120, edge-preserving smoothing filtering can also be performed on the image to be detected, and the image obtained after filtering is used to execute step S120.

[0078] For example, median filtering is an edge-preserving smoothing filtering. Its specific approach is as follows: for each pixel in the image to be detected, sort the pixel values of all pixels within the neighborhood of this pixel (such as an N×N window centered on this pixel, where N is an odd number not less than 3, such as 3, 5, etc.), and take the median of the sequence as the pixel value of this pixel after filtering. In addition to median filtering, bilateral filtering, guided image filtering, etc. also belong to edge-preserving smoothing filtering.

[0079] Smoothing filtering can reduce the high-frequency noise in the image to be detected, improve the calculation accuracy of subsequent gradient values, and thus improve the accuracy of foreign object detection. At the same time, retaining the edges in the detection image during filtering will not smooth out some valuable edges between the foreign object and the background, which is also beneficial to improving the accuracy of foreign object detection. On the contrary, some smoothing filtering methods that simply perform pixel value weighting, such as mean filtering and Gaussian filtering, have a relatively poor edge-preserving effect and are not conducive to subsequent differentiation between foreign objects and the background.

[0080] Figure 6 Is the image to be detected after median filtering. Comparing Figure 6 and Figure 3 It can be seen that although the image becomes smooth after filtering, Figure 6 the yellow part (corresponding to the foam) in

[0081] is still well retained, which is more conducive to distinguishing from the background, that is, it can improve the accuracy of foreign object detection.

[0082] Calculate the gradient values of the pixels in the image to be detected row by row. For the calculation results of each row, take at least one pixel with the largest corresponding gradient value as the contour pixel among the pixels in this row; and / or calculate the gradient values of the pixels in the image to be detected column by column. For the calculation results of each column, take at least one pixel with the largest corresponding gradient value as the contour pixel among the pixels in this column.

[0083] In the image to be detected, the pixel values change relatively quickly at the contour of the foreign object, and the gradient value at a certain pixel can exactly reflect the speed of change of the pixel values in the image to be detected at this position. Therefore, pixels with relatively large corresponding gradient values in the image to be detected can be selected as contour pixels.

[0084] Taking the calculation of the gradient values corresponding to a row of pixels as an example, methods such as forward difference, backward difference, and central difference can be used for gradient calculation. For example:

[0085] Forward difference G(i) = f(i+1)-f(i)

[0086] Backward difference G(i) = f(i) - f(i-1)

[0087] Central difference G(i) = (f(i+1) - f(i-1)) / 2

[0088] Where i represents the current pixel, i+1 represents the next pixel after the current pixel in the row direction, i-1 represents the previous pixel in the row direction, f(i), f(i+1), and f(i-1) represent the pixel values of pixels i, i+1, and i-1 respectively, and G(i) represents the gradient value corresponding to pixel i. For the first or last pixel in a row, the gradient value calculation may require special processing. In addition, the calculated gradient value can be taken as the absolute value, that is, the direction of the gradient is ignored and only its magnitude is considered. Unless otherwise specified in the following text, the gradient value mentioned can be understood as the gradient value after taking the absolute value.

[0089] After calculating the gradient values corresponding to a row of pixels, the gradient values are sorted, and then the largest one or more gradient values are taken. The corresponding pixels are the contour pixels to be extracted in this row, because the gradient at the contour is larger.

[0090] For calculating the gradient value of a column of pixels, the method is similar and will not be repeated.

[0091] The gradient value can be calculated only in the row direction or column direction to improve the efficiency of contour extraction, but the gradient value can also be calculated in both the row direction and the column direction to improve the integrity of contour extraction. For example, if the gradient value is calculated only in the row direction (that is, only the row-by-row gradient calculation is performed without the column-by-column gradient calculation), the contour distributed along the row direction may not be extracted, but if it is determined in advance that the foreign body has only contours distributed along the column direction, or there are contours distributed in both the row and column directions (as long as one of them is detected), then this processing is also feasible.

[0092] For each row or column, only one pixel with the largest gradient value can be taken as the contour pixel in the row or column to improve the efficiency of contour extraction, but multiple pixels with the largest gradient values can also be taken as the contour pixels in the row or column to improve the integrity of contour extraction. However, the more contour pixels are retained, the more likely it is that some pixels that are not actually contours will be extracted. Therefore, the number of contour pixels to be retained can also be determined based on some prior information or detection requirements of foreign objects. In addition, in some optional schemes, the number of contour pixels retained by rows and columns is allowed to be different, and even the number of contour pixels retained by different rows or columns is allowed to be different.

[0093] Refer againFigure 5 , Figure 5 This is the case where gradient values are calculated in both the row direction and the column direction, and only one contour pixel is retained in each row or each column. Intuitively, Figure 5 the contour pixels on the left and above of Figure 5 form two approximate straight lines. If only linear features are detected to determine whether there is white foam in the image to be detected, then Figure 5 the retained contour pixels in

[0094] are sufficient, and it is not very necessary to extract more contours. However, if the goal of foreign object detection is to determine the approximate position of the foreign object, since

[0095] Way 1:

[0096] Step A1: Traverse all the contour pixels to determine the value combinations of the parameters corresponding to a group of straight lines passing through each contour pixel.

[0097] The parameters here include the first parameter and the second parameter, and the values of the parameters are quantized. For example, the first parameter can take discrete values at equal intervals within a preset range, and this interval is the quantization step of the first parameter. Thus, these discrete values can all be regarded as the quantization values of the first parameter. Then, by combining the value of the first parameter and the coordinates of the contour pixel (for example, the coordinates in the rectangular coordinate system), the value of the second parameter can be calculated using the straight line equation, and it is quantized using the quantization step of the second parameter. The quantized set of values of the first parameter and the second parameter constitutes a value combination of the parameters. Of course, it is also possible to determine the value of the second parameter first and then calculate the value of the first parameter. The process is similar to the above and will not be repeated here.

[0098] Therefore, the number of "a group of straight lines" in Step A1 can be the same as the number of values of the first parameter or the second parameter. In addition, for different contour pixels, the number of "a group of straight lines" passing through this pixel is the same.

[0099] For example, the first parameter is θ, where θ represents the inclination angle of the line segment passing through the origin and perpendicular to the above-mentioned straight line (which can be the angle with the x-axis), and the second parameter is ρ, where ρ represents the length of the line segment passing through the origin and perpendicular to the above-mentioned straight line. θ and ρ can be considered to form a polar coordinate system. If the coordinates of the contour pixel in the rectangular coordinate system are denoted as (x, y), then the straight line equation passing through the point (x, y) can be expressed in polar coordinates as ρ = x * cosθ + y * sinθ.

[0100] According to the method introduced above, θ takes discrete values at equal intervals within a preset angle range. For example, within the range of [-90, 90] (the unit of θ is °, omitted for simplicity), discrete values are taken with a quantization step of 1, that is, -90, -89, …, 90. Substituting the values of θ, x, and y into the straight-line equation, the value of ρ can be calculated, and then it is quantized with a quantization step of 1. For example, assuming that the calculated ρ is 5 (already quantized) when θ = 90, then (90, 5) constitutes a combination of parameter values. It should be understood that the above preset angle range and the quantization steps of θ and ρ can also take other values.

[0101] Figure 7 is a straight line passing through the contour pixel (x, y) with parameters (θ1, ρ1) and (θ2, ρ2) respectively. Refer to Figure 7 , (θ1, ρ1) and (θ2, ρ2) are two combinations of θ and ρ values. (θ1, ρ1) corresponds to a blue straight line with an angle of θ1 with the x-axis and a distance of ρ1 from the origin, and (θ2, ρ2) corresponds to a green straight line with an angle of θ2 with the x-axis and a distance of ρ2 from the origin. Both straight lines pass through the contour pixel (x, y).

[0102] It should be noted that the first parameter and the second parameter can also be parameters with other meanings, not necessarily the inclination angle and length above. For example, the first parameter can also represent the slope of the straight line, and the second parameter can also represent the intercept of the straight line. However, in this case, special processing is required for straight lines parallel to the y-axis (with an infinite slope), and so on.

[0103] Step A2: Count the occurrence times of each combination of parameter values obtained during the traversal process to obtain a statistical result.

[0104] There are multiple ways to execute Step A2: After each combination of parameter values is determined in Step A1, the occurrence times of this combination of values can be counted, that is, incremented by 1; after all combinations of parameter values corresponding to a contour pixel are determined in Step A1, the occurrence times of these combinations of parameter values can be counted; after all combinations of parameter values corresponding to all contour pixels are determined in Step A1, the occurrence times of these combinations of parameter values can be counted, and so on.

[0105] Figure 8 is a statistical table of the occurrence times of combinations of parameter values. Refer to Figure 8, the first row represents the values of θ, which range from -90, -89, …, 90, and the first column represents the values of ρ, which range from 0, 1, …, ρ_max. The values of both θ and ρ are quantized. ρ_max can be a preset value or can be the maximum value of ρ calculated actually. The part starting from the second row and second column is the occurrence count of each value combination. Initially, the counts in these cells are all 0. When a certain value combination appears once, the count in the corresponding cell is incremented by 1. Note that different from θ, some values of ρ in [0, ρ_max] may never appear, that is, some value combinations of certain parameters in the statistical table do not correspond to the straight lines passing through the contour pixels. However, the count corresponding to such value combinations is 0, which does not affect the subsequent detection results.

[0106] Step A3: Determine the foreign object detection result of the image to be detected according to the relationship between the statistical result and the first threshold.

[0107] Among them, if the occurrence count of a value combination of a parameter in the statistical result (which may be processed by normalization, etc., see the example later) is greater than the first threshold, then this value combination of the parameter identifies a straight line detected from the contour pixels. At this time, the occurrence count of this value combination of the parameter can be regarded as the target geometric feature (linear feature) corresponding to this straight line, that is, an index that can determine the existence of a straight line in the contour pixels. The principle analysis is as follows:

[0108] Since a straight line equation can be uniquely determined by only two parameters, the first parameter and the second parameter constitute a parameter space. Each point in this parameter space, that is, a value combination of a parameter, corresponds to a straight line. Conversely, each straight line corresponds to a point in the parameter space. For example, for θ and ρ, this coordinate space can be regarded as the polar coordinate system.

[0109] Since there are infinitely many points in the parameter space, which is not convenient for calculation, the values of the first parameter and the second parameter can be quantized, so that the parameter space degenerates into a set of finite points. Within the quantization accuracy, some relatively close straight lines can correspond to the same point in the parameter space.

[0110] In the case of parameter quantization, the value combinations of the parameters corresponding to the straight lines passing through the contour pixels can be statistically counted, which is equivalent to counting the occurrence times of each point in the parameter space. If a certain point appears n times (n is a non - negative integer), it means that the straight line corresponding to this point passes through n contour pixels at the same time. If n is greater than the first threshold, it indicates that the straight line corresponding to this point passes through a relatively large number of contour pixels at the same time, that is, these contour pixels form a relatively obvious straight line. In this way, through simple statistics, the straight lines existing in the contour pixels can be detected quickly and accurately, and then the final foreign object detection result can be determined.

[0111] If only detecting whether there is a foreign object with linear features in the image to be detected (for example, Figure 4 the white foam shown), step A3 can be performed as follows:

[0112] First, determine the maximum occurrence times of the value combinations of the parameters in the statistical result. Referring to Figure 8 , the value combination (θ_target, ρ_target) has the most occurrences in the statistical table, which is 100 times. Thus, the maximum occurrence times is 100.

[0113] Then, compare the maximum occurrence times with the first threshold. If the maximum occurrence times is greater than the first threshold, it is determined that there is a foreign object with linear features in the image to be detected; otherwise, it is determined that there is no foreign object with linear features in the image to be detected. The first threshold can be a preset value, such as 40, 50, etc.

[0114] In the above implementation, since it is only necessary to determine whether there is a foreign object with linear features in the image to be detected, without determining other information such as the number and position of the foreign object, the maximum occurrence times of the value combinations of the parameters in the statistical result can be used only, without using other data in the statistical result, improving the detection efficiency.

[0115] In an alternative solution, after obtaining the maximum occurrence times, the maximum occurrence times can also be normalized according to the size of the image to be detected first to obtain the normalized maximum occurrence times, and then the normalized maximum occurrence times is compared with the first threshold. If the normalized maximum occurrence times is greater than the first threshold, it is determined that there is a foreign object with linear features in the image to be detected; otherwise, it is determined that there is no foreign object with linear features in the image to be detected.

[0116] Among them, the normalization operation can be implemented by calculating the ratio of the maximum occurrence times to the parameter representing the size of the image to be detected. For example, if the image to be detected is a square image of W*W, it can be normalized with 100 / W. If the image to be detected is a rectangular image of W*H (W≠H), it can be normalized with 100 / (W*H).

[0117] In this alternative solution, considering that the image to be detected may have different sizes, before comparing the maximum occurrence times with the first threshold, the maximum occurrence times can also be normalized according to the size of the image to be detected first, which can improve the versatility of the solution, and the same first threshold can be used for detecting images to be detected with different sizes.

[0118] In some other implementation manners, the occurrence times of all value combinations greater than the first threshold in the statistical result can also be taken out to obtain more information about the foreign object. For example, if a relatively large number of contours are extracted in step S120, the occurrence times of each value greater than the first threshold correspond to a straight-line contour. Thus, based on these straight-line contours, the position and even the shape of the foreign object can be roughly analyzed, and so on.

[0119] Method 2:

[0120] Step B1: Pair up all contour pixels to construct pixel pairs.

[0121] Step B2: Traverse all the pixel pairs obtained in step B1. For each pixel pair, calculate the straight line passing through the two contour pixels using the coordinates of the two contour pixels.

[0122] Step B3: For each straight line obtained in step B2, calculate the distance from all contour pixels to the straight line. If the calculated distance is less than the second threshold, it is considered that the corresponding contour pixel is on the straight line; otherwise, it is considered that the corresponding contour pixel is not on the straight line.

[0123] Figure 9 is the schematic diagram of Method 2 for detecting a straight line provided by the embodiment of the present application. Refer to Figure 9 , the circles represent contour pixels, and the two red contour pixels are the pixel pair being currently processed. The dashed line y = k1 * x + b1 is the straight line passing through these two contour pixels. Step B3 is to calculate the distance from all the gray contour pixels to y = k1 * x + b1 (the distance from the red contour pixels to the straight line is 0 and can be directly determined).

[0124] Step B4: After completing the calculation in B3 for all straight lines, count the number of contour pixels included in each straight line, and determine the foreign object detection result of the image to be detected according to the relationship between the statistical result and the third threshold. Among them, if the number of contour pixels included in a straight line in the statistical result (possibly after processing such as normalization) is greater than the third threshold, it can be considered that the straight line is a straight line detected from the contour pixels. At this time, it can be considered that the number of contour pixels greater than the third threshold included in a straight line in the statistical result is the linear feature to be detected.

[0125] Step B4 is similar to step A3, except that the object of statistics changes from the occurrence times of the value combinations of the parameters to the number of contour pixels included in the straight line. Therefore, it can be understood with reference to step A3.

[0126] For example, the straight line with the largest number of contour pixels included in the statistical result can be determined first, and the number of contour pixels included in this straight line is called the maximum pixel number. Then, the maximum pixel number is normalized according to the size of the image to be detected to obtain the normalized maximum pixel number. Finally, if the normalized maximum pixel number is greater than the third threshold, it is determined that there is a foreign object with a linear feature in the image to be detected.

[0127] Method 3:

[0128] Step C1: Use the parameter θ to represent the inclination angle of the straight line passing through the origin (which can be the angle with the x-axis). θ takes discrete values at equal intervals within a preset angle range to obtain corresponding straight lines. For example, within the range of θ ∈ [-90, 90] (the unit of θ is °, omitted for simplicity), discrete values are taken at an interval of 1, that is, -90, -89, …, 90.

[0129] Figure 10 This is the schematic diagram of Method 3 for detecting straight lines provided by the embodiments of this application. Refer to Figure 10 , if θ = θ1, a straight line passing through the origin y = tanθ1 * x can be obtained, where θ1 can take values of -89, -88, …, 89. Special processing is required for the cases of θ1 = 90 or -90. At this time, the straight line equation is x = 0.

[0130] Step C2: For each straight line obtained in Step C1, calculate the distance between all contour pixels and this straight line, and distinguish which side of the straight line the contour pixels are located on. Then determine the maximum number of contour pixels that are located on the same side of this straight line and have similar distances to this straight line. Among them, "similar" can be controlled by certain thresholds.

[0131] Refer to Figure 10 , the circles represent contour pixels, and the dotted lines drawn from the contour pixels and connected to y = tanθ1 * x are the distances from the contour pixels to this straight line. Figure 10 In

[0132] Among them, 4 contour pixels with distances close to y = tanθ1 * x are determined from 5 contour pixels. For example, the distances corresponding to 5 contour pixels located on the left side of the straight line can be sorted, and the differences between adjacent distances in the sorting result are calculated. If the difference is less than a certain threshold, it is considered that the distances are close. For example, if sorted in ascending order of distance, the differences (taking absolute values) between the first 4 distances are less than the threshold of 0.5, and the difference between the 4th and 5th distances is 0.8, then it can be determined that the first 4 distances are close. Of course, there are also other methods to determine the maximum number of contour pixels with close distances, such as clustering, maximum clique search algorithm, etc., and no more examples will be given one by one.

[0133] Step C3: After completing the calculations in B3 for all straight lines, count the maximum number of contour pixels corresponding to each straight line, and determine the foreign object detection result of the image to be detected according to the relationship between the statistical result and the fourth threshold. Among them, if the maximum number of contour pixels corresponding to a straight line in the statistical result (possibly after normalization and other processes) is greater than the fourth threshold, it can be considered that this straight line is a straight line detected from the contour pixels. At this time, it can be considered that the maximum number of contour pixels greater than the fourth threshold contained in a straight line in the statistical result is the linear feature to be detected.

[0134] Step C4 is similar to step A3, except that the statistical object changes from the occurrence times of parameter value combinations to the maximum number of contour pixels corresponding to a straight line. Therefore, it can be understood by referring to step A3.

[0135] For example, the maximum value of the maximum number of contour pixels in the statistical result can be determined first; then, this maximum value is normalized according to the size of the image to be detected to obtain the normalized maximum value; finally, if the normalized maximum value is greater than the fourth threshold, it is determined that there is a foreign object with a linear feature in the image to be detected.

[0136] A simple analysis of the principle of method three is as follows: Method two is essentially based on contour pixels to find a straight line that meets certain conditions, such as the straight line passing through the most contour pixels, which may be called the first target straight line. However, the first target straight line contains two parameters, namely slope and intercept. This problem can be equivalent to: finding a second target straight line that is parallel to the first target straight line and passes through the origin of coordinates. Once the second target straight line is found, the first target straight line can also be deduced, and the second target straight line only contains one parameter, namely slope (or inclination angle), which is conducive to simplifying the problem. Since the second target straight line is only parallel to the first target straight line, the contour pixels located near the first target straight line (with a distance less than the second threshold from the first target straight line) in method two correspondingly become the contour pixels located on the same side of the second target straight line and with distances close to the second target straight line in method three.

[0137] Figure 11 The functional block diagram of the foreign object detection device 200 provided by an embodiment of the present application is shown. This device can be used to implement the foreign object detection method provided by the embodiment of the present application. Refer to Figure 11 , the foreign object detection device 200 includes:

[0138] An image acquisition module 210, configured to acquire a to-be-detected image collected by a fingerprint module disposed below the screen;

[0139] A contour extraction module 220, configured to perform contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image;

[0140] A foreign object detection module 230, configured to detect target geometric feature according to the contour pixels, and determine a foreign object detection result of the to-be-detected image according to the detection situation; wherein, the target geometric feature characterizes the feature presented by the foreign object in the to-be-detected image.

[0141] In an implementation manner of the foreign object detection device 200, the contour extraction module 220 performs contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image, including: calculating the gradient value of the pixels in the to-be-detected image row by row, and for the calculation result of each row, taking at least one pixel with the largest corresponding gradient value as the contour pixel in the pixels of this row; and / or calculating the gradient value of the pixels in the to-be-detected image column by column, and for the calculation result of each column, taking at least one pixel with the largest corresponding gradient value as the contour pixel in the pixels of this column.

[0142] In an implementation manner of the foreign object detection device 200, the target geometric figure is a straight line. The foreign object detection module 230 detects the target geometric feature according to the contour pixels, and determines the foreign object detection result of the to-be-detected image according to the detection situation, including: traversing all the contour pixels, determining a value combination of parameters corresponding to a group of straight lines passing through each contour pixel, the parameters including a first parameter and a second parameter, and the values of the parameters being quantized; counting the occurrence times of each value combination of the parameters obtained during the traversing process to obtain a statistical result; determining the foreign object detection result of the to-be-detected image according to the relationship between the statistical result and a first threshold; wherein, if the occurrence times of a value combination of a parameter in the statistical result is greater than the first threshold, then this value combination of the parameter identifies a straight line detected from the contour pixels, and the occurrence times of this value combination of the parameter is the target geometric feature corresponding to this straight line.

[0143] In an implementation of the foreign object detection device 200, the first parameter characterizes the inclination angle of a line segment passing through the origin and perpendicular to the line, the second parameter characterizes the length of a line segment passing through the origin and perpendicular to the line, the first parameter takes discrete values at equal intervals within a preset angle range, and the value of the second parameter is calculated according to the value of the first parameter and the coordinates of the contour pixels.

[0144] In an implementation of the foreign object detection device 200, the foreign object detection module 230 determines the foreign object detection result of the to-be-detected image according to the relationship between the statistical result and the first threshold, including: determining the maximum occurrence times of the value combination of the parameters in the statistical result; normalizing the maximum occurrence times according to the size of the to-be-detected image to obtain the normalized maximum occurrence times; if the normalized maximum occurrence times are greater than the first threshold, it is determined that there is a foreign object with linear characteristics in the to-be-detected image.

[0145] In an implementation of the foreign object detection device 200, the device further includes: an image processing module, configured to perform edge-preserving smoothing filtering on the to-be-detected image after the image acquisition module 210 acquires the to-be-detected image collected under the screen and before the contour extraction module 220 extracts the contour of the to-be-detected image.

[0146] In an implementation of the foreign object detection device 200, the foreign object is a white foam, and the to-be-detected image is an image collected by the fingerprint module when a black object presses the fingerprint collection area of the screen.

[0147] For the foreign object detection device 200 provided in the embodiments of the present application, its implementation principle and the technical effects produced have been introduced in the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding contents in the method embodiments.

[0148] Figure 12 This is a structural diagram of the electronic device 300 provided in the embodiments of the present application. Referring to Figure 12 , the electronic device 300 includes: a processor 310 and a memory 320, which are interconnected and communicate with each other through a communication bus 330 and / or other forms of connection mechanisms (not shown).

[0149] Among them, the memory 320 includes one or more (only one is shown in the figure), which can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The processor 310 and other possible components can access, read, and / or write data in the memory 320.

[0150] The processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with the ability to process signals. The above-mentioned processor 310 can be a general-purpose or dedicated processor, such as a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0151] One or more computer program instructions can be stored in the memory 320, and the processor 310 can read and run these computer program instructions to implement the foreign object detection method provided by the embodiments of the present application and other desired functions.

[0152] It can be understood that Figure 12 The structure shown is only schematic, and the electronic device 300 may further include more or fewer components than those Figure 12 shown in Figure 12The different configurations shown. For example, the electronic device 300 may further include a communication module for communicating with other electronic devices. For another example, the electronic device 300 may further include a screen and a fingerprint module, and the fingerprint module is disposed under the screen (reference may be made to Figure 1 ), for collecting an image to be detected, and so on.

[0153] In addition, Figure 12 Each component shown in may be implemented by hardware, software, or a combination thereof. The electronic device 300 may be a physical device, such as a PC, a laptop, a tablet, a mobile phone, a server, an embedded device, etc., or may be a virtual device, such as a virtual machine, a virtualization container, etc. And, the electronic device 300 is not limited to a single device, and may also be a combination of multiple devices or a cluster composed of a large number of devices.

[0154] The embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored. When these computer program instructions are read and run by a processor of a computer, some or all of the steps of the sports training assistance method provided by the embodiment of the present application are executed. For example, the computer-readable storage medium may be implemented as Figure 12 the memory 320 in the electronic device 300 in, or an external hard drive, an optical disc, a USB flash drive, etc. independent of the electronic device 300.

[0155] The embodiment of the present application further provides a computer program product, which includes computer program instructions. When these computer program instructions are read and run by a processor of a computer, some or all of the steps of the sports training assistance method provided by the embodiment of the present application are executed. For example, the computer program product may be stored in Figure 12 the memory 320 in the electronic device 300 in, and read and run by the processor 310. Of course, the computer program product may also be stored in a storage medium independent of the electronic device 300, such as an external hard drive, an optical disc, a USB flash drive.

[0156] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A foreign object detection method, characterized in that, Including: Obtain a to-be-detected image collected by a fingerprint module disposed below the screen; Perform contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image; Detect target geometric shape features based on the contour pixels, and determine a foreign object detection result of the to-be-detected image according to the detection situation; wherein, the target geometric shape features represent the features presented by a foreign object in the to-be-detected image.

2. The foreign object detection method according to claim 1, characterized in that The performing contour extraction on the to-be-detected image to obtain contour pixels in the to-be-detected image includes: Calculate the gradient value of pixels in the to-be-detected image row by row, and for the calculation result of each row, take at least one pixel with the largest corresponding gradient value as a contour pixel in the pixels of this row; and / or, Calculate the gradient value of pixels in the to-be-detected image column by column, and for the calculation result of each column, take at least one pixel with the largest corresponding gradient value as a contour pixel in the pixels of this column.

3. The foreign object detection method according to claim 1, characterized in that The target geometric shape is a straight line, and the detecting target geometric shape features based on the contour pixels and determining a foreign object detection result of the to-be-detected image according to the detection situation includes: Traverse all the contour pixels to determine a value combination of parameters corresponding to a group of straight lines passing through each contour pixel, the parameters include a first parameter and a second parameter, and the values of the parameters are quantized; Count the occurrence times of each value combination of the parameters obtained during the traversal to obtain a statistical result; Determine a foreign object detection result of the to-be-detected image according to the relationship between the statistical result and a first threshold; wherein, if the occurrence times of a value combination of a parameter in the statistical result is greater than the first threshold, then this value combination of the parameter identifies a straight line detected from the contour pixels, and the occurrence times of this value combination of the parameter is the target geometric shape feature corresponding to this straight line.

4. The foreign object detection method according to claim 3, characterized in that The first parameter represents the inclination angle of a line segment passing through the origin and perpendicular to the straight line, the second parameter represents the length of a line segment passing through the origin and perpendicular to the straight line, the first parameter takes discrete values at equal intervals within a preset angle range, and the value of the second parameter is calculated according to the value of the first parameter and the coordinates of the contour pixels.

5. The foreign object detection method according to claim 3, wherein, The determining a foreign object detection result of the to-be-detected image according to the relationship between the statistical result and a first threshold includes: Determine the maximum occurrence times of the value combinations of the parameters in the statistical result; Normalize the maximum occurrence times according to the size of the to-be-detected image to obtain a normalized maximum occurrence times; If the normalized maximum occurrence times is greater than the first threshold, then determine that there is a foreign object with linear features in the to-be-detected image.

6. The foreign object detection method according to claim 1, characterized in that, After obtaining the to-be-detected image collected by the acquisition disposed below the screen and before performing contour extraction on the to-be-detected image, the method further includes: Perform edge-preserving smoothing filtering on the to-be-detected image.

7. The foreign object detection method according to any one of claims 1-6, characterized in that The foreign object is white foam, and the to-be-detected image is an image collected by the fingerprint module when a black object presses the fingerprint collection area of the screen.

8. A computer program product, characterized in that, Including computer program instructions, when the computer program instructions are read and run by a processor, the method according to any one of claims 1-7 is executed.

9. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are read and run by a processor, the method according to any one of claims 1-7 is executed.

10. An electronic device, characterized in that, Including a memory and a processor, computer program instructions are stored in the memory, and when the computer program instructions are read and run by the processor, the method according to any one of claims 1-7 is executed.

Images