Image screening method and electronic device

Abstract

One of the objectives of this invention is to filter objects in an image based on their clarity. [Solution] The present invention provides an image screening method and an electronic device. The method includes: capturing a digital image through a photographic lens assembly; extracting multiple objects from the digital image; performing a spatial frequency transform on the digital image to obtain a frequency image; removing a first portion of the frequency image and retaining a second portion of the frequency image; performing a filtering process on the second portion to obtain a filtered image; performing a frequency-space transform on the filtered image to obtain a reconstructed image; and screening objects by comparing the reconstructed image with the digital image.

Description

[Technical Field]

[0001] The present invention relates to a method and electronic apparatus capable of screening clear objects from digital images. [Background technology]

[0002] Conventional image focusing techniques typically determine whether an image is in focus by mechanically scanning in the depth of field direction, and then determine the focal plane position based on the image clarity to acquire a sharp image. This usually requires mechanical focus scanning and is not applicable to fast-moving objects because these objects cannot pause for scanning in the depth direction. In particular, in high-speed fluid environments such as fluid microscopy techniques like flow cytometry, the high detection throughput characteristics make flow velocity and image acquisition speed crucial factors affecting detection efficiency.

[0003] According to Abbe's diffraction limit theory, increasing the numerical aperture (NA) of the objective lens can effectively improve image resolution. However, a larger numerical aperture results in a shallower depth of field (DOF) during imaging, potentially leading to a mixture of sharp and out-of-focus images. Therefore, in high-speed fluid microscopy, effectively selecting sharp objects from captured images and excluding out-of-focus objects is an urgent technical challenge that needs to be addressed. [Overview of the project] [Problems that the invention aims to solve]

[0004] One of the objectives of this invention is to effectively select clear objects from captured images and exclude out-of-focus objects. [Means for solving the problem]

[0005] The present invention presents an image screening method applicable to electronic devices. This image screening method includes: capturing a digital image through a photographic lens assembly; extracting multiple objects from the digital image; performing a spatial frequency transformation on the digital image to obtain a frequency image; removing a first portion of the frequency image and retaining a second portion of the frequency image; performing a filtering process on the second portion to obtain a filtered image; performing a frequency-space transformation on the filtered image to obtain a reconstructed image; and screening objects by comparing the reconstructed image with the digital image.

[0006] In one embodiment of the present invention, the imaging lens assembly includes an infinity-corrected objective lens, and the field of view of the imaging lens assembly includes a fluid.

[0007] In one embodiment of the present invention, removing a first portion of a frequency image and retaining a second portion of the frequency image includes setting a mask that includes a central region and a peripheral region, and applying the mask to the frequency image to remove the first portion located in the central region and retain the second portion located in the peripheral region, wherein the first portion is a low-frequency portion and the second portion is a high-frequency portion.

[0008] In one embodiment of the present invention, performing a filtering process on the second portion to obtain a filtered image includes applying a Gaussian filter to the edges of the high-frequency portion to generate a plurality of reconstructed frequency coefficients located in the central region.

[0009] In one embodiment of the present invention, screening objects by comparing a reconstructed image with a digital image includes: obtaining, for each object, a plurality of corresponding reconstructed pixels in the reconstructed image and a plurality of corresponding original pixels in the digital image, wherein the positions of the reconstructed pixels are the same as the positions of the original pixels; obtaining a fraction by dividing the statistical value of the plurality of grayscale values ​​of the reconstructed pixels by the statistical value of the plurality of grayscale values ​​of the original pixels; and screening objects by determining whether the score corresponding to each object is greater than a threshold.

[0010] From another perspective, one embodiment of the present invention provides an electronic device comprising a photographic lens assembly, an image sensor, and a processor. The image sensor is used to capture a digital image via the photographic lens assembly. The processor is electrically connected to the image sensor and is used to perform the image screening method described above. [Effects of the Invention]

[0011] The above method and apparatus can screen for clear objects using frequency analysis and can solve the challenges faced when photographing objects in a fluid using an infinity-corrected objective lens.

[0012] To allow for a clearer understanding of the above-mentioned features and advantages of the present invention, embodiments are described below and explained in detail with reference to the accompanying drawings. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of an electronic device according to one embodiment. [Figure 2] This is a schematic diagram of a digital image according to one embodiment. [Figure 3] This is a flowchart of an image screening method according to one embodiment. [Figure 4] This is a schematic diagram illustrating the operation of a frequency image according to one embodiment. [Figure 5]This is a schematic diagram of a path according to one embodiment. [Modes for carrying out the invention]

[0014] Several embodiments of the present invention will be described in detail below with reference to the drawings. Reference numerals used in the following description indicate the same or similar elements when the same numerals are used in different drawings. These embodiments represent only a part of the present invention and do not disclose all possible embodiments of the present invention. Rather, these embodiments are merely illustrative of systems and methods within the scope of the present application.

[0015] The terms "first," "second," etc., used herein do not specifically refer to order or sequence, but are used solely to distinguish between elements or operations described by the same technical terminology.

[0016] Figure 1 is a schematic diagram of an electronic device according to one embodiment. Referring to Figure 1, in this embodiment, the electronic device 100 is a detection device, but the present invention is not limited thereto. In this example, the electronic device 100 includes a channel 110, a photographic lens assembly 120, an image sensor 130, and a processor 140. In this embodiment, the electronic device 100 is a fluid detection device, but the present invention is not limited thereto. A fluid 111 is present in the channel 110, which flows in from the top of Figure 1 along the flow direction 112, passes through the field of view of the photographic lens assembly 120, and flows out from the bottom of Figure 1. In other words, the field of view of the photographic lens assembly 120 includes the fluid 111. Light 113 reflected from the fluid 111 passes through the photographic lens assembly 120 and is received by the image sensor 130, thereby generating a digital image. In this embodiment, the photographic lens assembly 120 includes an infinity corrected objective lens. The numerical aperture (NA) of this infinity-corrected objective lens is below a threshold. This critical value is, for example, 0.42, but the present invention is not limited to this.

[0017] The image sensor 130 may include a charge-coupled device (CCD) sensor, a complementary metal-oxide semiconductor (CMOS) sensor, or other suitable photosensitive elements. In some embodiments, the image sensor 130 may further include a dual camera, a structured light sensing device, a laser, or other elements capable of detecting the depth of the scene.

[0018] The processor 140 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, an image processing chip, an application specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The processor 140 is electrically connected to the image sensor 130 and executes an image screening method on the digital image. This method will be described in detail below.

[0019] FIG. 2 is a schematic diagram of a digital image according to an embodiment. The digital image 200 includes particles (e.g., particles 201-203) in a fluid. Since each particle may be located on a different focal plane (having a different depth with respect to the image sensor 130), it is difficult for the optical system to sharply focus all the planes simultaneously at a single focus, and as a result, the sharpness of these particles is different from each other. For example, particle 201 is relatively sharp, particle 202 is slightly blurred, and particle 203 is more blurred. The image screening method proposed herein can classify these particles based on sharpness. The image screening method proposed below can also be applied to other types of images, such as natural landscapes, portrait photos, optical inspections in factories, etc., and the present invention is not limited thereto.

[0020] FIG. 3 is a flowchart of an image screening method according to an embodiment. Referring to FIG. 3, in step 301, a digital image is taken through a photographing lens assembly 120. In this embodiment, the digital image is a color image including red, green, and blue channels. In some embodiments, the red, green, and blue channels can be converted into other color spaces including luminance such as YUV, YCbCr, or HIS, but the present invention is not limited thereto. The following image processing can be applied to any channels such as luminance, red, green, and blue. In some embodiments, the digital image may first be preprocessed such as noise removal or local contrast enhancement.

[0021] In step 302, a plurality of objects are extracted from the digital image. For obtaining the objects, any known segmentation algorithm can be adopted. For example, a neural network can also be adopted for segmentation. In some embodiments, the digital image is first binarized based on luminance. For example, luminance above a threshold is set to 0, and luminance below the threshold is set to 1. Next, operations such as erosion and dilation are performed on the binarized image to obtain a plurality of objects. In the embodiment of FIG. 2, these objects are particles 201 to 203. In other embodiments, the objects to be photographed may be electronic components, component defects, tumors in medical images, etc., and the present invention is not limited thereto.

[0022] In step 303, spatial frequency conversion is performed on the digital image to obtain a frequency image. This spatial frequency conversion is, for example, a fast Fourier transform (FFT), but in other embodiments, it may be a wavelet transform. FIG. 4 is a schematic diagram showing operations related to a frequency image according to an embodiment. Referring to FIG. 4, a frequency image 410 is obtained by spatial frequency conversion. The pixels in this image represent the amplitude of the frequency coefficients, and this amplitude also represents the magnitude of the energy.

[0023] Referring to Figures 3 and 4, step 304 then removes a first portion of the frequency image and retains a second portion of the frequency image. In some embodiments, the first portion to be removed is the low-frequency portion, and the second portion to be retained is the high-frequency portion. Specifically, one mask 420 can be set first. The size of this mask 420 is the same as the size of the frequency image 410. This mask 420 includes a central region 421 and a peripheral region 422, where the peripheral region 422 includes the central region 421. The central region 421 corresponds to the low-frequency portion, and the peripheral region 422 corresponds to the high-frequency portion. In this example, the value of the central region 421 is "0" and the value of the peripheral region 422 is "1". Next, the mask 420 is applied to the frequency image 410. This is done by performing an AND operation between the values ​​of the frequency image 410 and the corresponding values ​​in the mask 420. This makes it possible to remove the first portion 431 located in the central region 421 and retain the second portion 432 located in the peripheral region 422. In other embodiments, the value of the central region 421 may be set to "1", the value of the peripheral region 422 may be set to "0", and then an OR operation may be performed; however, the present invention is not limited thereto. For illustrative purposes, the position of the central region 421 will be represented below as the position of the low-frequency portion, and the position of the peripheral region 422 will be represented as the position of the high-frequency portion.

[0024] In this embodiment, the radius of the central region 421 is determined artificially. In other embodiments, the radius of the central region 421 may be determined based on the energy ratio of the low-frequency portion. For example, multiple sample images may be collected first, and a spatial frequency transformation may be performed on each sample image. The larger the radius of the central region 421, the greater the energy covered. Here, an energy ratio (e.g., 30%) may be set, the radius corresponding to each sample image may be calculated, and finally, the average of these radii may be calculated.

[0025] In step 305, a filtering process is performed on the second portion 432 to obtain a filtered image. In this embodiment, one filter 440 can be used. This filter 440 is, for example, a Gaussian filter, but the present invention is not limited thereto. Furthermore, the present invention does not limit the size of the filter 440, the parameters of the Gaussian function, etc. Next, the filter 440 is applied to the edge 450 of the second portion 432 to generate a coefficient located in the central region 421 (called the reconstructed frequency coefficient). The filter 440 covers part of the surrounding region 422 and also covers part of the central region 421. The center point of the filter 440 is located in the central region 421. The reconstructed frequency coefficient generated after the filtering process is used to replace the value of the center point of the filter 440 (which was originally removed). The purpose of doing this is to avoid discontinuities in the frequency coefficient and to prevent ripple effects in the subsequent frequency space transformation. In some embodiments, the filter 440 moves along a path 460. Figure 5 is a schematic diagram of a path according to one embodiment. Referring to Figure 5, this path includes multiple concentric circles 501-503. The center point of the filter 440 moves along the concentric circles 501-503. In some embodiments, the filter 440 first moves along the outer concentric circle 501, completes a full circle, then moves along the central concentric circle 502, and finally moves along the inner concentric circle 503. In other words, the filtering process first generates the reconstruction frequency coefficients for the outer periphery of the central region 421, and then gradually generates the reconstruction frequency coefficients for the inner periphery. With such a method, relatively continuous reconstruction frequency coefficients can be generated, and the present invention does not limit the number of concentric circles.

[0026] Referring to Figures 3 and 4, in step 306, a frequency-space transform is performed on the filtered image to obtain a reconstructed image. This frequency-space transform is, for example, an inverse Fourier transform.

[0027] Next, in step 307, the reconstructed image and the digital image are compared to screen the object. Specifically, for each of the objects photographed in step 302, a plurality of corresponding reconstructed pixels in the reconstructed image and a plurality of corresponding original pixels in the digital image are acquired. The positions of these reconstructed pixels are the same as the positions of the original pixels, and both are located within the corresponding object. Next, a statistic of the plurality of grayscale values of the reconstructed pixels is divided by a statistic of the plurality of grayscale values of the original pixels to obtain a fraction. This statistic can be either a sum or an average value, and the results of both are the same. Specifically, when the statistic is a sum, the above calculation can be expressed by the following formula 1. [Number] In the formula, o i represents the i-th object. i is a positive integer. x j represents the grayscale value of the reconstructed pixel corresponding to the i-th object, and j represents the coordinates within the i-th object. y i represents the grayscale value of the original pixel corresponding to the i-th object. The above grayscale value may belong to the luminance channel or may belong to other channels such as green, red, and blue. S i represents the score of the i-th object. The larger the score S i , the more high-frequency components are included in the i-th object in the original digital image, which also indicates that the image is clearer. Conversely, the smaller the score S i , the more blurred the i-th object is.

[0028] Here, the object can be screened based on the score S i . For example, it is determined whether the score S i corresponding to each object is greater than a threshold value (for example, 80%, 85%, or other values) to screen the object, and an object having a score greater than the threshold value can be selected. In other embodiments, the score S iMultiple objects can also be divided into multiple groups based on this. For example, in Figure 2, particle 201 belongs to the group with the highest score and represents the clearest object. Particle 202 belongs to the group with the second highest score and represents the least clear but acceptable object. Particle 203 belongs to the group with the lowest score and is excluded. In some embodiments, the objects after screening can undergo subsequent processing, but the present invention does not limit the content of the subsequent processing.

[0029] The image screening method described above effectively overcomes inaccuracies caused by optical focus shifts by combining spatial domain analysis and frequency domain analysis, and reduces the effects of depth-of-field changes during multi-view observation. By using objective scores, this technology can reduce the possibility of misjudgment.

[0030] The present invention is disclosed with reference to the embodiments described above, but these do not limit the invention. Those skilled in the art can make some changes and modifications without departing from the spirit and scope of the invention. Accordingly, the scope of protection of the present invention shall be determined by the scope of the appended patent application. [Industrial applicability]

[0031] The image screening method proposed above is highly versatile and can be widely applied to various optical imaging systems, such as in fields requiring high-precision measurement of semiconductors. [Explanation of symbols]

[0032] 100:Electronic equipment 110: Channel 111:Fluid 112:Flow direction 113: Light 120: Photographing lens assembly 130: Image sensor 140: Processor 200: Digital Images 201~203: Particles 301~307: Process 410: Frequency image 420: Mask 421: Central area 422: Peripheral area 431: Part 1 432:Second part 440: Filter 450: Edge 460: Pass 501-503: Concentric circles

Claims

1. An image screening method applicable to electronic devices, Taking a digital image through a photographic lens assembly, Extracting multiple objects from the aforementioned digital image, The process involves performing a spatial frequency transformation on the aforementioned digital image to obtain a frequency image, wherein the frequency image includes a low-frequency portion located in the center and a high-frequency portion located outside the low-frequency portion. The low-frequency portion of the frequency image is removed, and the high-frequency portion of the frequency image is retained. The process involves performing a filtering operation on the aforementioned high-frequency portion to obtain a filtered image, The filtered image is subjected to frequency space transformation to obtain a reconstructed image, The process involves comparing the reconstructed image with the digital image to screen the multiple objects, Image screening methods, including...

2. The aforementioned photographic lens assembly includes an infinity-corrected objective lens, and the field of view of the photographic lens assembly includes a fluid. The image screening method according to claim 1.

3. Removing the low-frequency portion of the frequency image and retaining the high-frequency portion of the frequency image is, Setting a mask that includes a central region and a peripheral region, wherein the central region has a predetermined radius. Applying the mask to the frequency image to identify the portion of the frequency image located in the central region as the low-frequency portion, This includes removing the low-frequency portion located in the central region and retaining the high-frequency portion located in the peripheral region. The image screening method according to claim 1.

4. Performing the filtering process on the high-frequency portion and obtaining the filtered image is, A Gaussian filter is applied to the edges of the high-frequency portion to generate a plurality of reconstructed frequency coefficients located in the central region. The image screening method according to claim 3, including the method described in claim 3.

5. Screening the multiple objects by comparing the reconstructed image with the digital image is For each of the aforementioned multiple objects, the acquisition of a corresponding number of reconstructed pixels in the reconstructed image and a corresponding number of original pixels in the digital image, wherein the positions of the multiple reconstructed pixels are the same as the positions of the multiple original pixels. The score is obtained by dividing the statistical values ​​of the multiple grayscale values ​​of the multiple reconstructed pixels by the statistical values ​​of the multiple grayscale values ​​of the multiple original pixels. The process involves determining whether the score corresponding to each of the aforementioned objects is greater than a threshold, and screening the aforementioned objects. The image screening method according to claim 4, including the method described in claim 4.

6. The photographic lens assembly and An image sensor used to capture a digital image via the aforementioned photographic lens assembly, A processor electrically connected to the image sensor, Extracting multiple objects from the aforementioned digital image, The process involves performing a spatial frequency transformation on the aforementioned digital image to obtain a frequency image, wherein the frequency image includes a low-frequency portion located in the center and a high-frequency portion located outside the low-frequency portion. The low-frequency portion of the frequency image is removed, and the high-frequency portion of the frequency image is retained. The process involves performing a filtering operation on the aforementioned high-frequency portion to obtain a filtered image, The filtered image is subjected to frequency space transformation to obtain a reconstructed image, The process involves comparing the reconstructed image with the digital image to screen the multiple objects, The processor used to perform multiple steps, Electronic devices, including those mentioned above.

7. The aforementioned photographic lens assembly includes an infinity-corrected objective lens, and the field of view of the photographic lens assembly includes a fluid. The electronic device according to claim 6.

8. Removing the low-frequency portion of the frequency image and retaining the high-frequency portion of the frequency image is, Setting a mask that includes a central region and a peripheral region, wherein the central region has a predetermined radius. Applying the mask to the frequency image to identify the portion of the frequency image located in the central region as the low-frequency portion, This includes removing the low-frequency portion located in the central region and retaining the high-frequency portion located in the peripheral region. The electronic device according to claim 6.

9. Performing the filtering process on the high-frequency portion and obtaining the filtered image is, A Gaussian filter is applied to the edges of the high-frequency portion to generate a plurality of reconstructed frequency coefficients located in the central region. The electronic device according to claim 8, including the electronic device described in claim 8.

10. Screening the multiple objects by comparing the reconstructed image with the digital image is For each of the aforementioned multiple objects, the acquisition of a corresponding number of reconstructed pixels in the reconstructed image and a corresponding number of original pixels in the digital image, wherein the positions of the multiple reconstructed pixels are the same as the positions of the multiple original pixels. The score is obtained by dividing the statistical values ​​of the multiple grayscale values ​​of the multiple reconstructed pixels by the statistical values ​​of the multiple grayscale values ​​of the multiple original pixels. The process involves determining whether the score corresponding to each of the aforementioned objects is greater than a threshold, and screening the aforementioned objects. The electronic device according to claim 9, including the electronic device described in claim 9.

Images