Image processing method, image processing device, and optical detection apparatus

Abstract

This disclosure relates to an image processing method, an image processing apparatus, and an optical detection device. The image processing method includes: acquiring an image to be processed, wherein the image to be processed includes a foreground region and a background region with different brightness; performing binarization processing on the image to be processed, and determining one or more foreground regions in the image to be processed based on the result of the binarization processing; and for each of the one or more foreground regions, determining whether the foreground region is a target object image, and if the foreground region is a target object image, determining target object parameters based on the foreground region.

Description

Technical Field

[0001] This disclosure relates to the field of optical inspection technology, and more specifically, to an image processing method, an image processing device, and an optical inspection apparatus. Background Technology

[0002] Optical detection is increasingly widely used in fields such as chemistry and biology. Through optical detection, target objects (such as cells, cell debris, yeast, algae, and other biological particles) in samples can be counted, their morphology observed, and their location determined, thereby obtaining relevant properties of the sample. However, in current optical detection methods, to better observe the sample, it is often necessary to pre-treat the target objects, especially transparent ones, by staining. This complicates the optical detection process, making it inconvenient for direct online, in-situ sample detection. Furthermore, staining and other treatments may cause other changes to the sample, leading to inaccurate detection results. Summary of the Invention

[0003] One of the purposes of this disclosure is to provide an image processing method, an image processing device, and an optical detection apparatus.

[0004] According to a first aspect of this disclosure, an image processing method is provided, the image processing method comprising:

[0005] Obtain an image to be processed, wherein the image to be processed includes a foreground region and a background region with different brightness;

[0006] The image to be processed is binarized, and one or more foreground regions in the image to be processed are determined based on the result of the binarization process; and

[0007] For each of the one or more foreground regions, determine whether the foreground region is a target object image, and if the foreground region is a target object image, determine the target object parameters based on the foreground region.

[0008] In some embodiments, the image to be processed is obtained by photographing a sample containing the target object under a combination of bright field illumination and dark field illumination.

[0009] In some embodiments, the target object in the sample has a different refractive index than other parts of the sample, the target object is transparent, and the target object is not dyed.

[0010] In some embodiments, binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization process includes:

[0011] The grayscale value of each pixel in the image to be processed is compared with a preset grayscale threshold.

[0012] Based on the comparison result between the gray value of the pixel and the preset gray value threshold, it is determined whether the pixel belongs to the first pixel of the foreground region or the second pixel of the background region.

[0013] Divide the continuously distributed first pixels into the same foreground region;

[0014] In this process, different foreground regions in the image to be processed are separated by background regions.

[0015] In some embodiments, binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization further includes:

[0016] Based on the brightness distribution of the image to be processed, a preset grayscale threshold corresponding to the image to be processed is determined.

[0017] In some embodiments, determining whether a pixel belongs to a first pixel in the foreground region or a second pixel in the background region based on a comparison between the pixel's grayscale value and the preset grayscale threshold includes:

[0018] When the grayscale value of the pixel is greater than the preset grayscale threshold, the pixel is determined to be the first pixel belonging to the foreground region;

[0019] When the grayscale value of the pixel is less than or equal to the preset grayscale threshold, the pixel is determined to be the second pixel belonging to the background region.

[0020] In some embodiments, binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization further includes:

[0021] Correct the defect in at least one of the one or more foreground regions.

[0022] In some embodiments, correcting a defect in at least one foreground region within the one or more foreground regions includes:

[0023] For at least one second pixel in the image to be processed, the gray value of the second pixel is compared with the average gray value of a preset number of pixels surrounding the second pixel;

[0024] Based on the comparison result between the gray value of the second pixel and the average gray value, it is determined whether to correct the second pixel to the first pixel.

[0025] In some embodiments, determining whether to correct the second pixel to the first pixel based on a comparison between the grayscale value of the second pixel and the average grayscale value includes:

[0026] When the grayscale value of the second pixel is less than or equal to the average grayscale value, the second pixel is corrected to the first pixel.

[0027] In some embodiments, correcting a defect in at least one foreground region within the one or more foreground regions includes:

[0028] For at least one of the one or more foreground regions, determine whether the edge of the foreground region matches the edge of a preset shape;

[0029] When the edge of the foreground region does not conform to the edge of a preset shape, the foreground region is expanded to correct its edge; and

[0030] The expanded foreground region is eroded to make the area of ​​the corrected foreground region the same as the area of ​​the original foreground region.

[0031] In some embodiments, the preset shape edge includes at least one of a circular arc edge and an elliptical arc edge.

[0032] In some embodiments, the foreground region is represented by the coordinates of pixels located on the edges of the foreground region.

[0033] In some embodiments, for each of the one or more foreground regions, determining whether the foreground region is a target object image, and if the foreground region is a target object image, determining the target object parameters based on the foreground region includes:

[0034] Determine whether the foreground region is a target object image based on the edges of the foreground region;

[0035] When the foreground region is a target object image, it is determined whether the target objects contained in the foreground region are in a clustered state;

[0036] When the target objects contained in the foreground region are in a clustered state, the multiple target objects in the clustered state are separated according to a preset algorithm, and the target object parameters of each of the multiple target objects are determined.

[0037] When the target objects contained in the foreground region are not in a clustered state, the target object parameters of the target objects are determined.

[0038] In some embodiments, determining whether the foreground region is a target object image based on the edge of the foreground region includes:

[0039] Based on the edges of the foreground region, determine an envelope rectangle that can contain the foreground region and has the smallest size;

[0040] Compare the width of the envelope rectangle with a first preset threshold;

[0041] When the width of the envelope rectangle is greater than or equal to the first preset threshold, the foreground region is determined to be the target object image;

[0042] When the width of the envelope rectangle is less than the first preset threshold, it is determined that the foreground region is not the target object image.

[0043] In some embodiments, when the foreground region is a target object image, determining whether the target objects contained in the foreground region are in a clustered state includes:

[0044] The width of the envelope rectangle is compared with a second preset threshold, and the length of the envelope rectangle is compared with the second preset threshold.

[0045] When the width and length of the envelope rectangle are both less than or equal to the second preset threshold, it is determined that the target objects contained in the foreground region are not in a clustered state.

[0046] When at least one of the width and length of the envelope rectangle is greater than the second preset threshold, it is determined that the target objects contained in the foreground region are in a clustered state.

[0047] Wherein, the second preset threshold is greater than or equal to the first preset threshold.

[0048] In some embodiments, when the target objects contained in the foreground region are in a clustered state, separating the multiple target objects in the clustered state according to a preset algorithm includes:

[0049] Separate multiple target objects in a clustered state using at least one of the following algorithms: Hough circle recognition algorithm, watershed algorithm, hotspot detection algorithm, support vector machine algorithm, and u-net algorithm.

[0050] In some embodiments, the target object parameters include at least one of the target object count, target object size, and target object location.

[0051] In some embodiments, before binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization, the image processing method further includes performing at least one of the following preprocessing steps on the image to be processed:

[0052] Adjust at least one of the contrast and brightness of the image to be processed;

[0053] The image to be processed is converted to grayscale.

[0054] The image to be processed is normalized; and

[0055] The image to be processed is then subjected to noise reduction.

[0056] In some embodiments, the image processing method further includes:

[0057] After determining the target object parameters for all foreground regions in the image to be processed, the target object parameters are displayed in the form of at least one of a labeled map and a labeled list.

[0058] In some embodiments, the target object includes cells.

[0059] According to a second aspect of this disclosure, an image processing apparatus is provided, the image processing apparatus including a processor and a memory, the memory storing instructions that, when executed by the processor, implement the steps of the image processing method as described above.

[0060] According to a third aspect of this disclosure, an optical detection device is provided, the optical detection device comprising:

[0061] Light source device, the light source device comprising:

[0062] An illumination source, the illumination source being configured to generate illumination light; and

[0063] An aperture stop, wherein the aperture stop is disposed in the outgoing light path of the illumination source, the aperture stop comprising:

[0064] A light shield, configured to block part of the illumination light;

[0065] A first light-transmitting portion is formed on the light-shielding screen and covers the center of the aperture. The first light-transmitting portion is configured to allow partial illumination light to pass through to form bright-field illumination of the sample; and

[0066] The second light-transmitting part is formed on the light-shielding screen and is located around the first light-transmitting part. The second light-transmitting part is configured to allow some illumination light to pass through in order to form dark field illumination of the sample.

[0067] A sample stage, configured to hold the sample;

[0068] An imaging device configured to image the sample under the illumination of the light source device to produce an image to be processed, and the imaging device includes an objective lens; and

[0069] The image processing device described above.

[0070] In some embodiments, the distance R1 between the outer edge of the first light-transmitting portion and the center of the aperture, the distance l between the aperture and the sample stage, and the numerical aperture n of the objective lens satisfy the following relationship:

[0071] R1≤l·tg[arcsin(n) / 3].

[0072] In some embodiments, the distance R2 between the inner edge of the second light-transmitting portion and the center of the aperture, the distance l between the aperture and the sample stage, and the numerical aperture n of the objective lens satisfy the following relationship:

[0073] R2>l·tg[arcsin(n)].

[0074] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided, on which instructions are stored, which, when executed, implement the steps of the image processing method as described above.

[0075] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product comprising instructions that, when executed by the processor, implement the steps of the image processing method as described above.

[0076] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0077] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0078] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0079] Figure 1 A schematic flowchart of an image processing method according to an exemplary embodiment of the present disclosure is shown;

[0080] Figure 2 A schematic diagram of the structure of a light source device, a sample, and an objective lens according to an exemplary embodiment of the present disclosure is shown;

[0081] Figure 3 It shows Figure 2 A schematic diagram showing the structural parameters of the light source device, sample, and objective lens in the image.

[0082] Figure 4A flowchart illustrating step S200 of an image processing method according to an exemplary embodiment of the present disclosure is shown;

[0083] Figure 5 An image to be processed is shown in a specific example of this disclosure;

[0084] Figure 6 A flowchart illustrating step S200 of an image processing method according to another exemplary embodiment of the present disclosure is shown.

[0085] Figure 7 A flowchart illustrating step S300 of an image processing method according to an exemplary embodiment of the present disclosure is shown;

[0086] Figure 8 A block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure is shown.

[0087] Note that in the embodiments described below, the same reference numerals are sometimes used across different figures to denote the same parts or parts having the same function, and repeated descriptions are omitted. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0088] For ease of understanding, the positions, dimensions, and extents of the structures shown in the accompanying drawings and other materials may not represent actual positions, dimensions, and extents. Therefore, the disclosed invention is not limited to the positions, dimensions, and extents disclosed in the accompanying drawings and other materials. Furthermore, the drawings are not necessarily drawn to scale, and some features may be enlarged to show details of specific components. Detailed Implementation

[0089] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0090] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. That is, the chip testing methods and computing chips described herein are shown in an exemplary manner to illustrate different embodiments of the circuits or methods in this disclosure, and are not intended to be limiting. Those skilled in the art will understand that they merely illustrate exemplary ways that can be used to implement the invention, and not exhaustive ways.

[0091] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0092] To achieve online, in-situ optical detection of target objects, especially transparent target objects, this disclosure proposes an image processing method, an image processing device, and an optical detection apparatus. In the technical solution of this disclosure, the potential target objects are analyzed based on the brightness distribution in the image to be processed obtained from the sample. In the following description, the technical solution of this disclosure will be mainly illustrated using the example of a transparent living cell as the target object. However, those skilled in the art will understand that the target object can also be other biological particles such as dead cells, algae, or yeast.

[0093] In one exemplary embodiment of this disclosure, such as Figure 1 As shown, image processing methods may include:

[0094] Step S100: Obtain the image to be processed, wherein the image to be processed includes a foreground region and a background region with different brightness.

[0095] Specifically, due to the difference in refractive index of the target object and other objects (such as impurities, substrate, etc.) in the sample, the resulting image can have a certain brightness distribution. The brightness of the foreground region can be brighter or darker than that of the background region. In subsequent steps, depending on the optical properties of the target object to be analyzed, the corresponding foreground region can be extracted from the image, and the rest of the image can be used as the background region. It should be noted that when analyzing multiple target objects contained in the same image, the foreground and background regions determined for each target object in subsequent steps can be different.

[0096] In some embodiments, a sample that may contain a target object can be photographed under a combination of bright field illumination and dark field illumination to obtain an image to be processed.

[0097] like Figure 2 As shown, the light source device for lighting may include a lighting source (not shown) and an aperture 112.

[0098] The illumination source can be configured to generate illumination light, which is generally in the visible light band. In some embodiments, the illumination source may include at least one of a thermal radiation source and a light-emitting diode to generate visible light, such as white light or near-white light, thereby facilitating optical observation of the sample.

[0099] The aperture 112 can be disposed in the output light path of the illumination source, and by adjusting the transmitted portion of the illumination light, a composite illumination combining bright-field and dark-field illumination can be generated at the sample position where the sample 200 is located. Specifically, the aperture 112 may include a light-shielding screen 112a, a first light-transmitting portion 112b, and a second light-transmitting portion 112c. The light-shielding screen 112a can be made of a non-transparent material and is configured to block part of the illumination light. The first light-transmitting portion 112b is formed on the light-shielding screen 112a, covering the center of the aperture 112, and is configured to allow part of the illumination light to pass through to form bright-field illumination. The second light-transmitting portion 112c is also formed on the light-shielding screen 112a, located around the first light-transmitting portion 112b, and is configured to allow part of the illumination light to pass through to form dark-field illumination.

[0100] like Figure 2 As shown, when the illumination light incident on the sample 200 at a small angle through the first light-transmitting part 112b interacts with the sample 200, a portion of direct light 410 (white area) close to the optical axis is generated. This corresponds to bright-field illumination and can enhance the brightness of the entire observation field, thus improving the observation effect. When the illumination light incident on the sample 200 at a larger angle through the second light-transmitting part 112c interacts with the sample 200, scattered light 420 (gray area) relatively far from the optical axis is generated. This corresponds to dark-field illumination and can better display the edges of transparent objects in the sample, thus improving the observation effect. Furthermore, in Figure 2 As can be seen, there may be a portion of direct light 410 outside the scattered light 420. By designing the relevant parameters of the objective lens 300, this portion of direct light will not be collected by the objective lens 300 and will not have much impact on the observation. Therefore, it will not be described in detail.

[0101] In the light source device, by adjusting the relative dimensions of the first light-transmitting part 112b and the second light-transmitting part 112c, the ratio of bright-field illumination to dark-field illumination in the composite illumination can be changed to achieve the desired illumination effect. Specifically, as the relative light-transmitting size of the first light-transmitting part 112b increases, more illumination light incident on the sample at a smaller angle is transmitted, the proportion of bright-field illumination increases, and the brightness of the observed field of view will be brighter, but the imaging effect of transparent targets may deteriorate; while as the relative light-transmitting size of the second light-transmitting part 112c increases, more illumination light incident on the sample at a larger angle is transmitted, the proportion of dark-field illumination increases, and the imaging effect of transparent targets will be better, but the imaging effect of non-transparent objects or stained objects may deteriorate, and the overall brightness of the field of view will be lower.

[0102] Figure 3 The diagram shows some relevant structural parameters of the light source device, sample, and objective lens. By adjusting the relationship between them, a more ideal lighting effect can be obtained.

[0103] In some embodiments, in order to avoid the proportion of bright field illumination being too large in composite imaging, the following relationship usually needs to be satisfied: ∠β1≤1 / 3∠α, where tanβ=R1 / l, sinα=n, R1 is the distance between the outer edge of the first light-transmitting part 112b and the center of the aperture 112, l is the distance between the aperture 112 and the sample position, and n is the numerical aperture of the objective lens 300 used in conjunction with the light source device. It can be derived that: R1≤l·tg[arcsin(n) / 3].

[0104] In some embodiments, in order to prevent direct light from near the outside from entering the objective lens 300 and interfering with imaging, that is, to ensure that the scattered light generated by the interaction with the sample can cover the aperture of the objective lens 300, the following relationship usually needs to be satisfied: ∠β2>∠α, where tanβ=R2 / l, sinα=n, R2 is the distance between the inner edge of the second light-transmitting part 112c and the center of the aperture stop 112, l is the distance between the aperture stop 112 and the sample position, and n is the numerical aperture of the objective lens 300 used in conjunction with the light source device. It can be derived that: R2>l·tg[arcsin(n)].

[0105] In addition, such as Figure 2 and Figure 3 As shown, in some embodiments, the light source device may further include a light attenuator 114. The light attenuator 114 may be disposed in the outgoing light path of the illumination source and configured to reduce the brightness of the illumination light to improve the effect of composite illumination and avoid excessive brightness in the bright field. The light attenuator 114 may be a ground glass plate or a polarizer, etc., and may be disposed at one or more locations between the illumination source and the aperture 112 or between the aperture 112 and the first lens 113. To facilitate determining the size and position of the light attenuator 114 on the optical axis, the light attenuator 114 may be located in the optical path traversed by the outgoing light in the collimated state.

[0106] By incorporating an aperture in the light source device, a composite illumination combining dark-field and bright-field illumination can be created on the sample. This allows for clearer visualization of transparent target objects while maintaining sufficient imaging brightness. Furthermore, under composite illumination conditions, the size of the target object image can be similar to or even substantially identical to the actual target object size. This allows the target object size to be determined directly from the image size. Alternatively, the target object size can be obtained by multiplying the image size by a predetermined size conversion factor.

[0107] When the sample is illuminated using the light source device described above to obtain the image to be processed, a clear image can be produced, thus eliminating the need to stain the target object, thereby enabling online, in-situ optical measurement, simplifying the measurement process and improving the measurement results.

[0108] After acquiring the image to be processed, it can be directly binarized and the potential target objects and their parameters analyzed. Alternatively, preprocessing can be performed on the image before binarization to optimize the subsequent processing and analysis.

[0109] Specifically, in some embodiments, at least one of the contrast and brightness of the image to be processed can be adjusted to make the brightness distribution in the image to be processed more reasonable, so as to better distinguish the foreground area and the background area in subsequent steps.

[0110] In some embodiments, the image to be processed can be grayscaled to remove unnecessary color information, thereby simplifying the image data to be processed and also helping to better distinguish between foreground and background areas.

[0111] In some embodiments, the image to be processed can also be normalized so that the gray values ​​of the pixels in the image to be processed are distributed within a preset gray value range, thereby simplifying data processing.

[0112] In some embodiments, the image to be processed may also be denoised to minimize any defects that may exist before binarization, thereby helping to better distinguish between foreground and background regions.

[0113] return Figure 1 Image processing methods may also include:

[0114] Step S200: Binarize the image to be processed, and determine one or more foreground regions in the image to be processed based on the result of the binarization process.

[0115] The foreground region may contain the target object; however, it is understandable that in some cases, the foreground region may also contain other objects such as impurities. The background region typically includes the substrate, such as the solution or glass slide that supports the target object.

[0116] In some embodiments, binarization can be performed based on a preset grayscale threshold, specifically, such as Figure 4 As shown, step S200 may include:

[0117] Step S210: Compare the gray value of each pixel in the image to be processed with the preset gray value threshold.

[0118] Step S220: Based on the comparison result between the gray value of the pixel and the preset gray value threshold, determine whether the pixel belongs to the first pixel of the foreground region or the second pixel of the background region.

[0119] Step S230: Divide the continuously distributed first pixels into the same foreground region;

[0120] In this context, different foreground regions in the image to be processed can be separated by background regions. For example, in... Figure 5 In the image to be processed, foreground regions 510 and background regions 520 may have different brightness levels, and different foreground regions 510 may be separated by background regions 520. Figure 5 After binarization, the grayscale value of the foreground region 510 can be 1, while the grayscale value of the background region 520 can be 0.

[0121] Furthermore, when determining one or more foreground regions in an image based on the results of binarization, algorithms such as the U-Net algorithm and the watershed algorithm can be used to determine the foreground regions. For example, the U-Net algorithm can be used to determine the seed for the foreground regions, and then the watershed algorithm can be used to further determine the foreground regions.

[0122] In some embodiments, before comparing the grayscale value of each pixel in the image to be processed with a preset grayscale threshold, a preset grayscale threshold corresponding to the image to be processed can be determined based on the brightness distribution of the image to be processed. This is because the concentration of the target object may vary greatly in different samples, resulting in significant differences in the overall brightness of different images to be processed. For example, when the concentration of live cells in the sample is high, the overall brightness of the image to be processed is generally bright; while when the concentration of live cells in the sample is low, the overall brightness of the image to be processed may be dark. Considering this difference, if a fixed preset grayscale threshold is used for different images to be processed, it may be difficult to adapt to such variations in overall brightness, leading to inappropriate binarization, such as incorrectly identifying essentially the entire image to be processed as a foreground or background region.

[0123] Various methods can be used to determine the preset grayscale threshold corresponding to the image to be processed based on the brightness distribution of the image. In some embodiments, the preset grayscale threshold can be calculated based on a pre-fitted function according to the grayscale distribution in the image to be processed (e.g., characterized by parameters such as grayscale histogram, average brightness, or brightness variance). The pre-fitted function can be a linear function, a quadratic function, other polynomial functions, exponential functions, logarithmic functions, etc. Alternatively, in some embodiments, a model trained based on machine learning methods can be used to obtain the preset grayscale threshold to adapt to the overall brightness variations of different images to be processed and optimize binarization processing.

[0124] When the brightness of the target object is higher than that of other components in the sample (e.g., when the target object is a living cell), determining whether a pixel belongs to the first pixel of the foreground region or the second pixel of the background region based on the comparison between the pixel's grayscale value and a preset grayscale threshold can include:

[0125] When the gray value of a pixel is greater than a preset gray value threshold, the pixel is determined to be the first pixel belonging to the foreground region.

[0126] When the grayscale value of a pixel is less than or equal to a preset grayscale threshold, the pixel is determined to be the second pixel belonging to the background area.

[0127] Of course, in some other embodiments, when the brightness of the target object is lower than that of other components in the sample (e.g., when the target object is a dead cell that is darker than the background area), the opposite judgment criterion can also be used, namely:

[0128] When the gray value of a pixel is less than or equal to a preset gray value threshold, the pixel is determined to be the first pixel belonging to the foreground region.

[0129] When the grayscale value of a pixel is greater than a preset grayscale threshold, the pixel is determined to be the second pixel belonging to the background area.

[0130] In some embodiments, considering factors such as changes in local brightness of the target object during the acquisition, preprocessing, or binarization of the image to be processed, the obtained foreground region corresponding to the target object may have defects. Taking a transparent living cell as an example, if the sample substrate is uneven, the illumination changes, or there are defects in the preprocessing or binarization process, the living cell may not appear as a fully filled circle or ellipse, but rather as a "C" shape with a gap or an "O" shape with a dark center and bright edges, etc., which may adversely affect the subsequent determination of the target object parameters. Therefore, as Figure 6 As shown, to better optimize the determination of the foreground region, step S200 may further include:

[0131] Step S240: Correct defects in at least one of the foreground regions.

[0132] Specifically, gradient correction can be used to correct defects in the foreground region through methods such as expansion and erosion.

[0133] In some embodiments, when using gradient correction, the determination of whether a pixel should be included in the foreground region can be based on the grayscale value distribution of pixels within a certain range surrounding a given pixel. For example, correcting defects in at least one foreground region among one or more foreground regions may include: for at least one second pixel in the image to be processed, comparing the grayscale value of the second pixel with the average grayscale value of a predetermined number of pixels surrounding the second pixel; and determining whether to correct the second pixel to a first pixel based on the comparison result between the grayscale value of the second pixel and the average grayscale value. Specifically, determining whether to correct the second pixel to a first pixel based on the comparison result between the grayscale value of the second pixel and the average grayscale value may include: correcting the second pixel to a first pixel when the grayscale value of the second pixel is less than or equal to the average grayscale value. Alternatively, in some other embodiments, the opposite criterion may be used, i.e., correcting the second pixel to a first pixel when the grayscale value of the second pixel is greater than the average grayscale value. Gradient correction can effectively correct "O"-shaped defects. It should be noted that the grayscale value here can refer to the original grayscale value of the pixel, which may include 256 integer values ​​from 0 to 255, or it can refer to the grayscale value obtained after binarization, which may only include the two values ​​0 and 1.

[0134] In a specific example, when the grayscale value of a second pixel is 0, and the average grayscale value of the eight nearest neighbors surrounding that pixel is 1, the second pixel can be considered defective and should be corrected to the first pixel to be consistent with the surrounding pixels. When the grayscale value of a second pixel is 0, and the average grayscale value of the eight nearest neighbors surrounding that pixel is also 0, it can be considered that no correction is needed for the second pixel.

[0135] In some embodiments, the foreground region can also be modified based on the known shape of the target object. For example, when the target object is a living cell, the edge of the foreground region should be arc-shaped or elliptical. Therefore, modifying defects in at least one of the one or more foreground regions can include: determining whether the edge of the foreground region conforms to a preset shape edge; when the edge of the foreground region does not conform to the preset shape edge, dilating the foreground region to correct its edge; and eroding the dilated foreground region to make the area of ​​the modified foreground region consistent with the area of ​​the foreground region before modification. Specifically, dilation can expand the highlighted or white areas of the image, resulting in a larger highlighted area than the original image; while erosion can reduce and refine the highlighted or white areas of the image, resulting in a smaller highlighted area than the original image. After dilation and erosion, for example, gaps at the edges of a "C"-shaped foreground region can be filled, and its area can be consistent with the area before processing.

[0136] In some embodiments of this disclosure, a foreground region can be represented by the coordinates of pixels located on the edges of the foreground region. For example, multiple foreground regions can be stored in a list, where each entry represents a foreground region, and a single entry includes the coordinates of all pixels located on the edges of the corresponding foreground region. In subsequent processing, each foreground region can be analyzed item by item based on this list, including potential target objects and their parameters.

[0137] return Figure 1 Image processing methods may also include:

[0138] Step S300: For each of the one or more foreground regions, determine whether the foreground region is a target object image, and if the foreground region is a target object image, determine the target object parameters based on the foreground region.

[0139] It is important to note that when analyzing foreground regions, it is necessary not only to determine whether the foreground region corresponds to the target object, but also to consider the cases where the foreground region contains a single target object and the cases where it contains multiple target objects. For example, when live cells are the target object, if the concentration of live cells in the sample is low, each foreground region may correspond to only a single live cell, while if the concentration of live cells in the sample is high, at least some foreground regions may correspond to multiple live cells.

[0140] In some embodiments of this disclosure, such as Figure 7 As shown, step S300 may include:

[0141] Step S310: Determine whether the foreground region is a target object image based on the edge of the foreground region;

[0142] Step S320: When the foreground region is the target object image, determine whether the target objects contained in the foreground region are in a clustered state.

[0143] Step S331: When the target objects contained in the foreground region are in a clustered state, separate the multiple target objects in the clustered state according to the preset algorithm, and determine the target object parameters of each target object among the multiple target objects.

[0144] Step S332: When the target objects contained in the foreground region are not in a clustered state, determine the target object parameters of the target objects.

[0145] In some embodiments, it can be determined whether the foreground region is a target object image and whether it contains a single target object or a cluster of multiple target objects based on the shape and size of the foreground region.

[0146] For example, determining whether a foreground region is a target object image based on the edges of the foreground region can include:

[0147] Based on the edges of the foreground region, determine the envelope rectangle that can contain the foreground region and has the smallest size;

[0148] Compare the width of the envelope rectangle with a first preset threshold;

[0149] When the width of the envelope rectangle is greater than or equal to the first preset threshold, the foreground region is determined as the target object image;

[0150] When the width of the envelope rectangle is less than the first preset threshold, the foreground region is determined not to be the target object image.

[0151] It is important to note that in this paper, the shorter side of the enclosing rectangle is used as its width, and the longer side as its length. In a specific example, such as when the target object is a living cell, the first preset threshold can be set to be slightly less than or equal to the diameter of the living cell. Then, if the width of the enclosing rectangle is less than the diameter of the living cell, it can be determined that the foreground region cannot be the target object image; otherwise, it can be considered that the foreground region corresponds to the target object image.

[0152] Similarly, when the foreground region is an image of the target object, the size of the envelope rectangle can be used to determine whether the target objects contained in the foreground region are in a clustered state. That is, step S320 may include:

[0153] Compare the width of the envelope rectangle with the second preset threshold and the length of the envelope rectangle with the second preset threshold, respectively;

[0154] When the width and length of the envelope rectangle are both less than or equal to the second preset threshold, it is determined that the target objects contained in the foreground region are not in a clustered state.

[0155] When at least one of the width and length of the envelope rectangle is greater than the second preset threshold, it is determined that the target objects contained in the foreground region are in a clustered state.

[0156] The second preset threshold is greater than or equal to the first preset threshold. For example, the second preset threshold may be greater than or equal to the diameter of a single living cell.

[0157] In a specific example, if at least one of the width and length of the envelope rectangle is greater than the diameter of a single live cell, it can be determined that there are two or more live cells in the foreground region; while if the width and length of the envelope rectangle are both slightly less than or equal to the diameter of a single live cell, it can be considered that the foreground region contains only a single live cell.

[0158] In some embodiments, when the target objects contained in the foreground region are in a clustered state, multiple target objects in the clustered state can be separated according to a preset algorithm, thereby determining the target object parameters of each target object separately. These algorithms can be, for example, neural network and machine learning algorithms such as the Hough circle recognition algorithm, watershed algorithm, hot spot detection algorithm, support vector machine (SVM) algorithm, and u-net algorithm.

[0159] When analyzing an image of a target object, the target object parameters may include at least one of the following: target object count, target object size, and target object location.

[0160] Specifically, in the process of determining the target object count, the number of target objects contained in each foreground region of the image to be processed can be accumulated to obtain the total number of target objects in the image to be processed.

[0161] When determining the size and / or location of a target object, the Hough circle recognition algorithm can be used to identify the circle corresponding to each target object, and the diameter of this circle can be used to determine the size of the target object, and the center of this circle can be used to determine the location of the target object.

[0162] It should be noted that in some cases, even if only a portion of the target object is in the image to be processed (e.g., only half of the target object is in the edge region of the image to be processed), the size and position of the target object can be determined based on the foreground region using the Hough circle recognition algorithm.

[0163] Furthermore, when determining the location of a target object based on the center of a Hough circle, it is important to distinguish between the center of the target object itself and the center of a cluster of multiple target objects. For example, the aforementioned error can be avoided by imposing a constraint on the center of the Hough circle, requiring it to fall within the initially defined foreground area.

[0164] Alternatively, when determining the size of a target object, the maximum distance between the two darkest points of the foreground region can be found in multiple directions to obtain the diameter of the foreground region in each direction. The average value of the diameters in multiple directions can then be calculated to obtain the size of the target object.

[0165] In some embodiments of this disclosure, the image processing method may further include:

[0166] After determining the target object parameters for all foreground regions in the image to be processed, the target object parameters are displayed in the form of at least one of a labeled map and a labeled list.

[0167] Specifically, the location and size of the identified target object can be displayed in the annotation diagram to visually represent the analysis results of the image to be processed. Alternatively, the identified target object parameters can be displayed in a list, and further processing such as summarizing the identified target object parameters can be performed in the list to facilitate sample analysis for users.

[0168] This disclosure also proposes an image processing device, such as Figure 8 As shown, the image processing device 800 may include a processor 810 and a memory 820. The memory 820 stores instructions, and when the instructions are executed by the processor 810, the steps in the image processing method described above can be implemented.

[0169] The processor 810 can perform various actions and processes according to instructions stored in the memory 820. Specifically, the processor 810 can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor, and can be an x86 architecture or an ARM architecture, etc.

[0170] Memory 820 stores executable instructions that are executed by processor 810 using the image processing method described above. Memory 820 may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory used in the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0171] This disclosure also proposes an optical detection device, which may include a light source device, a sample stage, an imaging device, and an image processing device as described above. This optical detection device can be used for online, in-situ detection of biological particles, such as cells, without requiring staining or other treatments on transparent target objects.

[0172] This disclosure also proposes a computer-readable storage medium storing instructions that, when executed, implement the steps of the image processing method described above. Similarly, the computer-readable storage medium in the embodiments of this disclosure may be volatile memory or non-volatile memory, or may include both. It should be noted that the computer-readable storage medium described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0173] This disclosure also proposes a computer program product that may include instructions that, when executed by a processor, can implement the steps of the image processing method described above.

[0174] Instructions can be any set of instructions that will be executed directly by one or more processors, such as machine code, or any set of instructions that will be executed indirectly, such as a script. The terms “instruction,” “application,” “process,” “step,” and “program” used herein are interchangeable. Instructions can be stored in object code format for direct processing by one or more processors, or stored in any other computer language, including scripts or sets of independent source code modules that are interpreted on demand or compiled ahead of time. Instructions can include instructions that cause one or more processors to act as the various neural networks described herein. The function, methods, and routines of instructions are explained in more detail in other parts of this document.

[0175] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0176] The terms “front,” “back,” “top,” “bottom,” “above,” “below,” etc., used in the specification and claims, if present, are for descriptive purposes and are not necessarily used to describe unchanging relative positions. It should be understood that such terms are interchangeable where appropriate, so that embodiments of this disclosure described herein can, for example, operate on orientations different from those shown or otherwise described herein.

[0177] As used herein, the term "exemplary" means "serving as an example, instance, or illustration," and not as a "model" to be precisely copied. Any implementation described herein by example is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, this disclosure is not limited to any theory expressed or implied as given in the foregoing technical field, background, summary of invention, or detailed description.

[0178] As used herein, the term "substantially" means any minor variation resulting from design or manufacturing defects, device or component tolerances, environmental influences, and / or other factors. The term "substantially" also allows for differences from the perfect or ideal situation due to parasitic effects, noise, and other practical considerations that may exist in the actual implementation.

[0179] The above description may refer to elements, nodes, or features that are “connected” or “coupled” together. As used herein, unless otherwise expressly stated, “connected” means that one element / node / feature is directly connected (or directly communicates) with another element / node / feature electrically, mechanically, logically, or otherwise. Similarly, unless otherwise expressly stated, “coupled” means that one element / node / feature can be directly or indirectly connected to another element / node / feature mechanically, electrically, logically, or otherwise to allow interaction, even if the two features may not be directly connected. That is, “coupled” is intended to include both direct and indirect connections of elements or other features, including connections using one or more intermediate elements.

[0180] It should also be understood that when the term “including / contains” is used herein, it indicates the presence of the indicated feature, whole, step, operation, unit and / or component, but does not preclude the presence or addition of one or more other features, wholes, steps, operations, units and / or components and / or combinations thereof.

[0181] Those skilled in the art will recognize that the boundaries between the above operations are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed among additional operations, and operations may be performed with at least partial overlap in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments. However, other modifications, variations, and substitutions are equally possible. Therefore, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

[0182] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. The various embodiments disclosed herein can be combined in any way without departing from the spirit and scope of this disclosure. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. An image processing method, characterized by, The image processing method includes: Acquire an image to be processed, wherein the image to be processed includes a foreground region and a background region with different brightness, and the image to be processed is obtained by photographing a sample that can contain a target object, the target object being a transparent object that has not been stained; The image to be processed is binarized, and one or more foreground regions in the image to be processed are determined based on the result of the binarization process. For each of the one or more foreground regions, determine whether the foreground region is a target object image based on the edge of the foreground region; When the foreground region is a target object image, it is determined whether the target objects contained in the foreground region are in a clustered state; When the target objects contained in the foreground region are in a clustered state, the multiple target objects in the clustered state are separated according to a preset algorithm, and the target object parameters of each of the multiple target objects are determined. When the target objects contained in the foreground region are not in a clustered state, the target object parameters of the target objects are determined.

2. The image processing method of claim 1, wherein, The image to be processed is obtained by photographing a sample containing the target object under a combination of bright field and dark field illumination.

3. The image processing method according to claim 2, characterized in that, The target object in the sample has a different refractive index than the rest of the sample.

4. The image processing method according to claim 1, characterized in that, Binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization process includes: The grayscale value of each pixel in the image to be processed is compared with a preset grayscale threshold. Based on the comparison result between the gray value of the pixel and the preset gray value threshold, it is determined whether the pixel belongs to the first pixel of the foreground region or the second pixel of the background region. Divide the continuously distributed first pixels into the same foreground region; In this process, different foreground regions in the image to be processed are separated by background regions.

5. The image processing method according to claim 4, characterized in that, Binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization process further includes: Based on the brightness distribution of the image to be processed, a preset grayscale threshold corresponding to the image to be processed is determined.

6. The image processing method according to claim 4, characterized in that, Determining whether a pixel belongs to the first pixel of the foreground region or the second pixel of the background region based on the comparison result between the pixel's grayscale value and the preset grayscale threshold includes: When the grayscale value of the pixel is greater than the preset grayscale threshold, the pixel is determined to be the first pixel belonging to the foreground region; When the grayscale value of the pixel is less than or equal to the preset grayscale threshold, the pixel is determined to be the second pixel belonging to the background region.

7. The image processing method according to claim 4, characterized in that, Binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization process further includes: Correct the defect in at least one of the one or more foreground regions.

8. The image processing method according to claim 7, characterized in that, Correcting defects in at least one of the one or more foreground regions includes: For at least one second pixel in the image to be processed, the gray value of the second pixel is compared with the average gray value of a preset number of pixels surrounding the second pixel; Based on the comparison result between the gray value of the second pixel and the average gray value, it is determined whether to correct the second pixel to the first pixel.

9. The image processing method according to claim 8, characterized in that, Determining whether to correct the second pixel to the first pixel based on the comparison result between the grayscale value of the second pixel and the average grayscale value includes: When the grayscale value of the second pixel is less than or equal to the average grayscale value, the second pixel is corrected to the first pixel.

10. The image processing method according to claim 7, characterized in that, Correcting defects in at least one of the one or more foreground regions includes: For at least one of the one or more foreground regions, determine whether the edge of the foreground region matches the edge of a preset shape; When the edge of the foreground region does not conform to the edge of a preset shape, the foreground region is expanded to correct its edge; and The expanded foreground region is eroded to make the area of ​​the corrected foreground region the same as the area of ​​the original foreground region.

11. The image processing method according to claim 10, characterized in that, The preset shape edge includes at least one of a circular arc edge and an elliptical arc edge.

12. The image processing method according to claim 1, characterized in that, The foreground region is represented by the coordinates of the pixels located on the edge of the foreground region.

13. The image processing method according to claim 1, characterized in that, Determining whether the foreground region is a target object image based on the edges of the foreground region includes: Based on the edges of the foreground region, determine an envelope rectangle that can contain the foreground region and has the smallest size; Compare the width of the envelope rectangle with a first preset threshold; When the width of the envelope rectangle is greater than or equal to the first preset threshold, the foreground region is determined to be the target object image; When the width of the envelope rectangle is less than the first preset threshold, it is determined that the foreground region is not the target object image.

14. The image processing method according to claim 13, characterized in that, When the foreground region is a target object image, determining whether the target objects contained in the foreground region are in a clustered state includes: The width of the envelope rectangle is compared with a second preset threshold, and the length of the envelope rectangle is compared with the second preset threshold. When the width and length of the envelope rectangle are both less than or equal to the second preset threshold, it is determined that the target objects contained in the foreground region are not in a clustered state. When at least one of the width and length of the envelope rectangle is greater than the second preset threshold, it is determined that the target objects contained in the foreground region are in a clustered state. Wherein, the second preset threshold is greater than or equal to the first preset threshold.

15. The image processing method according to claim 1, characterized in that, When the target objects contained in the foreground region are in a clustered state, separating the multiple target objects in the clustered state according to a preset algorithm includes: Separate multiple target objects in a clustered state using at least one of the following algorithms: Hough circle recognition algorithm, watershed algorithm, hotspot detection algorithm, support vector machine algorithm, and u-net algorithm.

16. The image processing method according to claim 1, characterized in that, The target object parameters include at least one of the following: target object count, target object size, and target object position.

17. The image processing method according to claim 1, characterized in that, Before binarizing the image to be processed and determining one or more foreground regions in the image to be processed based on the result of the binarization, the image processing method further includes performing at least one of the following preprocessing steps on the image to be processed: Adjust at least one of the contrast and brightness of the image to be processed; The image to be processed is converted to grayscale. The image to be processed is normalized. as well as The image to be processed is then subjected to noise reduction.

18. The image processing method according to claim 1, characterized in that, The image processing method further includes: After determining the target object parameters for all foreground regions in the image to be processed, the target object parameters are displayed in the form of at least one of a labeled map and a labeled list.

19. The image processing method according to claim 1, characterized in that, The target object includes cells.

20. An image processing device, characterized in that, The image processing device includes a processor and a memory, the memory storing instructions that, when executed by the processor, implement the steps of the image processing method as described in any one of claims 1 to 19.

21. An optical detection device, characterized in that, The optical detection device includes: Light source device, the light source device comprising: An illumination source, the illumination source being configured to generate illumination light; and An aperture stop, wherein the aperture stop is disposed in the outgoing light path of the illumination source, the aperture stop comprising: A light shield, configured to block part of the illumination light; A first light-transmitting portion is formed on the light-shielding screen and covers the center of the aperture. The first light-transmitting portion is configured to allow partial illumination light to pass through to form bright-field illumination of the sample; and The second light-transmitting part is formed on the light-shielding screen and is located around the first light-transmitting part. The second light-transmitting part is configured to allow some illumination light to pass through in order to form dark field illumination of the sample. A sample stage, configured to hold the sample; An imaging device configured to image the sample under the illumination of the light source device to produce an image to be processed, and the imaging device includes an objective lens; and The image processing apparatus according to claim 20.

22. The optical detection device according to claim 21, characterized in that, The following relationship exists between the distance R1 between the outer edge of the first light-transmitting part and the center of the aperture, the distance l between the aperture and the sample stage, and the numerical aperture n of the objective lens: R1≤l·tg[arcsin(n) / 3].

23. The optical detection device according to claim 21, characterized in that, The following relationship exists between the distance R2 between the inner edge of the second light-transmitting part and the center of the aperture, the distance l between the aperture and the sample stage, and the numerical aperture n of the objective lens: R2>l·tg[arcsin(n)].

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, implement the steps of the image processing method as described in any one of claims 1 to 19.

25. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, implement the steps of the image processing method as described in any one of claims 1 to 19.

Images