Electron microscope image quality assessment method, device, electronic device and storage medium

Abstract

The present invention provides an electron microscope image quality assessment method that can be applied to the field of biological image processing technology. The method includes: performing regional differentiation processing on a target electron microscope image to obtain a foreground region and a background region of the target electron microscope image, and generating masks for the foreground region and the background region of the target electron microscope image, respectively; calculating the average pixel value of the foreground region using the mask of the foreground region, and calculating the average pixel value of the background region using the mask of the background region; calculating the standard deviation of the foreground region using the average pixel value of the foreground region, and calculating the standard deviation of the background region using the average pixel value of the background region; and calculating the noise contrast used to assess the quality of the target electron microscope image using the average pixel value and standard deviation of the foreground region and the average pixel value and standard deviation of the background region. The present invention also provides an electron microscope image quality assessment device, equipment, and storage medium.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, specifically to the field of biological image processing technology, and more specifically to an electron microscope image quality assessment method, device, electronic device and storage medium. Background Art

[0002] After biological samples are prepared and sliced, electron microscopy can be used to obtain biological electron microscopy images with nanometer-level resolution. These images can be used to distinguish membrane structures such as cell membranes and organelle membranes in biological tissues, thereby discerning the microscopic structure of cells. However, the quality of biological electron microscopy images is affected by multiple factors, including sample preparation technology, electron beam dwell time, and electron beam voltage, necessitating a quantitative image quality assessment method. If the conditions under which optimal imaging quality is achieved can be assessed, shorter electron beam dwell times (i.e., faster imaging speeds) and larger imaging areas can be achieved within the same timeframe. This can shorten imaging time and reduce costs for large-scale biological electron microscopy imaging, which can often take several months.

[0003] Existing electron microscopy image quality assessment methods based on contrast cannot accurately evaluate image quality when image noise is significant. Electron microscopy image quality assessment methods that use the contrast-to-noise ratio (CNR) estimate the relative noise level, which cannot be directly correlated with foreground and background resolution. They also ignore the difference in standard deviation between foreground and background regions in electron microscopy images, leading to inaccurate noise assessments. Other existing electron microscopy image quality assessment methods do not distinguish between foreground and background, providing general estimates of image quality and failing to accurately describe foreground and background resolution. Summary of the Invention

[0004] In view of the above problems, the present invention provides an electron microscope image quality assessment method, device, electronic device and storage medium.

[0005] According to a first aspect of the present invention, a method for evaluating electron microscope image quality is provided, comprising:

[0006] Performing regional differentiation processing on the target electron microscope image to obtain a foreground region and a background region of the target electron microscope image, and generating a mask of the foreground region and a mask of the background region respectively;

[0007] Calculate the average pixel value of the foreground area using the mask of the foreground area, and calculate the average pixel value of the background area using the mask of the background area;

[0008] The standard deviation of the foreground area is calculated using the average pixel value of the foreground area, and the standard deviation of the background area is calculated using the average pixel value of the background area;

[0009] The noise contrast used to evaluate the quality of the target electron microscope image is calculated using the average pixel value and standard deviation of the foreground area and the average pixel value and standard deviation of the background area.

[0010] According to an embodiment of the present invention, the above-mentioned region differentiation processing of the target electron microscope image to obtain the foreground region and the background region of the target electron microscope image includes:

[0011] Based on user needs, the pixel points where the target object in the target electron microscope image is located are used as the foreground area, and the pixel points where the non-target object is located are used as the background area, wherein the target object includes the membrane structure of the biological cell;

[0012] The target electron microscope image is subjected to image segmentation based on a preset image morphological processing method or a machine learning method to obtain a target electron microscope image that distinguishes the foreground area and the background area, wherein the machine learning method includes a semantic segmentation neural network and a random forest classifier.

[0013] According to an embodiment of the present invention, the above-mentioned region differentiation processing of the target electron microscope image to obtain the foreground region and the background region of the target electron microscope image further includes:

[0014] When the pixel value of the foreground area is smaller than the pixel value of the background area, the maximum pixel value of the target electron microscope image is obtained, and the pixel value of the target electron microscope image is inverted using the maximum pixel value, thereby completing the preprocessing of the target electron microscope image.

[0015] According to an embodiment of the present invention, the steps of generating a mask for the foreground area and a mask for the background area include:

[0016] Calculate the pixel median value of the target electron microscope image, and use the pixel median value to obtain the initial mask of the foreground area and the initial mask of the background area through threshold binarization operation;

[0017] Performing a closing operation on the initial mask of the foreground area to remove noise points in the initial mask of the foreground area; and removing connected areas in the initial mask of the foreground area whose connected areas are smaller than a preset value to obtain an intermediate mask of the foreground area;

[0018] Deleting a region whose thickness is greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground region to obtain a mask of the foreground region;

[0019] The initial mask of the background area is subjected to an erosion operation and a dilation operation to complete an opening operation of the mask of the background area, thereby obtaining a mask of the background area after the holes are filled.

[0020] According to an embodiment of the present invention, the preset value is set based on the attribute value of the target electron microscope image.

[0021] According to an embodiment of the present invention, the above-mentioned deleting the area having a thickness greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground area to obtain the mask of the foreground area includes:

[0022] Performing an erosion operation and a dilation operation on the intermediate mask of the foreground area to complete an opening operation on the intermediate mask of the foreground area, thereby obtaining a thin film of the foreground area;

[0023] When the value of the area of ​​the intermediate mask in the foreground area is 1 and the value of the area corresponding to the thin film in the foreground area is 0, the value of the corresponding area of ​​the intermediate mask in the foreground area is set to 1;

[0024] When the value of the area of ​​the intermediate mask in the foreground area is 0 and the value of the area corresponding to the thin film in the foreground area is 1, the value of the corresponding area of ​​the intermediate mask in the foreground area is set to 0.

[0025] According to an embodiment of the present invention, calculating the average pixel value of the foreground area using the mask of the foreground area includes:

[0026] Calculating the pixel value of the target electron microscope image with the pixel value of the area corresponding to the mask of the foreground area, and calculating the calculation result with the sum of the pixel values ​​of the mask of the foreground area to obtain the average pixel value of the foreground area; or

[0027] The sum of the pixel values ​​of the target electron microscope image and the total number of pixels of the target electron microscope image are calculated, and the calculation result is used as the average pixel value of the foreground area.

[0028] According to an embodiment of the present invention, calculating the standard deviation of the foreground area by using the average pixel value of the foreground area includes:

[0029] Calculate the standard deviation of the foreground area using the average pixel value of the foreground area, the pixel value of the mask of the foreground area, the sum of the pixel values ​​of the mask of the foreground area, and the pixel value of the target electron microscope image; or

[0030] The standard deviation of the foreground area is calculated using the pixel value of the target electron microscope image, the total number of pixels of the target electron microscope image, and the average pixel value of the target electron microscope image.

[0031] A second aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0032] The third aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0033] The electron microscope image quality assessment method provided by the present invention distinguishes between the foreground and background regions of the electron microscope image and provides a method for generating foreground and background region masks. This eliminates interference factors that affect the electron microscope image quality assessment by eliminating noise points and uninteresting, too small or too large connected regions in the foreground and background regions. Thus, the method provided by the present invention can accurately distinguish between the foreground and background, thereby enabling accurate assessment of foreground and background resolution. Furthermore, the noise-containing contrast provided by the present invention not only allows for intuitive assessment of the difference between the foreground and background, but also takes noise into account in the assessment, thereby accounting for the impact of different foreground and background noise levels on foreground and background resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0035] Figure 1 A diagram schematically illustrates an application scenario of the electron microscope image quality assessment method according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of pixel value intensity curves of foreground and background according to an embodiment of the present invention;

[0037] Figure 3 is a flow chart of a method for evaluating electron microscope image quality according to an embodiment of the present invention;

[0038] Figure 4 1 is a schematic diagram of a process decomposition of an electron microscope image quality assessment method based on foreground and background area distinction according to an embodiment of the present invention;

[0039] Figure 5 The following schematically shows a structural block diagram of an electron microscope image quality assessment device according to an embodiment of the present invention;

[0040] Figure 6 The block diagram of an electronic device suitable for implementing the electron microscope image quality assessment method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0042] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0043] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0044] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0045] Figure 1 The following schematically illustrates an application scenario of the electron microscope image quality assessment method according to an embodiment of the present invention.

[0046] like Figure 1 As shown, the application scenario 100 according to this embodiment may include the field of biological electron microscopy image processing technology. A network 104 is used as a medium for providing a communication link between a first terminal device 101, a second terminal device 102, a third terminal device 103, and a server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0047] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0048] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0049] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0050] It should be noted that the electron microscope image quality assessment method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the electron microscope image quality assessment device provided in the embodiment of the present invention can generally be set in the server 105. The electron microscope image quality assessment method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the electron microscope image quality assessment device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0051] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0052] The following will be based on Figure 1 The scene described by Figures 2 to 4 The electron microscope image quality evaluation method of the disclosed embodiment is described in detail.

[0053] Shortening imaging time and reducing imaging costs has long been a research hotspot in the field of biological electron microscopy image processing technology. Electron microscopy image quality assessment methods can assess the conditions under which optimal imaging quality is achieved, thereby enabling shorter electron beam dwell times (i.e., faster imaging speeds) and larger imaging areas within the same timeframe. This is crucial for shortening imaging time and reducing imaging costs during biological electron microscopy imaging.

[0054] In the field of biological electron microscopy image processing technology, the ability to resolve membrane structures is the core criterion for judging the quality of biological electron microscopy images. Generally, the membrane structure and other structures that are expected to be resolved are called the foreground, and the substrate or dark area opposite to it is called the background.

[0055] Contrast is a common evaluation method in the field of electron microscopy to distinguish the relative difference between the foreground and background. Its expression is shown in formula (1):

[0056] (1),

[0057] Among them, I target and I background The contrast ratios represent the average pixel values ​​(average intensity) of the foreground and background areas, respectively. Contrast is 1 when the background is zero, and 0 when the foreground and background are identical. Larger values ​​indicate a greater difference between the foreground and background. While contrast does reflect the relative difference between the foreground and background to some extent, it doesn't account for the effects of noise, making it difficult to accurately assess image quality when significant noise is present.

[0058] Figure 2 : is a schematic diagram of the pixel value intensity curve of the foreground and background according to an embodiment of the present invention. Figure 2 The left figure is a schematic diagram of the pixel value intensity curve when the background and foreground have a uniform standard deviation in the contrast-to-noise ratio. Figure 2 The right figure is a schematic diagram of the pixel value intensity curve under normal circumstances. Generally, the foreground has a larger pixel value intensity fluctuation.

[0059] like Figure 2 As shown in Figure 2, the contrast to noise ratio (CNR) is a common evaluation index used to quantify the difference between the signal and background in an image relative to the noise level. Its expression is shown in formula (2):

[0060] (2),

[0061] Among them, I target and I background are the average pixel values ​​(average intensity) of the foreground and background regions, respectively, and the standard deviation is used Estimate the overall noise of the image. The contrast-to-noise ratio can be used to evaluate image quality, but the contrast-to-noise ratio estimation ignores the difference in standard deviation between the foreground and background. Usually, the foreground area has greater numerical fluctuations than the background, and the foreground pixel values ​​have a larger standard deviation. Therefore, the contrast-to-noise ratio cannot accurately describe the image quality. Moreover, under the condition of the same noise, the contrast-to-noise ratio is target and I background When 1, 0 and 11, 10 are respectively selected, they have the same value. However, the foreground and background of 1 and 0 have higher contrast, which means better foreground and background discrimination. Therefore, the background-to-noise ratio cannot be directly correlated with the foreground and background discrimination.

[0062] Furthermore, existing general image quality assessment methods do not distinguish between foreground and background, and instead estimate image quality through statistical methods such as image variance and the sum of squared image gradient amplitudes. However, for biological electron microscopy images or other electron microscopy images, there are other interfering factors besides the foreground and background, such as bright areas of no interest caused by aggregated stained proteins or dust contamination on the sample. Evaluating only the overall image quality takes all pixels into account and fails to accurately assess the image's ability to distinguish between foreground and background. Accurately distinguishing between foreground and background to eliminate interfering factors is essential. In other words, general image quality assessment methods do not distinguish between foreground and background, and are general estimates of image quality that fail to accurately describe the ability to resolve foreground and background in an image.

[0063] Therefore, existing methods cannot fully meet the need for distinguishing foreground from background in the field of biological electron microscopy. In order to solve the technical problems existing in the prior art, the present invention provides an electron microscope image quality assessment method based on the ability to distinguish foreground from background.

[0064] Figure 3 4 is a flow chart of a method for evaluating electron microscope image quality according to an embodiment of the present invention.

[0065] like Figure 3 As shown, the electron microscope image quality assessment method includes operations S310 to S340.

[0066] In operation S310 , a region separation process is performed on the target electron microscope image to obtain a foreground region and a background region of the target electron microscope image, and a mask of the foreground region and a mask of the background region are generated respectively.

[0067] For the target electron microscope image, it should be ensured that the foreground area is brighter than the background area, that is, the pixel value of the foreground area is greater than the pixel value of the background area.

[0068] In operation S320, an average pixel value of the foreground area is calculated using the mask of the foreground area, and an average pixel value of the background area is calculated using the mask of the background area.

[0069] In operation S330, a standard deviation of the foreground area is calculated using the average pixel value of the foreground area, and a standard deviation of the background area is calculated using the average pixel value of the background area.

[0070] In operation S340, a noise contrast for evaluating the quality of the target electron microscope image is calculated using the average pixel value and standard deviation of the foreground region and the average pixel value and standard deviation of the background region.

[0071] The electron microscope image quality assessment method provided by the present invention distinguishes between the foreground and background regions of the electron microscope image and provides a method for generating foreground and background region masks. This eliminates interference factors that affect the electron microscope image quality assessment by eliminating noise points and uninteresting, too small or too large connected regions in the foreground and background regions. Thus, the method provided by the present invention can accurately distinguish between the foreground and background, thereby enabling accurate assessment of foreground and background resolution. Furthermore, the noise-containing contrast provided by the present invention not only allows for intuitive assessment of the difference between the foreground and background, but also takes noise into account in the assessment, thereby accounting for the impact of different foreground and background noise levels on foreground and background resolution.

[0072] According to an embodiment of the present invention, the above-mentioned region differentiation processing of the target electron microscope image to obtain the foreground area and background area of ​​the target electron microscope image includes: based on user needs, taking the pixel points where the target object in the target electron microscope image is located as the foreground area, and taking the pixel points where the non-target object is located as the background area, wherein the target object includes the membrane structure of a biological cell; performing image segmentation based on a preset image morphological processing method or based on a machine learning method on the target electron microscope image to obtain the target electron microscope image with the foreground area and background area distinguished, wherein the machine learning method includes a semantic segmentation neural network and a random forest classifier.

[0073] The above embodiments are illustrative. Those skilled in the art can set the foreground and background areas according to actual needs. For example, an expert marking method can be used to finely mark the foreground and background areas; or the graphics processing solution described in the present invention can be used to generate training data, and then a machine learning method (such as a U-net convolutional neural network model, a random forest classifier, etc.) can be used to perform image segmentation training to achieve foreground and background segmentation and obtain a mask.

[0074] According to an embodiment of the present invention, the above-mentioned region differentiation processing of the target electron microscope image to obtain the foreground region and the background region of the target electron microscope image further includes: when the pixel value of the foreground region is smaller than the pixel value of the background region, obtaining the maximum pixel value of the target electron microscope image, and using the maximum pixel value to perform an inverting operation on the pixel value of the target electron microscope image, thereby completing the preprocessing of the target electron microscope image.

[0075] For electron microscope images , ensure that the foreground is brighter than the background (if the brightness (i.e. pixel value) of the foreground is smaller than that of the background, for example, in the transmission electron microscope image of a biological sample, the membrane is darker than the cytoplasm, then you need to first negate it, i.e. , and use the preprocessed replace ).

[0076] According to an embodiment of the present invention, the above-mentioned generation of masks for the foreground area and the background area includes: calculating the pixel median value of the target electron microscope image, and using the pixel median value to perform a threshold binarization operation to obtain an initial mask for the foreground area and an initial mask for the background area respectively; performing a closing operation on the initial mask of the foreground area to delete noise points in the initial mask of the foreground area; and deleting connected areas in the initial mask of the foreground area whose connected areas are smaller than a preset value to obtain an intermediate mask of the foreground area; deleting areas with a thickness greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground area to obtain a mask of the foreground area; and performing an erosion operation and a dilation operation on the initial mask of the background area to complete an opening operation on the mask of the background area to obtain a mask of the background area with the holes filled.

[0077] According to an embodiment of the present invention, the preset value is set based on the attribute value of the target electron microscope image.

[0078] According to an embodiment of the present invention, the above-mentioned deleting the area whose thickness is greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground area to obtain the mask of the foreground area includes: performing an etching operation and a dilation operation on the intermediate mask of the foreground area to thereby complete an opening operation on the intermediate mask of the foreground area to obtain a thin film of the foreground area; when the area value of the intermediate mask of the foreground area is 1 and the area corresponding to the thin film of the foreground area is 0, setting the value of the corresponding area of ​​the intermediate mask of the foreground area to 1; when the area value of the intermediate mask of the foreground area is 0 and the area corresponding to the thin film of the foreground area is 1, setting the value of the corresponding area of ​​the intermediate mask of the foreground area to 0.

[0079] The following is a detailed description of the embodiments and the attached Figure 4 The process of generating the foreground and background area masks provided by the present invention is further described in detail.

[0080] Figure 4 3 is a schematic diagram of the process decomposition of an electron microscope image quality assessment method based on the distinction between foreground and background areas according to an embodiment of the present invention.

[0081] like Figure 4 As shown in FIG, the above-mentioned electron microscope image quality assessment method mainly includes generating foreground and background area masks, calculating the average pixel value of the foreground and background areas, calculating the standard deviation of the foreground and background areas, and evaluating the electron microscope image quality, among which, generating foreground and background area masks includes several sub-steps such as (a) to (f).

[0082] In the process of generating foreground and background area masks:

[0083] (a) For the selected image Find the median and get .

[0084] (b) Using threshold processing to get the foreground and background Mask, as shown in formula (3):

[0085] (3),

[0086] Among them, c0 and c1 are parameters, c0 should be greater than c1, and both are positive. Display the foreground and background masks, and adjust the parameters c0 and c1 starting from 1 to Preserve the foreground, such as membrane structures, in The background is included in c0, such as the cytoplasm. For example, for a biological electron microscope image with a pixel pitch of 8 nanometers, c0 = 1.2 and c1 = 1 can be used.

[0087] Optionally, the threshold processing implemented in steps (a) and (b) can be performed by other methods such as manually selecting a fixed threshold, calculating the image median or mean, Otsu's method for automatic threshold selection, and adaptive thresholding to obtain a threshold, and then used to obtain a mask in step (b).

[0088] (c) Use closing operation to delete The noise in The closing operation can be expressed by formula (4):

[0089] (4),

[0090] Where B0 is the structural element (a matrix with a central circle or square area of ​​1 and other areas of 0, and a radius of c3. For example, B can be a 3×3 matrix with all values ​​1), and ⊕ represents the dilation operation. Represents an erosion operation. (Dilation and erosion are two basic morphological operations in image processing, which respectively expand and shrink the foreground area of ​​an image based on the structuring element.)

[0091] (d) Delete The connected area with less than c4 points in the (The present invention adopts the definition of 4-neighbor connectivity.) For example, for a biological electron microscope image with a pixel pitch of 8 nanometers, c4 can be set to 200 to exclude connected areas of non-membrane structures (such as small bright clusters formed by certain protein aggregates after staining).

[0092] (e) Delete The area with a thickness greater than the theoretical thickness of the target resolution object. In step (e), first, Generate using opening operation , as shown in formula (5):

[0093] (5),

[0094] Where B1 is the structural element (a matrix with the central circular area taking 1 and the other elements taking 0, and a radius of c5. For example, for a biological electron microscope image with an 8-nanometer resolution, c5 can be 12, indicating that the membrane structure thickness is less than 12 × 2 pixels), and ⊕ represents the expansion operation. Represents an erosion operation.

[0095] Secondly, and The area corresponding to the value 1 is set to 0. As shown in formula (6):

[0096] (6).

[0097] (f) Open the background mask to fill the small holes in it to avoid individual noise points in the background being too bright and not being included in the background during threshold processing, as shown in formula (7):

[0098] (7),

[0099] Where B2 is a structural element (a matrix with a central circle or square area of ​​1 and other areas of 0, and a radius of c6. For example, B2 can be a 3×3 matrix with all values ​​1), and ⊕ represents the dilation operation. Represents an erosion operation.

[0100] Alternatively, in the field of biological electron microscopy, the method of the present invention can be used to distinguish membrane structures with little thickness variation. When imaging at different resolutions or downsampling the original electron microscope image, the parameters in the method can be adjusted to adapt to the resolution change. For example, if the parameters are feasible at a certain resolution, when the resolution is changed to r times (r can be a positive real number, such as 1 / 2 or 4), the area-related parameter c4 is multiplied by r 2 , the thickness related parameter c5 is multiplied by r, and other parameters can be adapted by fine-tuning.

[0101] Furthermore, when distinguishing between foreground and background, it is possible to exclude some correct foreground and background areas from the mask and still perform subsequent calculations. Simply using a larger electron microscope image (e.g., larger than 2000×2000 pixels) or multiple electron microscope images can ensure statistical stability of the evaluation results.

[0102] According to an embodiment of the present invention, the above-mentioned calculation of the average pixel value of the foreground area using the mask of the foreground area includes: performing an operation on the pixel values ​​of the target electron microscope image and the pixel values ​​of the area corresponding to the mask of the foreground area, and performing an operation on the operation result and the sum of the pixel values ​​of the mask of the foreground area to obtain the average pixel value of the foreground area; or performing an operation on the sum of the pixel values ​​of the target electron microscope image and the total number of pixels of the target electron microscope image, and using the operation result as the average pixel value of the foreground area.

[0103] In general, the average pixel value of the foreground and background areas is calculated and , as shown in formula (8):

[0104] (8).

[0105] According to an embodiment of the present invention, the above-mentioned calculation of the standard deviation of the foreground area using the average pixel value of the foreground area includes: calculating the standard deviation of the foreground area using the average pixel value of the foreground area, the pixel value of the mask of the foreground area, the sum of the pixel values ​​of the mask of the foreground area and the pixel value of the target electron microscope image; or calculating the standard deviation of the foreground area using the pixel value of the target electron microscope image, the total number of pixels of the target electron microscope image and the average pixel value of the target electron microscope image.

[0106] In general, the standard deviation of the foreground and background areas is calculated and , as shown in formula (9):

[0107] (9).

[0108] The present invention uses the contrast with noise to evaluate the quality of the electron microscope image. The contrast with noise (CWN) is calculated as shown in formula (10):

[0109] (10).

[0110] Alternatively, other schemes may be used to evaluate the quality of electron microscope images. For example, after the foreground and background masks are generated, the foreground and background contrast calculation formula (i.e., Formula 1) may be used to evaluate the image quality; after the foreground and background masks are generated, the contrast-to-noise ratio calculation formula (i.e., Formula 2) may be used to evaluate the image quality. When using Formulas (1) and (2), the calculation of the average pixel and the calculation of the standard deviation are shown in Formula (11):

[0111] (11),

[0112] Among them, I ave represents the average pixel value of the electron microscope image, represents the standard deviation, and N is the number of all pixels in the electron microscope image.

[0113] Based on the above electron microscope image quality assessment method, the present invention also provides an electron microscope image quality assessment device. Figure 5 The device is described in detail.

[0114] Figure 5The structure block diagram of the electron microscope image quality assessment device according to an embodiment of the present invention is schematically shown.

[0115] like Figure 5 As shown, the electron microscope image quality assessment device 500 includes a region classification and mask generation module 510 , an average pixel value calculation module 520 , a standard deviation calculation module 530 , and a noise contrast calculation module 540 .

[0116] The region differentiation and mask generation module 510 is configured to perform region differentiation processing on the target electron microscope image, obtain a foreground region and a background region of the target electron microscope image, and generate a mask for the foreground region and a mask for the background region, respectively. In one embodiment, the region differentiation and mask generation module 510 can be used to perform operation S310 described above, and will not be further described here.

[0117] The average pixel value calculation module 520 is used to calculate the average pixel value of the foreground area using the mask of the foreground area, and to calculate the average pixel value of the background area using the mask of the background area. In one embodiment, the average pixel value calculation module 520 can be used to perform the operation S320 described above, which will not be repeated here.

[0118] The standard deviation calculation module 530 is used to calculate the standard deviation of the foreground area using the average pixel value of the foreground area, and to calculate the standard deviation of the background area using the average pixel value of the background area. In one embodiment, the standard deviation calculation module 530 can be used to perform the operation S330 described above, which will not be repeated here.

[0119] Noise Contrast Calculation Module 540 is configured to calculate noise contrast for evaluating the quality of the target electron microscope image using the average pixel value and standard deviation of the foreground region and the average pixel value and standard deviation of the background region. In one embodiment, Noise Contrast Calculation Module 540 can be configured to perform operation S340 described above, and will not be further described herein.

[0120] According to an embodiment of the present invention, any multiple modules among the region segmentation and mask generation module 510, the average pixel value calculation module 520, the standard deviation calculation module 530, and the noise contrast calculation module 540 can be combined into a single module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present invention, at least one of the region segmentation and mask generation module 510, the average pixel value calculation module 520, the standard deviation calculation module 530, and the noise contrast calculation module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable method of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the region differentiation and mask generation module 510, the average pixel value calculation module 520, the standard deviation calculation module 530, and the noisy contrast calculation module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0121] The method or device provided by the present invention accurately distinguishes the foreground and background of a bioelectron microscope image and details the steps for generating foreground and background masks. The present invention also provides a corresponding processing method for eliminating noise points and uninteresting, too small or too large connected regions in the foreground and background, thereby eliminating these interfering factors. This allows for accurate distinction between the foreground and background, enabling accurate assessment of foreground and background resolution, a feat unattainable by conventional global assessment methods. Furthermore, the noise-containing contrast proposed by the method or device of the present invention not only allows for intuitive assessment of foreground and background differences, but also takes noise into account in the assessment, encompassing the impact of varying foreground and background noise levels on foreground and background resolution.

[0122] Specifically, when the foreground and background noise are significant, especially when When , which means that the foreground may appear smaller than the background due to noise. In this case, the noise contrast is less than or equal to 0, indicating that the foreground is indistinguishable. At the other extreme, if the foreground and background are distinct and there is no noise, the noise contrast can reach 1. Therefore, the noise contrast usually takes a value between 0 and 1, with a value of 0 indicating that the foreground and background are indistinguishable, and a value of 1 indicating that the foreground and background are clearly distinguishable, allowing for intuitive evaluation of the difference between the foreground and background.

[0123] On the other hand, noisy contrast can be degenerated into contrast plus a correction of the noise signal ratio under certain assumptions. For example, when the current background standard deviation is the same, you can use Instead, as shown in formula (12):

[0124] (12).

[0125] The above formula (12) shows that the noisy contrast takes into account both the contrast (the former) and the proportion of noise in the signal strength (the latter). If the influence of noise is not considered and the standard deviation is set to 0, the noisy contrast is completely consistent with the contrast expression. This shows that the noisy contrast not only retains the basic form of contrast, but also adds the estimation of foreground and background noise and its correction to the contrast. In the contrast-to-noise ratio, the standard deviation is calculated for the entire image to evaluate the noise, while in the noisy contrast proposed by the method of the present invention, the standard deviation is estimated for the foreground and background separately. When the foreground area noise (standard deviation ) or background area noise (standard deviation ) are significant, both may affect the distinction between foreground and background, which is consistent with actual conditions. Therefore, the method of the present invention uses different standard deviation estimates for foreground and background noise to more accurately estimate the impact of electron microscope image noise on imaging quality (especially foreground and background resolution).

[0126] Figure 6 The block diagram of an electronic device suitable for implementing the electron microscope image quality assessment method according to an embodiment of the present invention is schematically shown.

[0127] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 602 or programs loaded from a storage unit 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0128] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 executes the programs in the ROM 602 and / or RAM 603 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0129] According to an embodiment of the present invention, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.

[0130] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0131] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above, and / or one or more memories other than ROM 602 and RAM 603.

[0132] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code causes the computer system to implement the electron microscope image quality assessment method provided in an embodiment of the present invention.

[0133] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the computer program is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0134] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0135] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609 and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0136] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0139] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for evaluating electron microscope image quality, characterized in that: include: Performing region differentiation processing on the target electron microscope image to obtain a foreground region and a background region of the target electron microscope image, and generating a mask of the foreground region and a mask of the background region respectively; Calculating an average pixel value of the foreground area using the mask of the foreground area, and calculating an average pixel value of the background area using the mask of the background area; Calculating the standard deviation of the foreground area using the average pixel value of the foreground area, and calculating the standard deviation of the background area using the average pixel value of the background area; Calculating the noise contrast for evaluating the quality of the target electron microscope image using the average pixel value and standard deviation of the foreground area and the average pixel value and standard deviation of the background area, The calculation method of the noise contrast is shown in the following formula: Wherein, CWN represents the noisy contrast, represents the average pixel value of the foreground area, represents the average pixel value of the background area, represents the standard deviation of the foreground area, represents the standard deviation of the background area.

2. The method according to claim 1, characterized in that Performing regional differentiation processing on the target electron microscope image to obtain the foreground area and background area of ​​the target electron microscope image includes: Based on user needs, the pixel points where the target object in the target electron microscope image is located are used as the foreground area, and the pixel points where the non-target object is located are used as the background area, wherein the target object includes the membrane structure of a biological cell; The target electron microscope image is subjected to image segmentation based on a preset image morphology processing method or an image segmentation based on a machine learning method to obtain a target electron microscope image that distinguishes the foreground area and the background area, wherein the machine learning method includes a semantic segmentation neural network and a random forest classifier.

3. The method according to claim 2, characterized in that Also includes: When the pixel value of the foreground area is smaller than the pixel value of the background area, the maximum pixel value of the target electron microscope image is obtained, and the pixel value of the target electron microscope image is inverted using the maximum pixel value, thereby completing the preprocessing of the target electron microscope image.

4. The method according to claim 1, wherein Generating masks of the foreground area and the background area respectively includes: Calculating the pixel median value of the target electron microscope image, and using the pixel median value to perform a threshold binarization operation to obtain an initial mask of the foreground area and an initial mask of the background area respectively; Performing a closing operation on the initial mask of the foreground area to delete noise points in the initial mask of the foreground area; and deleting connected areas in the initial mask of the foreground area whose connected areas are smaller than a preset value to obtain an intermediate mask of the foreground area; Deleting a region whose thickness is greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground region to obtain a mask of the foreground region; An erosion operation and a dilation operation are performed on the initial mask of the background area to complete an opening operation on the mask of the background area, thereby obtaining a mask of the background area with the holes filled.

5. The method according to claim 4, characterized in that The preset value is set based on the attribute value of the target electron microscope image.

6. The method according to claim 4, characterized in that Deleting an area having a thickness greater than the theoretical thickness of the target resolution object from the intermediate mask of the foreground area, to obtain the mask of the foreground area includes: performing an etching operation and a dilation operation on the intermediate mask of the foreground area to thereby complete an opening operation on the intermediate mask of the foreground area, thereby obtaining a thin film of the foreground area; When the area value of the intermediate mask in the foreground area is 1 and the area value corresponding to the thin film in the foreground area is 0, setting the value of the corresponding area of ​​the intermediate mask in the foreground area to 1; When the area value of the intermediate mask in the foreground area is 0 and the area value corresponding to the thin film in the foreground area is 1, the value of the corresponding area of ​​the intermediate mask in the foreground area is set to 0.

7. The method according to claim 1, characterized in that Calculating the average pixel value of the foreground area using the mask of the foreground area includes: Calculating the pixel values ​​of the target electron microscope image and the pixel values ​​of the area corresponding to the mask of the foreground area, and calculating the calculation result and the sum of the pixel values ​​of the mask of the foreground area to obtain the average pixel value of the foreground area; or The sum of the pixel values ​​of the target electron microscope image and the total number of pixels of the target electron microscope image are calculated, and the calculation result is used as the average pixel value of the foreground area.

8. The method according to claim 7, characterized in that Calculating the standard deviation of the foreground area using the average pixel value of the foreground area includes: Calculating a standard deviation of the foreground area using an average pixel value of the foreground area, pixel values ​​of the mask of the foreground area, a sum of pixel values ​​of the mask of the foreground area, and pixel values ​​of the target electron microscope image; or The standard deviation of the foreground area is calculated using the pixel value of the target electron microscope image, the total number of pixels of the target electron microscope image, and the average value of pixels of the target electron microscope image.

9. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images