A method and device for automatic hole picking in cryo-em

Abstract

Embodiments of the present application provide a cryo-EM automatic hole selection method and device, the method comprising: obtaining a target image, the target image being obtained by a cryo-EM shooting a grid in a grid film carrying a biological sample after rapid freezing, the target image containing the grid, the grid having a plurality of round holes for binding with the biological sample, at least part of the plurality of round holes being different; performing image recognition on the target image to determine a first standard hole; filtering a plurality of the round holes based on the first standard hole to obtain first candidate holes; determining a hole selection area based on the first candidate holes; filtering the first candidate holes based on the position relationship between the hole selection area and the first candidate holes to obtain second candidate holes; filtering out a second candidate hole contaminated in the second candidate holes to obtain a second standard hole, and an electron microscope image obtained based on the second standard hole can show a two-dimensional structure of the biological sample.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of cryo-EM, and in particular to a method and device for automatic hole selection in cryo-EM. BACKGROUND

[0002] Cryo-EM is mainly applied in the field of structural biology, and provides high-resolution structures for proteins that are not suitable for crystallization, and explains the relationship between structure and function at the atomic level. In recent years, the shooting speed of cryo-EM has been greatly improved, and more than 5000 two-dimensional particle images can be taken per day. The process before shooting of cryo-EM mainly includes the following steps: 1. After placing the user's sample on the cryo-EM stage, adjust the machine; 2. Shoot the large map and the middle map (the large map contains the overall image of the support grid with biological samples, and the middle map contains the image of a grid on the support grid); 3. Select the position of the last round hole to be shot on the middle map. The round hole is located in the above-mentioned grid and belongs to the support grid, each grid has multiple round holes, and the round hole carries part of the biological sample, and the round hole is surrounded by carbon film. Cryo-EM can shoot a two-dimensional structure image of the biological sample, such as the internal structure of a protein or a virus, through the round hole. However, due to the deformation of the support grid with the biological sample during the rapid freezing process, or the difficulty in processing the support grid due to its small size, the size of each round hole is different, and other factors, so not every round hole on the support grid is suitable for shooting a two-dimensional structure image for subsequent auxiliary observation of the internal structure of the biological sample. There is even no biological sample in some round holes, or some round holes are contaminated during the rapid freezing process, such as ice contamination, which causes the biological sample inside to be contaminated and damaged, and the contrast of the biological sample structure in the two-dimensional structure image obtained by shooting is reduced, making it difficult to process later. Therefore, in order to shoot a good two-dimensional structure image of the biological sample, cryo-EM needs to select a hole in the multiple round holes in the grid, select a round hole that can clearly display the structure of the biological sample, and then shoot the round hole to obtain a two-dimensional structure image of the biological sample.

[0003] The current hole selection method is manually observed and selected, including: first, a user circles a polygon on a map; second, the user selects five points (representing circular holes) in the polygon according to a preset rule; third, the positions of other holes are generated according to the selected five points. The above method mainly relies on the assumption that the distance between holes is basically equal. However, the above method has the following problems: when the map is cracked, such as grid cracking, or the map is formed by image splicing corresponding to multiple different grids, the distance between holes changes, so the user needs to circle multiple polygons (equivalent to determining the hole selection area first), then select five points in each polygon, and then generate the remaining points; second, this hole selection method is very dependent on whether the initial five points are accurately selected at the center of the circle, and depends on the equal distance between holes. In most cases, the above two conditions cannot be met, so the user needs to manually adjust the position of the selected hole to be on the center of the circle; in addition, the other holes determined by this method cannot automatically exclude contaminated holes, such as ice-contaminated holes. When there is ice in the selected hole, the quality of the two-dimensional photo taken is very poor and has no reference value, so it also needs to be manually excluded. The process is very tedious. SUMMARY

[0004] The embodiment of the present application provides a cryo-EM automatic hole selection method, comprising:

[0005] obtaining a target image, the target image being obtained by photographing a grid of a grid net loaded with a biological sample after rapid freezing by cryo-EM, the target image containing the grid of the grid net, the grid having a plurality of circular holes for binding with the biological sample, and at least part of the plurality of circular holes being different;

[0006] performing image recognition on the target image to determine a first standard hole;

[0007] filtering a plurality of the circular holes based on the first standard hole to obtain first candidate holes;

[0008] determining a hole selection area based on the first candidate holes;

[0009] filtering the first candidate holes based on the positional relationship between the hole selection area and the first candidate holes to obtain second candidate holes;

[0010] filtering out a second candidate hole contaminated in the second candidate holes to obtain a second standard hole, and an electron microscope image obtained based on the second standard hole can show a two-dimensional structure of the biological sample.

[0011] Optionally, the image recognition on the target image to determine a first standard hole comprises:

[0012] preprocessing the target image to enhance contrast of the target image;

[0013] performing image recognition on the preprocessed target image, and forming a plurality of virtual holes for calibrating positions of the round holes, the virtual holes being non-overlapping with the round holes or at least partially overlapping with the round holes;

[0014] determining the first standard hole based on at least the overlapping of the round holes and the virtual holes.

[0015] Optionally, the determining the first standard hole based on at least the overlapping of the round holes and the virtual holes comprises:

[0016] extending an edge of each of the virtual holes by η pixels to obtain an extended ring formed by an outer edge of the virtual hole before and after the extension;

[0017] filtering out a first virtual hole with an outer edge not smaller than an outer edge of a corresponding round hole based on a pixel value of the extended ring;

[0018] determining the first standard hole based on at least the first virtual hole and the round hole corresponding to the first virtual hole.

[0019] Optionally, the determining the first standard hole based on at least the first virtual hole and the round hole corresponding to the first virtual hole comprises:

[0020] shrinking an edge of each of the first virtual holes by x pixels to obtain a shrunk ring formed by an outer edge of the first virtual hole before and after the shrinking;

[0021] filtering out a second virtual hole with an outer edge not larger than an outer edge of a corresponding round hole based on a pixel value of the shrunk ring;

[0022] determining the first standard hole based on at least the second virtual hole and the round hole corresponding to the second virtual hole.

[0023] Optionally, the determining the first standard hole based on at least the second virtual hole and the round hole corresponding to the second virtual hole comprises:

[0024] determining a difference between a pixel value of the shrunk ring and a pixel value of an extended ring of each of the second virtual holes;

[0025] filtering out a third virtual hole with a specific difference between the pixel value of the shrunk ring and the pixel value of the extended ring based on the difference;

[0026] determining the first standard hole based on at least the third virtual hole and the round hole corresponding to the third virtual hole.

[0027] Optionally, the determining the first standard hole based on the third virtual hole and the circular hole corresponding to the third virtual hole comprises:

[0028] determining a first gray value of the third virtual hole;

[0029] determining the third virtual hole with the highest first gray value as a candidate standard virtual hole;

[0030] adjusting the position of the candidate standard virtual hole relative to the circular hole corresponding to the candidate standard virtual hole, and determining a second gray value in the candidate standard virtual hole after each adjustment;

[0031] determining the adjusted candidate standard virtual hole with the lowest second gray value as a standard virtual hole;

[0032] determining the circular hole corresponding to the standard virtual hole as the first standard hole, the first standard hole and the standard virtual hole satisfying the same condition, the same condition indicating that the standard virtual hole is considered the same as the first standard hole.

[0033] Optionally, the filtering a plurality of circular holes based on the first standard hole to obtain first candidate holes comprises:

[0034] determining the similarity of each circular hole to the first standard hole;

[0035] determining the first candidate holes based on the similarity and a similarity threshold.

[0036] Optionally, the filtering the first candidate holes based on the positional relationship between the edge of the selected hole region and the first candidate holes to obtain second candidate holes comprises:

[0037] determining the edge of the selected hole region, the selected hole region containing the first candidate holes;

[0038] filtering the first candidate holes based on the positional relationship between the edge of the selected hole region and the edge of the first candidate holes to obtain second candidate holes whose distance between the edge of the selected hole region and the edge of the first candidate holes satisfies a distance threshold.

[0039] Optionally, the filtering out the second candidate holes that are contaminated in the second candidate holes to obtain second standard holes comprises:

[0040] performing contamination prediction on the second candidate holes by a contamination identification model;

[0041] filtering out the second candidate holes predicted to be contaminated based on the prediction result to obtain the second standard holes;

[0042] The training process of the contamination identification model comprises:

[0043] obtain training data, the training data including multiple images each having the round holes, and each of the round holes in the images being labeled to indicate each of the round holes as a contaminated round hole or an un-contaminated round hole;

[0044] train the established neural network model based on the training data to obtain the contamination identification model.

[0045] Optionally, the embodiment of the present application further provides a cryo-EM automatic hole selection device, comprising:

[0046] an obtaining module, configured to obtain a target image, the target image being obtained by a cryo-EM shooting a mesh carrying a biological sample after rapid freezing, the target image containing a grid in the mesh, and the grid having a plurality of round holes combined with the biological sample, and at least part of the round holes being different;

[0047] an identifying module, configured to perform image recognition on the target image to determine a first standard hole;

[0048] a first filtering module, configured to filter the plurality of round holes according to the first standard hole to obtain first candidate holes;

[0049] a determining module, configured to determine a hole selection area according to the first candidate holes;

[0050] a second filtering module, configured to filter the first candidate holes according to a positional relationship between the hole selection area and the first candidate holes to obtain second candidate holes;

[0051] a third filtering module, configured to filter out a second candidate hole contaminated in the second candidate holes to obtain a second standard hole, and an electron microscope image obtained based on the second standard hole being able to show a two-dimensional structure of the biological sample.

[0052] Based on the disclosure of the above embodiments, it can be known that the embodiment of the present application has the beneficial effects including:

[0053] 1. The target image is processed based on an image recognition algorithm, batch automatic selection of the round holes is realized, and the selected round holes are filtered layer by layer to obtain the second standard hole which can better show the real two-dimensional structure of the biological sample, the selection of the second standard hole does not involve the distance between the round holes, so even when the distance between the round holes changes, the second standard hole can be accurately selected, in addition, the hole selection process does not need manual intervention, and the hole selection efficiency is significantly improved.

[0054] 2. By determining the first standard hole, determining a plurality of first candidate holes similar to the first standard hole based on the first standard hole, and then automatically determining the hole selection area based on the first candidate hole, manual selection is no longer required, and the selection accuracy is higher. In addition, by determining the hole selection area, the first candidate hole whose position relationship with the hole selection area and the first candidate hole does not meet the requirements can be automatically determined and removed, further improving the hole selection accuracy.

[0055] 3. By training the pollution prediction model, the second candidate hole can be automatically detected and screened for pollution without manual identification and manual deletion of the polluted round hole, further improving the hole selection efficiency and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The flowchart of the cryo-EM automatic hole selection method in the embodiment of the present application.

[0057] Figure 2 The flowchart of the cryo-EM automatic hole selection method in another embodiment of the present application.

[0058] Figure 3 The flowchart of the cryo-EM automatic hole selection method in the embodiment of the present application.

[0059] Figure 4 The hole selection process chart generated in the actual execution process of the cryo-EM automatic hole selection method in the embodiment of the present application, which shows the state of the target image when the first standard hole is determined, wherein the number 1 is a (real) round hole, the number 2 is a virtual round hole, and the number 3 is a first standard hole.

[0060] Figure 5 The hole selection process chart generated in the actual execution process of the cryo-EM automatic hole selection method in the embodiment of the present application, which shows the state of the target image when the second standard hole is determined.

[0061] Figure 6 The structure block diagram of the cryo-EM automatic hole selection device in the embodiment of the present application. DETAILED DESCRIPTION

[0062] Hereinafter, specific embodiments of the present application will be described in detail with reference to the accompanying drawings, but not as a limitation of the present application.

[0063] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but only as an example of the embodiments. Those skilled in the art will think of other modifications within the scope and spirit of the present disclosure.

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and, together with the general description of the disclosure given above, and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0065] These and other characteristics of the present application will become apparent from the following description of the preferred forms thereof given, by way of non-limiting example only, with reference to the accompanying drawings.

[0066] It is also to be understood that even though a number of embodiments of the application have been described herein, the application covers all possible combinations and sub-combinations of the various elements and features described herein.

[0067] The above and other aspects, features, and advantages of the present disclosure will become more apparent from the following detailed description, when taken in conjunction with the accompanying drawings, in which:

[0068] Specific embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings; however, it will be understood that the disclosed embodiments are merely examples of the present disclosure, which can be implemented in numerous ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present disclosure unnecessarily. Therefore, specific structural and functional details disclosed herein are not intended to limit the present disclosure, but merely to set a representative basis for teaching one skilled in the art to employ the present disclosure in virtually any appropriate detailed structure.

[0069] The specification can use phrases such as "in one embodiment", "in another embodiment", "in yet another embodiment", or "in other embodiments", which can refer to one or more of the same or different embodiments of the present disclosure.

[0070] Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0071] As shown in the drawings, the embodiments of the present application provide a method for automatically selecting holes in cryo-EM, comprising: Figure 1

[0072] obtaining a target image, the target image being obtained by cryo-EM photographing a mesh supporting a biological sample after rapid freezing, the target image containing a mesh in the mesh, the mesh having a plurality of circular holes combined with the biological sample, at least part of the plurality of circular holes being different;

[0073] performing image recognition on the target image to determine a first standard hole;

[0074] filtering the plurality of circular holes based on the first standard hole to obtain a first candidate hole;

[0075] determining a hole selection area based on the first candidate hole;​

[0076] Filter the first candidate holes based on the positional relationship between the selected hole region and the first candidate holes, to obtain second candidate holes;

[0077] Filter out the second candidate holes that are contaminated in the second candidate holes, to obtain second standard holes, the second standard holes can be used for subsequent shooting of clear biological sample two-dimensional structure, that is, the electron microscope image shot based on the second standard holes can show the two-dimensional structure of the biological sample.

[0078] For example, the grid of the support grid carrying the biological sample after being frozen is imaged by cryo-EM to obtain a large map containing the overall image of the support grid, and then the large map is processed to obtain a medium map corresponding to the grid of the support grid, i.e., the target image. Alternatively, the grid of the support grid is directly imaged by cryo-EM to obtain the target image (hereinafter referred to as the medium map). The medium map in this embodiment only contains the image of one grid, which can be an overall grid or a partial area formed by splicing other grids. Since each grid in the support grid has a plurality of circular holes for carrying biological samples, i.e., the user can observe through the circular holes to capture the two-dimensional structure of the biological sample. However, since the plurality of circular holes in the grid are not completely the same after the support grid is frozen, for example, some circular holes change in shape, produce different shapes, have impurities in the holes, or do not have biological samples in the holes, etc., the plurality of circular holes in this embodiment are not completely the same. Further, when the system obtains the medium map, it can be converted into a JPG format image (the specific format is not fixed), and then the image recognition algorithm is used to perform image recognition on the medium map to determine the first standard hole. The first standard hole can be understood as a hole that can better display the two-dimensional structure of the biological sample, or the first standard hole can be understood as a second standard hole, i.e., the first standard hole belongs to the second standard hole, or the first standard hole can only meet the requirements in some aspects, and the specific aspect that meets the requirements can be determined by the user according to the needs. In actual application, different types of threshold ranges can be set to filter out the first standard hole, such as when the user wants to select a circular hole with a radius that meets the specified requirements, the first standard hole can be determined based on the specified requirements. When the first standard hole is determined, the plurality of circular holes can be filtered based on the first standard hole, for example, the first candidate hole that meets the requirements in terms of radius, grayscale value, etc. is filtered based on the first standard hole, and the first candidate hole is at least partially similar or identical to the first standard hole. Then, the system can determine the hole selection area on the medium map based on the determined first candidate hole, such as an area containing all first candidate holes, or an area containing part of the first candidate holes. The hole selection area is usually represented as a polygon on the medium map. In order to improve the efficiency of hole selection, reduce the subsequent screening load, and avoid errors in the selection of the hole selection area, affecting the accuracy of subsequent hole selection, the system filters the first candidate hole based on the relative position of the hole selection area and each first candidate hole to obtain a second candidate hole with a relative position relationship that meets the requirements. Then, the system automatically identifies whether each second candidate hole is contaminated, such as ice contamination, and if it is contaminated, it is determined that the two-dimensional structure of the biological sample in the second candidate hole cannot be clearly displayed, and the reference value is weak, so the second candidate hole is directly excluded, and the second standard hole that can clearly display the two-dimensional structure of the biological sample to the user is finally left.When the number of images capable of showing different two-dimensional structures of the biological sample reaches a certain amount, the user can reconstruct the three-dimensional structure of the biological sample based on the plurality of images, so as to more clearly show the internal structure of the biological sample.

[0079] Based on the disclosure of the above embodiment, it can be known that the beneficial effects of the embodiment include that the target image is processed by the image recognition algorithm, the batch automatic selection of the round holes is realized, the selected round holes can be filtered layer by layer, and the second standard hole capable of more clearly showing the two-dimensional structure in the biological sample is obtained. The selection of the second standard hole does not involve the distance between the round holes, so even when the short distance between the round holes changes, the second standard hole can be accurately selected, and the hole selection process does not need manual intervention, which significantly improves the hole selection efficiency. In addition, by determining the first standard hole, a plurality of first candidate holes similar to the first standard hole are determined based on the first standard hole, and then the hole selection area is automatically determined based on the first candidate hole, so that manual selection is no longer needed, and the selection accuracy is higher. Moreover, by determining the hole selection area, the first candidate hole whose position relationship with the hole selection area does not meet the requirement can be automatically determined and removed, further improving the hole selection accuracy. Further, the method in the embodiment can also automatically identify the second candidate hole, determine the second candidate hole that is contaminated, and automatically remove it, without the need for manual identification, completely eliminating the manual operation process, unifying the hole selection standard, and reducing the error caused by manual hole selection.

[0080] Specifically, as shown in Figure 2 The image recognition is performed on the target image to determine the first standard hole, including:

[0081] The target image is preprocessed to improve the contrast of the target image;

[0082] The preprocessed target image is subjected to image recognition, and a plurality of virtual round holes for calibrating the positions of the round holes are formed. The virtual round hole is either non-overlapping with the round hole or at least partially overlapping with the round hole.

[0083] The first standard hole is determined based at least on the overlapping condition of the round hole and the virtual round hole.

[0084] For example, the obtained mid-map is first preprocessed to enhance the contrast of the picture, facilitating the identification of the round holes therein. The pre-processing process includes but is not limited to image normalization and histogram equalization processing. After the pre-processing, the preprocessed mid-map is subjected to image recognition by a Hough transform circle recognition algorithm to form a plurality of virtual round holes for calibrating the positions of the real round holes on the mid-map, i.e., the virtual round holes are actually calculated based on the Hough transform circle method and have a certain error. Each virtual round hole cannot exactly enclose the real round hole, i.e., cannot be highly overlapped with the round hole, for example, as shown inFigure 4 As shown, the size of the overlapping area of the real circular hole referred to by number 1 and the virtual circular hole referred to by number 2 has various cases. Many virtual circular holes can not select any real circular hole, that is, the position of the virtual circular hole is on the carbon film outside the real circular hole, and there is no intersection with the real circular hole. Some virtual circular holes only select part of the real circular hole, have an intersection area. Some virtual circular holes select the real circular hole, but the radius is greater than or less than the real circular hole, resulting in a failure to select the real circular hole. In order to select the real circular hole that meets the requirements, the system needs to determine the virtual circular hole that is highly overlapped with the real circular hole, so as to realize the selection of the real circular hole through the virtual circular hole. Therefore, the system needs to determine the overlapping condition of each virtual circular hole and the real circular hole, and then determine the first standard hole based on at least the overlapping condition. The first standard hole can be understood as a virtual circular hole, or as a real circular hole. When the virtual hole is highly overlapped with the real hole, the two holes can be considered as the same hole.

[0085] Further, in the embodiment, when the first standard hole is determined based on at least the overlapping condition of the circular hole and the virtual circular hole, the method comprises:

[0086] The edge of each virtual circular hole is expanded by η pixels to obtain an expanded circular ring formed by the outer edges of the virtual circular hole before and after expansion;

[0087] The first virtual hole with at least part of the outer edge not less than the outer edge of the corresponding circular hole is filtered out based on the pixel value of the expanded circular ring;

[0088] The first standard hole is determined based on at least the first virtual hole and the circular hole corresponding to the first virtual hole.

[0089] For example, as shown in FIG. 6, the first standard hole is determined based on at least the first virtual hole and the circular hole corresponding to the first virtual hole. Figure 3As shown, in order to eliminate the virtual circular hole with a radius smaller than that of the real circular hole, so that the virtual circular hole falls within the real circular hole, and the virtual circular hole and the real circular hole, the outer edge of each virtual circular hole can be expanded by η pixels (the specific pixel value is not fixed), to obtain an expanded circular ring formed by the outer edges of the virtual circular hole before and after expansion, calculate the average pixel value α of all expanded circular rings, and select the virtual circular hole with an average pixel value of the expanded circular ring greater than α. Because the higher the pixel, the higher the brightness value, and the outer part of the real circular hole is usually a carbon film, because in the process of taking the map, the proportion of electrons passing through the carbon film is smaller than the proportion of electrons passing through the real circular hole, so the color of the carbon film on the map is relatively deep, and the color of the real circular hole is relatively shallow. The method in this embodiment can accurately determine the virtual circular hole that falls within the real circular hole or is partially located within the real circular hole, i.e. the virtual circular hole and the corresponding real circular hole that fails to be selected, by determining the brightness value of the expanded circular ring, the color depth method, and then eliminating it, to obtain the first virtual circular hole with an outer edge at least partially not smaller than the outer edge of the corresponding real circular hole, for example, to obtain the first virtual circular hole with an outer edge or at least partially located outside the real circular hole, or highly coinciding with the outer edge of the real circular hole. At this time, the system can assist in determining the first standard hole based on the first virtual circular hole and the corresponding real circular hole.

[0090] Further, in this embodiment, when determining the first standard hole based on at least the first virtual hole and the circular hole corresponding to the first virtual hole, it includes:

[0091] The edge of each first virtual hole is inwardly retracted by x pixels to obtain an inwardly retracted circular ring formed by the outer edges of the first virtual hole before and after retraction;

[0092] Filtering out the second virtual hole with an outer edge at least partially not larger than the outer edge of the corresponding circular hole based on the pixel value of the inwardly retracted circular ring;

[0093] Determining the first standard hole based on at least the second virtual hole and the circular hole corresponding to the second virtual hole.

[0094] For example, continuing to combine Figure 3Similarly, in order to screen out the virtual circular holes that are highly coincident with the outer edges of the real circular holes, the system can shrink the edge of the first virtual hole by x pixels (the specific pixel value is indefinite) to obtain a shrinked circular ring formed by the outer edges of the first virtual hole before and after shrinking, then determine the average pixel value β of all the shrinked circular rings, and screen out the second virtual circular hole whose average pixel value of the shrinked circular ring is less than β, i.e., the shrinked circular ring is located on the carbon film and has a lower pixel value, when it is less than the average pixel value β, it can be determined that the radius of the virtual circular hole is greater than the radius of the corresponding real circular hole, or the virtual circular hole is at least partially located outside the real circular hole. In this embodiment, the second virtual circular hole filtered out is at least partially outside the outer edge of the corresponding real circular hole, i.e., the outer edge of the second virtual circular hole does not fall outside the corresponding real circular hole. Through the above two screenings, the filtering method can effectively remove the virtual circular holes and real circular holes that are positionally biased or have a large error in shape, and obtain the second virtual hole and the corresponding real circular hole that can effectively help the system to determine the first standard hole.

[0095] Further, in this embodiment, in determining the first standard hole based on at least the second virtual hole and the circular hole corresponding to the second virtual hole, the following steps are included:

[0096] determining the difference between the pixel value of the shrinked circular ring and the pixel value of the expanded circular ring of each second virtual hole;

[0097] filtering out the third virtual hole whose pixel value between the shrinked circular ring and the expanded circular ring has a specific difference based on the difference;

[0098] determining the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole.

[0099] For example, when a virtual circular hole does not enclose any real circular hole, such as when the virtual circular hole is entirely on the carbon film, or when the radius of the virtual circular hole is much smaller than that of a real circular hole and is located inside the real circular hole, etc., for such a virtual circular hole, its pixel values do not change significantly after expansion or contraction, but it still meets the above two screening conditions, respectively. The system may mistakenly identify such a virtual circular hole as a second virtual circular hole, resulting in errors and affecting the accuracy of hole selection. Therefore, in order to screen out such virtual circular holes, the system can determine the difference in pixel values between the inner and outer rings of each second virtual hole, and then determine the second virtual circular hole whose pixel values of the inner and outer rings have no significant difference, and eliminate it. For example, a second virtual circular hole with an inner ring average pixel value - an outer ring average pixel value > ε indicates that the pixel values of the two rings have a required difference, i.e., a specific difference (the specific difference value is indefinite). At this time, the system can identify the screened second virtual hole as a third virtual hole, and the third virtual hole is highly coincident with the outer edge of the corresponding real circular hole. The system can determine the first standard hole based on the third virtual hole and its corresponding real circular hole.

[0100] Further, the system determines the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole, including:

[0101] determining a first gray value of the third virtual hole;

[0102] determining the third virtual hole with the highest first gray value as a candidate standard virtual hole;

[0103] adjusting the position of the candidate standard virtual hole relative to the circular hole corresponding to the candidate standard virtual hole, and determining a second gray value in the candidate standard virtual hole after each adjustment;

[0104] determining the adjusted candidate standard virtual hole with the lowest second gray value as a standard virtual hole;

[0105] determining the circular hole corresponding to the standard virtual hole as the first standard hole, and the first standard hole and the standard virtual hole meet the same condition, which indicates that the standard virtual hole is considered the same as the first standard hole.

[0106] For example, continuing to combine Figure 3, the overall gray value of the real circular hole is high when it has a biological sample, and the overall gray value of the real circular hole is low when it has no biological sample inside, so based on this phenomenon, the system can determine the gray value of each third virtual hole, and then arrange it in ascending order of gray value, determine the maximum gray value, and determine the corresponding virtual circular hole as the candidate standard virtual hole based on the maximum gray value. Although the candidate standard virtual hole is highly coincident with the outer edge of the corresponding real circular hole, and has a biological sample inside, in order to increase the subsequent hole selection accuracy, in this embodiment, the system will further adjust the position of the candidate standard virtual hole to ensure that the center of the candidate standard virtual hole coincides with the center of the real circular hole, and is located at the center position of the circle, to ensure the specification of the determined first standard hole. In actual application, the center coordinates of the currently selected candidate standard virtual hole can be assumed to be (x, y), and the radius is r, and then the circles with (x-δ, y-δ) to (x+δ, y+δ) as the center and r as the radius are traversed in turn. By comparing the average gray values of the circles formed by different centers and the same radius, the position of the center of the circle corresponding to the minimum average gray value is found, and the position is determined as the new center of the candidate standard virtual hole. The position of the candidate standard virtual hole is adjusted based on the new center, and the standard virtual hole is obtained. The outer edge of the standard virtual hole is almost completely coincident with the outer edge of the corresponding real circular hole, and the coincidence degree is higher than before. At this time, the system can determine the real circular hole corresponding to the standard virtual hole as the first standard hole. The first standard hole and the standard virtual hole satisfy the same condition, which indicates that the first standard hole and the standard virtual hole can be regarded as the same hole, and the hole data can be regarded as the same. When filtering other real circular holes based on the first standard hole, it is equivalent to filtering other virtual circular holes based on the standard virtual hole. Finally, the second standard hole is determined, and the specific steps can be referred to in Figure 4 The circular hole referred to in number 3 in the embodiment is the first standard hole formed by the high coincidence of the virtual circular hole and the real circular hole. Optionally, the system can save the determined first standard hole as a picture for subsequent use.

[0107] Further, in this embodiment, the first candidate hole is obtained by filtering a plurality of circular holes based on the first standard hole, which includes:

[0108] determining the similarity of each circular hole to the first standard hole;

[0109] determining the first candidate hole based on the similarity and a similarity threshold.

[0110] Specifically, continue to combine Figure 3The matching degree of each circular hole in the selected hole region can be calculated from left to right and from top to bottom with respect to the first standard hole. The greater the matching degree, the more similar the circular hole is to the first standard hole, and the more likely the circular hole meets the requirements of the second standard hole. After determining the matching degree of each circular hole, the circular holes with a similarity greater than a threshold θ provided by a user can be screened out. The circular holes can be virtual circular holes or real circular holes. In actual applications, the first standard circular hole can be represented by T(x, y), the real circular hole or the virtual circular hole can be represented by I(x, y), and R(x, y) is a function used to describe the similarity. The similarity calculation function includes, but is not limited to, a standard correlation matching function (CV_TM_CCOEFF_NORMED) and a standard square difference matching function (CV_TM_SQDIFF_NORMED).

[0111] Further, in the embodiment, the first candidate hole is filtered based on the positional relationship between the edge of the selected hole region and the first candidate hole to obtain the second candidate hole, including:

[0112] The edge of the selected hole region is determined, and the selected hole region contains the first candidate hole.

[0113] The first candidate hole is filtered based on the positional relationship between the edge of the selected hole region and the edge of the first candidate hole to obtain the second candidate hole whose distance between the edge of the selected hole region and the edge of the first candidate hole satisfies a distance threshold.

[0114] For example, the selected hole region can be generated by the selected first candidate hole in combination with a geographic space algorithm or a convex hull algorithm. The selected hole region contains all the first candidate holes or only part of the first candidate holes. The selected hole region is usually a polygon. After the selected hole region and its edge are determined, the system can calculate the distance between the edge of the selected first candidate hole and the edge of the selected hole region according to a preset filtering distance ζ, and filter out the first candidate hole with a distance within ζ. Then, the remaining first candidate hole can be scaled, converted and saved as a picture with a size of σ*σ to train the pollution prediction model described below.

[0115] Further, in the embodiment, the second candidate hole that is polluted is filtered out from the second candidate hole to obtain the second standard hole, including:

[0116] The second candidate hole is predicted to be polluted by the pollution prediction model.

[0117] The second candidate hole predicted to be polluted is filtered out based on the prediction result to obtain the second standard hole.

[0118] The training process of the pollution prediction model includes:

[0119] The training data consists of multiple images, each containing a circular hole. The holes in the images are labeled to indicate whether each hole is either contaminated or uncontaminated.

[0120] A neural network model trained based on training data is used to obtain a pollution identification model.

[0121] Specifically, continue to combine Figure 3 As shown, to accelerate the hole selection process, reduce manual operation, and improve selection accuracy, this embodiment uses a contamination prediction model to filter the second candidate holes, eliminating those deemed contaminated by the model, and ultimately obtaining the second standard holes that meet the requirements. The model training includes the following steps:

[0122] Training data is obtained, which includes multiple images, each containing a circular hole (or a second candidate hole). The circular holes in the images are manually labeled to indicate whether each hole is either contaminated or uncontaminated.

[0123] The training data is then divided into training and test sets according to a set ratio. A convolutional neural network model is built using the PyTorch deep learning framework. The neural network model parameters are set, and the training set (training data) is input into the neural network model for training. The model is saved every n rounds during training. The model with the smallest loss function value is selected as the optimal model using the test set, and this optimal model forms the pollution prediction model.

[0124] Based on this contamination prediction model, the system can predict whether the second candidate wells are contaminated, obtaining a prediction result for each candidate well. The system can then filter the candidate wells based on these prediction results to obtain the desired second standard wells. For example, ... Figure 5 In the diagram, the black impurities represent ice contamination, and the corresponding round holes marked with crosshairs are the selected holes, also known as the second standard holes. After determining the second standard holes, the system can generate a coordinate file and convert it into a coordinate file that can be recognized and received by SerialEM (an open-source cryo-electron microscopy operating software). It can also be converted to other formats, depending on the interface used by the equipment to receive the coordinate file.

[0125] Specifically, to better describe the hole selection method in this embodiment, the following detailed description is provided in conjunction with specific embodiments:

[0126] Export the medium-sized map captured by cryo-electron microscopy as a JPG image;

[0127] Open the navigation file generated by cryo-electron microscopy using SerialEM;

[0128] Set the directory A for exporting images;

[0129] Through the SerialEM script command, traverse the SerialEM object, find the middle map, read the middle map file through the LoadOtherMap instruction, and export the JPG file to directory A through the SaveToOtherFile instruction;

[0130] The middle map is preprocessed, and each middle map is sequentially normalized through the normalize function of opencv and histogram equalization through the equalizeHist function of opencv;

[0131] The image after preprocessing is subjected to circle recognition processing through the HoughCircles (Hough transform circle detection algorithm) of opencv, wherein the minRadius parameter of HoughCircles is set to 35, and the maxRadiu parameter is set to 45. The recognized circle holes are as shown in Figure 4 , that is, a plurality of virtual circle holes are formed on the middle map (as shown in figure 2);

[0132] The generated virtual circle holes are respectively expanded by 5 pixels, and the outer expanded circle ring formed by the circumference of the current circle and the circumference of the expanded circle is obtained. The average pixel value α of all outer expanded circle rings is counted, and the circle with an average pixel value greater than α is selected, that is, the first virtual circle hole is obtained;

[0133] Similarly, the first virtual circle hole is respectively shrunk by 5 pixels, and the inner shrunk circle ring formed by the circumference of the current circle and the circumference of the shrunk circle is obtained. The average pixel value β of all inner shrunk circle rings is counted, and the circle with an average pixel value less than β is selected, that is, the second virtual circle hole is obtained. Through the above two steps, the position-biased virtual circle hole can be screened out. (Note that the execution order of the above two steps is not fixed and can be changed at will);

[0134] The virtual circle hole with an average pixel value of the inner shrunk circle ring minus the average pixel value of the outer expanded circle ring greater than 20 is selected, that is, the third virtual circle hole;

[0135] The average gray value of the third virtual circle hole selected by the above screening is calculated, and the virtual circle holes are arranged in ascending order according to the gray value. The virtual circle hole with the highest gray value or the corresponding circle hole is selected as the candidate standard virtual circle hole;

[0136] The selected candidate standard virtual circle hole is fine-tuned to ensure that the position of its center is in the center of the circle. Assuming that the center coordinates of the currently selected candidate standard virtual circle hole are (x, y), and the radius is r, the circles with (x-5, y-5) to (x+5, y+5) as the center and r as the radius are sequentially traversed. By comparing the average gray value changes of the circles formed by different centers and the same radius, the center position with the smallest average gray value is found as the center of the adjusted virtual circle hole, as shown inFigure 4 The virtual circular hole referred to by the number 3 shown is saved as a picture by the system.

[0137] The remaining virtual circular holes are identified using the first standard hole matching. The matchTemplate function of opencv is used to calculate the matching degree from left to right and from top to bottom on the map or the hole selection area, and the TM_CCOEFF_NORMED method of opencv is selected for the calculation method of the matching degree. Then, the system can filter out the virtual circular holes with a similarity greater than θ according to the preset similarity filtering threshold θ.

[0138] The hole selection area is generated by the selected virtual circular hole, and the convex hull algorithm (convexhull) in the scipy package of python is used to generate the circumscribed polygon of the selected circular hole, i.e., the hole selection area.

[0139] The filtering distance is set to 100 pixels, and the distance between the edge of the virtual circular hole in the hole selection area and the circumscribed polygon is calculated in sequence. The holes with a distance within ζ are filtered out. The filtering distance is set to 100, which can filter out the virtual circular holes adjacent to the circumscribed polygon. The filtering distance is not fixed, and a reasonable filtering distance can be selected according to the actual map situation.

[0140] The remaining virtual circular holes after filtering are scaled, converted and saved as 80*80 size pictures, which are used for training of the pollution prediction model.

[0141] The pollution prediction model can be an ice pollution prediction model, and its establishment process includes:

[0142] Create training data, which contains multiple images in the sample, each image has a circular hole, and the circular hole data is manually labeled. The clean circular hole, i.e., the non-polluted circular hole, is marked as 0, and the ice-polluted circular hole is marked as 1. The training data contains not less than 20,000 sample hole pictures, and the proportion of ice-polluted circular holes and clean circular holes is close to 1:1.

[0143] Then the sample set is divided into a training set and a test set in a ratio of 8:2, a LeNet convolutional neural network model is constructed under a Pytorch deep learning framework, the model can be set as a 6-layer neural network, the first layer is a convolutional layer, the size of the convolution kernel is 5*5, the number of input channels is 3, and the number of output channels is 6; the second layer is a pooling layer, the size of the kernel is 2, and the step is 2, the maximum element is extracted from 4 elements by using a max-pooling method to represent the 4 elements; the third layer is a convolutional layer, the size of the convolution kernel is 5*5, the number of input channels is 6, and the number of output channels is 16; the fourth layer is a fully connected layer, the input parameter is 16*5*5, and the output size is 120; the fifth layer is a fully connected layer, the input parameter is 120, and the output size is 84; the sixth layer is a fully connected layer, the input parameter is 84, and the output size is 2. A cross-entropy loss function CrossEntropyLoss is selected as a loss function, and a learning rate lr is 0.001. The training set is input into the neural network model, the neural network model is trained based on the Pytorch deep learning framework, a model is saved once every 10 rounds in the training process, and the model with the minimum loss function value is selected as an optimal model by using the test set, that is, a pollution prediction model is formed;

[0144] The pollution prediction model obtained by training is used for detecting the screened virtual circular hole / real circular hole, predicting the classification of the virtual circular hole / real circular hole, and outputting a prediction result, wherein the circular hole can be a virtual circular hole or a real circular hole, because the two satisfy the same condition and can be regarded as the same circular hole;

[0145] The circular hole predicted as ice pollution is filtered, and clean circular holes are reserved;

[0146] A coordinate file is generated, and the coordinate positions of the finally screened circular holes are converted into a coordinate file that can be recognized by SerialEM.

[0147] As shown in Figure 6 Another embodiment of the present application also provides a cryo-EM automatic hole selection device, comprising:

[0148] An obtaining module is configured to obtain a target image, wherein the target image is obtained by photographing a support net loaded with a biological sample after rapid freezing by a cryo-EM, the target image contains a grid in the support net, the grid contains a plurality of circular holes combined with the biological sample, and at least part of the plurality of circular holes are different;

[0149] An identification module is configured to perform image recognition on the target image to determine a first standard hole;

[0150] A first filtering module is configured to filter the plurality of circular holes according to the first standard hole to obtain a first candidate hole;

[0151] A determination module is configured to determine a hole selection area according to the first candidate hole.

[0152] a second filtering module configured to filter the first candidate holes according to a positional relationship between the selected hole region and the first candidate holes, to obtain second candidate holes;

[0153] a third filtering module configured to filter out second candidate holes that are contaminated from the second candidate holes, to obtain second standard holes, which can clearly show a two-dimensional structure of the biological sample after subsequent photographing.

[0154] Optionally, the identification module performs image recognition on the target image to determine the first standard holes, including:

[0155] preprocessing the target image to improve a contrast of the target image;

[0156] performing image recognition on the preprocessed target image, and forming a plurality of virtual round holes for calibrating positions of the round holes, the virtual round holes being at least partially overlapped with the round holes or having no overlapping region with the round holes;

[0157] determining the first standard holes based at least on the overlapping condition of the round holes and the virtual round holes.

[0158] Optionally, determining the first standard holes based at least on the overlapping condition of the round holes and the virtual round holes includes:

[0159] extending an edge of each virtual round hole outward by η pixels to obtain an extended circular ring formed by outer edges of the virtual round holes before and after the extension;

[0160] filtering out first virtual holes having at least partially an outer edge not smaller than an outer edge of a corresponding round hole based on pixel values of the extended circular ring;

[0161] determining the first standard holes based at least on the first virtual holes and the round holes corresponding to the first virtual holes.

[0162] Optionally, the first filtering module determines the first standard holes based at least on the first virtual holes and the round holes corresponding to the first virtual holes, including:

[0163] shrinking an edge of each first virtual hole inward by x pixels to obtain a shrunk circular ring formed by outer edges of the first virtual holes before and after the shrinking;

[0164] filtering out second virtual holes having at least partially an outer edge not larger than an outer edge of a corresponding round hole based on pixel values of the shrunk circular ring;

[0165] determining the first standard holes based at least on the second virtual holes and the round holes corresponding to the second virtual holes.

[0166] Optionally, the first filtering module determines the first standard holes based at least on the second virtual holes and the round holes corresponding to the second virtual holes, including:

[0167] determining a difference value between a pixel value of the inner retracted annulus and a pixel value of the outer expanded annulus of each second virtual hole;

[0168] filtering out a third virtual hole with a specific difference between the pixel value of the inner retracted annulus and the pixel value of the outer expanded annulus based on the difference value;

[0169] determining the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole.

[0170] Optionally, the first filtering module determines the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole, comprising:

[0171] determining a first gray value of the third virtual hole;

[0172] determining a candidate standard virtual hole as the third virtual hole with the highest first gray value;

[0173] adjusting the position of the candidate standard virtual hole relative to the circular hole corresponding to the candidate standard virtual hole, and determining a second gray value in the candidate standard virtual hole after each adjustment;

[0174] determining the adjusted candidate standard virtual hole with the lowest second gray value as the standard virtual hole;

[0175] determining the circular hole corresponding to the standard virtual hole as the first standard hole, the first standard hole and the standard virtual hole satisfying the same condition, the same condition indicating that the standard virtual hole is considered the same as the first standard hole.

[0176] Optionally, the first filtering module filters the plurality of circular holes based on the first standard hole to obtain a first candidate hole, comprising:

[0177] determining a similarity between each circular hole and the first standard hole;

[0178] determining the first candidate hole based on the similarity and a similarity threshold.

[0179] Optionally, the second filtering module filters the first candidate hole based on a positional relationship between the edge of the hole selection area and the first candidate hole to obtain a second candidate hole, comprising:

[0180] determining an edge of a hole selection area, the hole selection area containing the first candidate hole;

[0181] filtering the first candidate hole based on a positional relationship between the edge of the hole selection area and the edge of the first candidate hole to obtain a second candidate hole with a distance between the edge of the hole selection area and the edge of the first candidate hole satisfying a distance threshold.

[0182] The device further comprises the above-mentioned pollution identification model, and the training process of the pollution identification model comprises:

[0183] Obtain training data, the training data including multiple images each having a circular hole, and the circular hole in each image is labeled to indicate that each circular hole is either a contaminated circular hole or an uncontaminated circular hole.

[0184] Train the established neural network model based on the training data to obtain a contamination prediction model.

[0185] Optionally, the third filtering module filters out a second candidate hole that is contaminated from the second candidate hole to obtain a second standard hole, including:

[0186] Contamination prediction is performed on the second candidate hole by using the contamination prediction model;

[0187] Based on the prediction result, a second candidate hole that is predicted to be contaminated is filtered out to obtain a second standard hole.

[0188] Another embodiment of the present application further provides an electronic device, including:

[0189] One or more processors;

[0190] A memory configured to store one or more programs;

[0191] When the one or more programs are executed by the one or more processors, the one or more processors implement the hole selection method described above.

[0192] An embodiment of the present application further provides a storage medium having a computer program stored thereon, and the program is executed by a processor to implement the hole selection method described above. It should be understood that each scheme in the embodiment has the corresponding technical effects in the method embodiments described above, and will not be described here.

[0193] An embodiment of the present application further provides a computer program product tangibly stored on a computer readable medium and including computer readable instructions that, when executed, cause at least one processor to perform a hole selection method such as in the embodiments described above. It should be understood that each scheme in the embodiment has the corresponding technical effects in the method embodiments described above, and will not be described here.

[0194] It should be noted that the computer storage media of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium may, for example, but not limited to, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a carrier wave part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit the program configured for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, cable, RF, etc. or any suitable combination of the above.

[0195] It should be understood that although the present application is described in terms of various embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand.

[0196] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the spirit and protection scope of the present application, and such modifications or equivalent replacements should also be considered to fall within the protection scope of the present application.

Claims

1. A cryo-EM automatic hole picking method, characterized in that, The method comprises the following steps: obtaining a target image, the target image being obtained by taking a cryo-EM image of a grid carrying a biological sample after rapid freezing, the target image containing a grid in the grid, the grid having a plurality of circular holes for binding with the biological sample, at least part of the plurality of circular holes being different; performing image recognition on the target image to form a plurality of virtual circular holes for calibrating each real circular hole to determine a first standard hole; filtering the plurality of circular holes based on the first standard hole to obtain first candidate holes; determining a hole selection area based on the first candidate holes; filtering the first candidate holes based on the positional relationship between the hole selection area and the first candidate holes to obtain second candidate holes; filtering out a second candidate hole contaminated in the second candidate holes to obtain a second standard hole, and an EM image obtained based on the second standard hole can show a two-dimensional structure of the biological sample.

2. The method of claim 1, wherein, The image recognition on the target image to determine the first standard hole comprises: preprocessing the target image to improve the contrast of the target image; performing image recognition on the preprocessed target image and forming a plurality of virtual circular holes for calibrating the position of each circular hole, the virtual circular hole having no overlapping area with the circular hole or at least partially overlapping with the circular hole; determining the first standard hole based on at least the overlapping condition of the circular hole and the virtual circular hole.

3. The method of claim 2, wherein, The determination of the first standard hole based on at least the overlapping condition of the circular hole and the virtual circular hole comprises: extending the edge of each virtual circular hole by η pixels to obtain an extended circular ring formed by the outer edges of the virtual circular hole before and after extension; filtering out a first virtual hole with at least part of the outer edge not smaller than the outer edge of the corresponding circular hole based on the pixel value of the extended circular ring; determining the first standard hole based on at least the first virtual hole and the circular hole corresponding to the first virtual hole.

4. The method of claim 3, wherein, The determination of the first standard hole based on at least the first virtual hole and the circular hole corresponding to the first virtual hole comprises: shrinking the edge of each first virtual hole by x pixels to obtain a shrunk circular ring formed by the outer edges of the first virtual hole before and after shrinking; filtering out a second virtual hole with at least part of the outer edge not larger than the outer edge of the corresponding circular hole based on the pixel value of the shrunk circular ring; determining the first standard hole based on at least the second virtual hole and the circular hole corresponding to the second virtual hole.

5. The method of claim 4, wherein, The determination of the first standard hole based on at least the second virtual hole and the circular hole corresponding to the second virtual hole comprises: determining the difference between the pixel value of the shrunk circular ring and the pixel value of the extended circular ring of each second virtual hole; filtering out a third virtual hole with a specific difference between the pixel value of the shrunk circular ring and the pixel value of the extended circular ring based on the difference; determining the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole.

6. The method of claim 5, wherein, The determination of the first standard hole based on at least the third virtual hole and the circular hole corresponding to the third virtual hole comprises: determining the first gray value of the third virtual hole; determining the third virtual hole with the highest first gray value as a candidate standard virtual hole; adjusting the position of the candidate standard virtual hole relative to the corresponding circular hole, and determining a second gray value in the candidate standard virtual hole after each adjustment; determining the adjusted candidate standard virtual hole with the lowest second gray value as a standard virtual hole; determining the circular hole corresponding to the standard virtual hole as the first standard hole, wherein the first standard hole and the standard virtual hole satisfy the same condition, and the same condition indicates that the standard virtual hole is considered the same as the first standard hole.

7. The method according to claim 1 or 6, characterized in that, the filtering of the plurality of circular holes based on the first standard hole to obtain first candidate holes, comprising: determining the similarity of each circular hole to the first standard hole; determining the first candidate holes based on the similarity and a similarity threshold.

8. The method of claim 1, wherein, the filtering of the first candidate holes based on the positional relationship between the edge of the hole selection region and the first candidate hole to obtain second candidate holes, comprising: determining the edge of the hole selection region, wherein the hole selection region contains the first candidate hole; filtering the first candidate holes based on the positional relationship between the edge of the hole selection region and the edge of the first candidate hole to obtain second candidate holes whose distance between the edge of the hole selection region and the edge of the first candidate hole satisfies a distance threshold.

9. The method of claim 1, wherein, the filtering of the second candidate holes that are contaminated to obtain second standard holes, comprising: contamination prediction of the second candidate holes by a contamination prediction model; filtering the second candidate holes that are predicted to be contaminated based on the prediction results to obtain the second standard holes; wherein the training process of the contamination identification model comprises: obtaining training data, wherein the training data comprises a plurality of images each having the circular holes, and each circular hole in the images is labeled to indicate that each circular hole is either a contaminated circular hole or an uncontaminated circular hole; training a neural network model to obtain the contamination prediction model based on the training data.

10. A cryo-EM automatic hole picking device, characterized in that, comprising: an obtaining module configured to obtain a target image, wherein the target image is obtained by cryo-EM imaging of a grid after rapid freezing, the target image contains a grid in a grid, and the grid has a plurality of circular holes for binding to a biological sample, and at least part of the plurality of circular holes are different; an identification module configured to perform image recognition on the target image to form a plurality of virtual circular holes for calibrating each real circular hole to determine a first standard hole; a first filtering module configured to filter a plurality of circular holes based on the first standard hole to obtain first candidate holes; a determination module configured to determine a hole selection region based on the first candidate holes; a second filtering module configured to filter the first candidate holes based on the positional relationship between the hole selection region and the first candidate holes to obtain second candidate holes; a third filtering module configured to filter out second candidate holes that are contaminated in the second candidate holes to obtain second standard holes, and an electron microscope image obtained based on the second standard holes can display a two-dimensional structure of the biological sample.

Images