Micro-flow cell-based autofocusing method, electronic device, and storage medium

Abstract

The application belongs to the technical field of camera focusing, and particularly relates to an automatic focusing method based on a micro flow cell, an electronic device and a storage medium, the method comprising: performing flatness correction on the micro flow cell to determine target focusing positions of each field of view in the micro flow cell; based on the target focusing positions of each field of view, collecting a plurality of sample images under each field of view and constructing a sample image set; based on the sample image set, training a fixed-point focusing model; collecting a preset number of to-be-measured images under each field of view to construct a to-be-measured image set; and based on the to-be-measured image set and the trained fixed-point focusing model, determining an automatic focusing result. The technical scheme provided in the application solves the problems of large flatness variation caused by flow cell process error, non-constant sample liquid flow rate caused by automatic sampling, and blurred particle images caused by platform movement, and accurately and quickly realizes real-time automatic focusing, improves the accuracy of the automatic focusing result, and thus realizes accurate particle analysis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of camera focusing, and particularly relates to an automatic focusing method based on a micro flow cell, an electronic device and a storage medium. BACKGROUND

[0002] The AI+ particle detector is a precursor particle shape and size full-automatic analyzer integrating microscopic imaging technology and an electric motion platform, high-performance visual image processing algorithm and an AI large model. The particle image is collected through an automatic sampling system, and the particle micro characteristics are analyzed by using the image processing algorithm according to the particle size measurement theory. The instrument can detect particle information under different concentrations and different particle shapes and appearances through the micro flow cell. The performance of the focusing algorithm of the micro flow cell directly affects the imaging quality of the particle image, and thus is related to the accuracy of particle analysis, the stability of the experiment and the repeatability index of the instrument itself.

[0003] However, due to the problems such as non-constant flow rate of sample liquid caused by automatic sampling, unevenness of the bottom of the flow cell groove, limited moving step and repeated positioning accuracy of the electric platform, different sample liquid concentrations, different particle sizes and stratification, the collected particle images are blurred, there are trailing shadows in the local or whole part, there are halos in the particle part or some particles deviate from the focusing plane, the focusing result is unstable, and the like, which reduces the accuracy of particle analysis.

[0004] Therefore, how to improve the automatic focusing effect based on the micro flow cell to realize accurate particle analysis has become a problem to be solved. SUMMARY

[0005] The application provides an automatic focusing method based on a micro flow cell, an electronic device and a storage medium, which aims to improve the automatic focusing effect based on the micro flow cell to realize accurate particle analysis.

[0006] In a first aspect, an automatic focusing method based on a micro flow cell is provided, and the method comprises the following steps:

[0007] The flatness of the micro flow cell is corrected to determine the target focusing position of each field of view in the micro flow cell;

[0008] Based on the target focusing position of each field of view, a plurality of sample images under each field of view are collected to construct a sample image set;

[0009] Based on the sample image set, a fixed-point focusing model is trained;

[0010] A preset number of to-be-measured images under each field of view are collected to construct a to-be-measured image set;

[0011] Determine an auto-focusing result based on the set of to-be-tested images and the trained fixed-point focusing model.

[0012] In a possible implementation, the flatness correction of the micro flow cell and the determination of the target focusing position of each field of view in the micro flow cell include:

[0013] According to the camera parameters, the detection area of the micro flow cell is evenly divided into a plurality of fields of view;

[0014] A plurality of original images are captured under each field of view;

[0015] Based on the sharpness of each original image, a target focusing position of each field of view is determined, and the target focusing position includes the coordinates of a target focusing point.

[0016] In a possible implementation, the training of the fixed-point focusing model based on the set of sample images includes:

[0017] Based on the pixel points of each sample image in the set of sample images, a particle edge point in each sample image is determined;

[0018] Based on the particle edge point in each sample image, an edge gradient map corresponding to the sample image is determined;

[0019] Based on the pixel values of each edge gradient map, a sharpness score of the sample image corresponding to each edge gradient map is determined;

[0020] Based on the sharpness score of each sample image and a preset mapping relationship, a sharpness level of each sample image is determined and a label data set is constructed, and the preset mapping relationship is used to indicate the mapping relationship between the sharpness level and the sharpness score range;

[0021] The fixed-point focusing model is trained based on the set of sample images and the label data set.

[0022] In a possible implementation, the determination of the auto-focusing result based on the set of to-be-tested images and the trained fixed-point focusing model includes:

[0023] Based on the pixel points of each to-be-tested image in the set of to-be-tested images, a particle edge point in each to-be-tested image is determined;

[0024] Based on the particle edge point in each to-be-tested image, an edge gradient map corresponding to the to-be-tested image is determined;

[0025] All to-be-tested images and corresponding edge gradient maps under the same field of view are input into the trained fixed-point focusing model, and a target sharpness level corresponding to each field of view is determined.

[0026] determine a target image corresponding to each of the fields of view based on the set of images to be tested and the target sharpness level corresponding to each of the fields of view;

[0027] determine the automatic focusing result based on each of the target images.

[0028] In a possible implementation, the point focusing model comprises a model input module, an initial feature extraction module, a multi-scale feature interaction extraction module, a classification module, and a model output module connected in sequence.

[0029] The model input module is configured to input all target images to be tested and corresponding target edge gradient maps, the target images to be tested being images to be tested under a target field of view, and the target field of view being any of the fields of view.

[0030] The initial feature extraction module is configured to perform feature extraction on each of the target images to be tested and the corresponding target edge gradient maps to obtain an initial feature map corresponding to each of the target images to be tested.

[0031] The multi-scale feature interaction extraction module is configured to perform multi-scale feature interaction extraction on each of the initial feature maps to obtain a final feature map corresponding to each of the target images to be tested.

[0032] The classification module is configured to determine a sharpness level of each of the target images to be tested based on each of the final feature maps, and determine a target sharpness level corresponding to the target field of view based on the sharpness levels of each of the target images to be tested.

[0033] The model output module is configured to output the target sharpness level.

[0034] In a possible implementation, the initial feature extraction module comprises:

[0035] A global feature extraction unit and an edge feature extraction unit connected to the model input module respectively and arranged side by side, the global feature extraction unit being configured to extract global features of each of the target images to be tested to obtain a global feature map of each of the target images to be tested, and the edge feature extraction unit being configured to extract edge features of each of the target edge gradient maps to obtain an edge feature result map corresponding to each of the target edge gradient maps.

[0036] A feature fusion unit connected to the global feature extraction unit and the edge feature extraction unit, and configured to perform feature fusion on the global feature map and the edge feature result map corresponding to each of the target images to be tested to obtain a temporary result map corresponding to each of the target images to be tested.

[0037] An attention weight extraction unit, connected with the feature fusion unit, is configured to perform attention weight extraction on each of the temporary result maps to obtain an attention weight map corresponding to each of the target test images;

[0038] An attention adjustment unit, connected with the global feature extraction unit, the feature fusion unit and the attention weight extraction unit, is configured to perform attention gate dynamic adjustment on the corresponding global feature map and edge feature result map based on each of the attention weight maps to obtain an initial feature map corresponding to each of the target test images.

[0039] In a possible implementation, the multi-scale feature interaction extraction module comprises:

[0040] A first feature map extraction unit, connected with the initial feature extraction module, is configured to perform feature extraction based on each of the initial feature maps to obtain a first feature map corresponding to each of the target test images;

[0041] A second feature map extraction unit, connected with the initial feature extraction module, is configured to perform feature extraction based on each of the initial feature maps to obtain a second feature map corresponding to each of the target test images;

[0042] A third feature map extraction unit, connected with the second feature map extraction unit, is configured to perform feature extraction based on each of the initial feature maps to obtain a third feature map corresponding to each of the target test images;

[0043] A first feature map splicing unit, connected with the first feature map extraction unit and the third feature map extraction unit, is configured to perform feature splicing on the first feature map and the third feature map corresponding to each of the target test images to obtain a first feature splicing map corresponding to each of the target test images;

[0044] A fourth feature map extraction unit, connected with the first feature map splicing unit, is configured to perform feature extraction based on each of the first feature splicing maps to obtain a fourth feature map corresponding to each of the target test images;

[0045] A fifth feature map extraction unit, connected with the fourth feature map extraction unit, is configured to perform feature extraction based on each of the first feature splicing maps to obtain a fifth feature map corresponding to each of the target test images;

[0046] A second feature map splicing unit, connected with the second feature map extraction unit and the fifth feature map extraction unit, is configured to perform feature splicing on the second feature map and the fifth feature map corresponding to each of the target test images to obtain a second feature splicing map corresponding to each of the target test images;

[0047] The sixth feature map extraction unit is connected with the second feature map splicing unit, configured to perform feature extraction based on each second feature splicing map to obtain a sixth feature map corresponding to each target test image.

[0048] The third feature map splicing unit is connected with the fourth feature map extraction unit and the sixth feature map extraction unit, configured to perform feature splicing on the fourth feature map and the sixth feature map corresponding to each target test image to obtain a final feature map corresponding to each target test image.

[0049] In a possible implementation, the collecting a preset number of test images under each field of view and constructing a test image set comprises:

[0050] Converting each test image into a gray image;

[0051] Performing image anomaly detection based on each gray image to filter out candidate test images from each test image;

[0052] Constructing the test image set based on each candidate test image.

[0053] In a second aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the method according to the first aspect or any possible implementation manner thereof is implemented.

[0054] In a third aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the method according to the first aspect or any possible implementation manner thereof is implemented.

[0055] In a fourth aspect, a computer program product is provided. When the computer program product runs on an electronic device, the electronic device performs the method according to the first aspect or any possible implementation manner thereof.

[0056] Compared with the prior art, the beneficial effects of the embodiments of the present application are that: before focusing, the flatness of the micro flow cell is corrected first to determine the target focusing position of each field of view in the micro flow cell, avoiding the situation that the traditional automatic acquisition system needs to acquire a large number of images for each field of view to find the target focusing position, or even the system directly fails to find the target focusing position; based on the target focusing position of each field of view, a plurality of sample images under each field of view are acquired to construct a sample image set to train the fixed-point focusing model, thereby improving the accuracy of the sample image acquisition, and further improving the training effect of the fixed-point focusing model and the accuracy of the fixed-point focusing model; in the focusing process, a preset number of to-be-measured images under each field of view are acquired to construct a to-be-measured image set; based on the to-be-measured image set and the trained fixed-point focusing model, the automatic focusing result is determined, effectively correcting the situation that the flatness changes greatly due to the process error of the flow cell, the sample liquid flow rate is not constant due to the automatic sampling, and the particle image is blurred due to the platform movement under the microscope imaging, accurately and quickly realizing real-time automatic focusing, improving the accuracy of the automatic focusing result, and further realizing accurate particle analysis.

[0057] It can be understood that the electronic device, the computer readable storage medium and the computer program product provided by the embodiments of the present application have the same beneficial effects as the automatic focusing method based on the micro flow cell as described above, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0059] Figure 1 A flowchart of an automatic focusing method based on a micro flow cell provided by an embodiment of the present application;

[0060] Figure 2 A schematic view of a micro flow cell detection area field division in an automatic focusing method based on a micro flow cell provided by an embodiment of the present application;

[0061] Figure 3 A structural schematic view of an automatic focusing system in an automatic focusing method based on a micro flow cell provided by an embodiment of the present application;

[0062] Figure 4 A schematic view of a particle image in an automatic focusing method based on a micro flow cell provided by an embodiment of the present application;

[0063] Figure 5 FIG. 6 is a schematic diagram of another particle image in an autofocus method based on a micro-flow cell according to an embodiment of the present application;

[0064] Figure 6 FIG. 7 is a schematic diagram of an abnormal particle image in an autofocus method based on a micro-flow cell according to an embodiment of the present application;

[0065] Figure 7 FIG. 8 is a schematic diagram of an architecture of a fixed-point focusing model according to an embodiment of the present application;

[0066] Figure 8 FIG. 9 is a schematic diagram of an architecture of an initial feature extraction module in a fixed-point focusing model according to an embodiment of the present application;

[0067] Figure 9 FIG. 10 is a schematic diagram of an architecture of a multi-scale feature interaction extraction module in a fixed-point focusing model according to an embodiment of the present application;

[0068] Figure 10 FIG. 11 is a schematic diagram of another architecture of a fixed-point focusing model according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0070] It is to be understood that the terminology "includes", "has", "holds", "contains" used in the present specification and annexed claims are used to describe the presence of a feature, integer, step, operation, element, and / or component but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or a set thereof.

[0071] It is also to be understood that the terminology "and / or" used in the present specification and annexed claims refers to any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0072] As used in the specification and the appended claims, the term "if' can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0073] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0074] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0075] In order to facilitate understanding, the technical solutions of the present application will be described in detail below in conjunction with the drawings.

[0076] Figure 1 A flowchart of an automatic focusing method based on a micro flow cell provided by an embodiment of the present application is shown. As shown in the figure, for the convenience of description, only the parts related to the present embodiment are shown, and the method provided by the present embodiment includes the following steps: Figure 1

[0077] S1, flatness correction is performed on the micro flow cell to determine the target focusing position of each field of view in the micro flow cell.

[0078] ​In a specific implementation, due to the manufacturer's process and other problems, the flatness of the bottom of the micro-flow cell is not a constant value. The shape of the bottom of the micro-flow cell can be measured manually, for example, in the form of irregular steps. If an uncorrected micro-flow cell is used, the automatic acquisition system will encounter an unavoidable problem: in the detection area of the micro-flow cell, the field of view i and the field of view i+1 differ greatly due to the Z-axis focusing surface, i.e. the most clear position Zi of the Z-axis, which causes the platform to move from the field of view i to the field of view i+1, and the clear images cannot be acquired on the 5 images above and below Zi. Even if the number of images collected in each field of view is increased to 20, 60 or even 100 in order to find the clear surface, it is still not an effective solution and wastes collection time. Therefore, it is of great significance to correct the flatness of the micro-flow cell.

[0079] In a possible implementation, step S1 can optionally include the following steps:

[0080] S11, uniformly dividing the detection area of the micro-flow cell into a plurality of fields of view according to the camera parameters.

[0081] Preferably, before step S11 is performed, the automatic sampling system is closed, uniform precursor particles are selected, water and dispersant are added in a certain proportion and mixed uniformly as a standard sample solution, the standard sample solution is manually injected into the micro-flow cell through a syringe, and the micro-flow cell is placed in the platform clamp and left for 5 minutes.

[0082] As an example, according to the camera resolution and camera height and other camera parameters, the detection area of the micro-flow cell is uniformly divided into 50 fields of view arranged in 5x10, as shown in FIG. 1. Figure 2

[0083] Illustratively, the optical path of the micro-flow cell (cuvette) is 0.1 millimeters (mm), the groove depth is 0.1 mm, the material is ultraviolet quartz, the light transmittance of the flow cell is greater than 83%, and the content of the groove is about 30 microliters (ul). The XY direction movement step of the platform is 1 mm, the camera resolution is 2048x2448, the starting position of the detection area of the micro-flow cell is taken as the field of view A, the platform is moved and the real-time image is observed from the field of view A, each movement is a movement step, i.e. 1 mm, and the content (i.e. particles) of the current new field of view B is moved until it is not repeated in the field of view A, at which time the number of steps recorded is how many millimeters apart between each field of view; and the size of the detection area of the micro-flow cell is known, and the number of divided fields of view can be calculated according to the distance between the measured field of view A and the field of view B and the size of the detection area. Under normal circumstances, the number of fields of view in each column is the same, and the number of fields of view in each row is the same.

[0084] S12, shooting a plurality of original images under each field of view.

[0085] ​S13, based on the sharpness of each original image, determine the target focus position for each field of view, including the coordinates of the target focus point.

[0086] As an example, such as Figure 2 As shown, the motorized platform moves along the XY axis to the location of the first field of view. It then moves along the Z-axis and acquires 40 raw images. The clearest image is observed visually, and its corresponding Z-axis position z1 and the platform's movement step size are recorded. This process is repeated for the second field of view, where the platform moves along the Z-axis and acquires 40 raw images. The clearest image is observed visually, and its corresponding Z-axis position z2 and the platform's movement step size are recorded. This process continues until 40 raw images are acquired along the Z-axis for the 50th field of view. The clearest image is observed visually, and its corresponding Z-axis position z50 and the platform's movement step size are recorded. Through this process, the Z-axis positions Z_standard = {z1, z2, z3, ..., z50} corresponding to the clearest raw images in all fields of view are obtained. The position of the motorized platform corresponding to the clearest raw image in each field of view is used as the target focus point, and the three-dimensional coordinates of the target focus point are used as the target focus position for each field of view based on Z_standard.

[0087] Optionally, observe the Z-axis position Zi of the target focus position in each field of view. If the fields of view are continuous and the most sharp position Zi is almost identical, then divide these fields of view into one region. For example, if the bottom of the micro-flow cell exhibits a stepped shape, then... Figure 2 Each row of 10 fields of view belongs to the same area because they are continuous and their Zi values ​​are almost identical. The distance difference between adjacent areas is calculated, and this distance difference is represented in the software as the step size of the electric platform in the Z-axis direction. For example, the distance difference between the first and second rows of fields of view is k12 = 2000 steps; the distance differences between the second and third rows, the third and fourth rows, and the fourth and fifth rows are k23, k34, and k45, respectively.

[0088] As an example, in the formal detection, the electric platform directly moves to the Zi position of the first 10 fields of view to collect pictures (5 pictures are collected above and below Zi for each field of view), then moves to the Zi-k12 position of the 11th to 20th field of view to collect pictures (5 pictures are collected above and below Zi-k12 for each field of view), then moves to the Zi-k12-k23 position of the 21st to 30th field of view to collect pictures (5 pictures are collected above and below Zi-k12-k23 for each field of view), then moves to the Zi-k12-k23-k34 position of the 31st to 40th field of view to collect pictures (5 pictures are collected above and below Zi-k12-k23-k34 for each field of view), and then moves to the Zi-k12-k23-k34-k45 position of the 41st to 50th field of view to collect pictures (5 pictures are collected above and below Zi-k12-k23-k34-k45 for each field of view).

[0089] The technical solution provided by the above embodiment effectively avoids the disadvantages of the traditional automatic collection system, which needs to collect a large number of images for each field of view to find the focusing surface in the case that the container is not flat due to process unevenness, instrument or platform slight inclination, etc., or even directly paralyzed due to the inability to find a clear position. It can be seen that the flatness correction of the flow cell is of great significance to the automatic collection system and has wide applicability. In addition to micro flow cells, the flatness of containers such as slides and cuvettes can also be effectively corrected. Whether it is the stepped unevenness in the embodiment or other unevenness without any regularity, the above correction process can be used to obtain the target focusing position.

[0090] S2, based on the target focusing position of each field of view, a plurality of sample images under each field of view are collected and a sample image set is constructed.

[0091] Preferably, in order to improve the training effect of the fixed-point focusing model and the precision of the model, based on the target focusing position of each field of view, a plurality of sample images under different pH values, different reaction temperatures, different ligand concentrations, different feeding speeds, and different particle types of sample solutions are collected and a sample image set is constructed.

[0092] As an example, for any kind of sample solution, such as Figure 3The shown auto-focusing system, when collecting sample images, the motorized platform moves from close to the camera along the Z-axis direction, slowly approaches the light source, until it moves to the target focusing position of each field of view, and finally slowly deviates from the target focusing position to approach the light source. In this process, the image gradually becomes clear and then gradually blurred. At the same time, through the camera, 5 sample images are collected in the process of moving the motorized platform to the target focusing position of each field of view, 1 sample image is collected when moving to the target focusing position of each field of view, and 5 sample images are collected in the process of moving from the target focusing position of each field of view to the light source, that is, a total of 11 sample images are collected under each field of view, and the 11 sample images under each field of view are labeled according to the shooting order.

[0093] S3, training the fixed-point focusing model based on the sample image set.

[0094] Preferably, as shown in Figure 4 and Figure 5 The edge of the particle is often sharpened to a high degree, which is positively correlated with the particle clarity. The center of the particle has no rich texture information, and has little relationship with the image clarity, so it is less helpful to calculate the clarity. Therefore, the present application discards the center of the particle and selects the image clarity based on the particle edge.

[0095] In one possible implementation, step S3 can optionally include the following steps:

[0096] S31, determining the particle edge points in each sample image based on the pixel points of each sample image in the sample image set.

[0097] As an example, first, each sample image is converted to a corresponding gray image grayimg, and then each gray image is binarized to obtain a corresponding binary image bwimg; each pixel point in each binary image bwimg is traversed, and if at least two pixel points in the 8-neighborhood (up, down, left, right, top-left, bottom-left, top-right, bottom-right) of a pixel point have a value of 0, the pixel point is regarded as a particle edge point.

[0098] S32, determining the edge gradient image corresponding to each sample image based on the particle edge points in the sample image.

[0099] As an example, for each particle edge point in each sample image, the Sobel gradient value of the particle edge point in the corresponding gray image grayimg is calculated, and the gradient value of the non-particle edge point is directly set to 0 to generate the edge gradient image edge_image corresponding to each sample image, and the size is matched with the subsequent network input image.

[0100] S33, determining the clarity score of the sample image corresponding to each edge gradient image based on the pixel value of each edge gradient image.

[0101] As an example, the mean value of the pixel values of each edge gradient image edge_image is calculated as the sharpness score of the corresponding sample image.

[0102] Optionally, the sharpness scores of each group of sample images in the sample image set are manually confirmed, and the sharpness scores that do not meet the naked eye standard are corrected, and the corrected sharpness scores are normalized as the final sharpness scores.

[0103] S34, based on the sharpness scores of each sample image and the preset mapping relationship, determine the sharpness level of each sample image and construct a label data set, the preset mapping relationship is used to indicate the mapping relationship between the sharpness level and the sharpness score range.

[0104] As an example, the sharpness level has 6 levels, and the preset mapping relationship can optionally include: the sharpness score is greater than 0.83, and the corresponding sharpness level is 6; the sharpness score is less than 0.17, and the corresponding sharpness level is 1; the sharpness score range is [0.17, 0.34], and the corresponding sharpness level is 2; the sharpness score range is (0.34, 0.51], and the corresponding sharpness level is 3; the sharpness score range is (0.51, 0.68], and the corresponding sharpness level is 4; the sharpness score range is (0.68, 0.83], and the corresponding sharpness level is 5.

[0105] As an example, for any sample liquid, the number of sample images collected under each field of view is 11, and the moving sequence of the motorized platform in the Z axis during the collection process is: close to the camera, slowly approach the light source, until it moves to the target focus position, and finally slowly deviate from the target focus position to approach the light source. Therefore, ideally, based on the order of shooting, the labels of the 11 sample images under each field of view, the clearest image is the 6th image (the sharpness score is greater than 0.83, and the sharpness level is 6), the sharpness scores of the 1st and 11th images are close (the sharpness score is less than 0.17, and the sharpness level is 1), the sharpness scores of the 2nd and 10th images are close (the sharpness score range is [0.17, 0.34], and the sharpness level is 2), the sharpness scores of the 3rd and 9th images are close (the sharpness score range is (0.34, 0.51], and the sharpness level is 3), the sharpness scores of the 4th and 8th images are close (the sharpness score range is (0.51, 0.68], and the sharpness level is 4), and the sharpness scores of the 5th and 7th images are close (the sharpness score range is (0.68, 0.83], and the sharpness level is 5). At this point, all sample images can be determined according to the calculated sharpness score to determine the corresponding sharpness level, and a label data set is constructed.

[0106] S35, based on the sample image set and the label data set, training the fixed point focusing model.

[0107] S4, collecting a preset number of test images under each field of view to construct a test image set.

[0108] Preferably, the preset number is 11.

[0109] As an example, based on the sample liquid, in the autofocus system as shown in Figure 3 When collecting test images under each field of view, the motorized stage moves from close to the camera to the light source along the Z-axis direction, and at the same time, 11 test images are collected by the camera during the movement of the motorized stage, and the 11 test images under each field of view are labeled according to the shooting order.

[0110] Further, as shown in Figure 6 After the automatic sampling system is started, due to different degrees of sample liquid flow rate, limited minimum step size and repeated positioning accuracy of the motorized stage, and other factors, the collected test images may have abnormal light paths, ghosting abnormalities, etc. In order to solve the above problems, step S4 can optionally include the following steps:

[0111] Converting each test image into a grayscale image;

[0112] Based on each grayscale image, image abnormality detection is performed to select candidate test images from each test image;

[0113] Based on each candidate test image, a test image set is constructed.

[0114] As an example, test images corresponding to grayscale images with a grayscale mean value less than a first preset threshold or greater than a second preset threshold are removed to determine the candidate test images;

[0115] And / or performing Laplacian gradient detection on each grayscale image to determine a set of pixel points with a gradient value greater than a third preset threshold, calculating a gradient mean value for the set of pixel points; removing test images corresponding to grayscale images corresponding to the set of pixel points with a gradient mean value less than a fourth preset threshold to determine the candidate test images;

[0116] And / or traversing the grayscale value of each pixel point in each grayscale image to calculate the neighborhood contrast of each pixel point; based on the neighborhood contrast of each pixel point, calculating the neighborhood contrast mean value of each grayscale image; removing test images corresponding to grayscale images with a neighborhood contrast mean value less than a fifth preset threshold to determine the candidate test images.

[0117] Illustratively, if the grayscale mean value of the grayscale image is less than the first preset threshold or greater than the second preset threshold, the image has an abnormal light path. It may be that the microscope light path is incorrect, or the sample liquid speed drawn into the automatic sampling system is too fast, or a large number of particles flow into the narrow flow cell and cannot be precipitated in time for collection, so that too many particles shield the light path, resulting in a dark image.

[0118] Exemplarily, Laplacian gradient detection is performed on each gray-scale image to determine a pixel point set with a gradient value greater than a third preset threshold, and a gradient mean value is calculated for the pixel point set. If the gradient mean value is less than a fourth preset threshold, the particles in the image can all have trailing blur.

[0119] Exemplarily, the gray-scale value value of each pixel point in each gray-scale image is traversed, the gray-scale values of the left, right, top and bottom pixel points of any pixel point are denoted as leftv, rightv, upv and bottomv, the neighborhood contrast of the pixel point is calculated according to the formula contra += (pow(value-leftv, 2) + pow(value-rightv, 2) + pow(value-upv, 2) + pow(value-bottomv, 2)), and the neighborhood contrast mean value of the entire image is calculated. If the neighborhood contrast mean value is less than a fifth preset threshold, the overall contrast of the image is too low, and the image is blurred.

[0120] After the above image anomaly detection, the blur caused by the fact that the sample solution has not stabilized when the automatic sampling module is turned on and the image is captured is preliminarily excluded. The phenomenon of blur of the entire image caused by the fact that the platform is moved or the distance between the platform and the microscope light source is too close or too far is also reduced. This reduces the difficulty of the subsequent fixed-point focusing algorithm and improves the detection performance.

[0121] S5, determining an automatic focusing result based on the set of images to be detected and the trained fixed-point focusing model.

[0122] In a possible implementation, step S5 can optionally include:

[0123] S51, determining particle edge points in each image to be detected based on the pixel points of the image to be detected.

[0124] S52, determining an edge gradient image corresponding to each image to be detected based on the particle edge points in the image to be detected.

[0125] It should be noted that the specific implementation method of steps S51 to S52 can refer to steps S31 to S32, which will not be described here.

[0126] S53, inputting all images to be detected and corresponding edge gradient images under the same field of view into the trained fixed-point focusing model to determine a target definition level corresponding to each field of view.

[0127] As an example, all images to be detected and corresponding edge gradient images under the same field of view are input into the trained fixed-point focusing model to obtain a predicted definition level of each image to be detected, and the highest level among them is taken as the target definition level of the field of view.

[0128] S54, determining the target image corresponding to each field of view based on the set of to-be-tested images and the target sharpness level corresponding to each field of view.

[0129] As an example, if the target sharpness level of one field of view is 1 or 2, it means that all the to-be-tested images under the field of view are very blurred; if the target sharpness level of one field of view is 3 or 4, it means that all the to-be-tested images under the field of view are not clear enough in the grain edges, and have ghosting; if the target sharpness level of one field of view is 5, any one of the to-be-tested images with labels 5, 6 and 7 in the 11 to-be-tested images corresponding to the field of view is taken as the target image corresponding to the field of view; if the target sharpness level of one field of view is 6, the to-be-tested image with label 6 in the 11 to-be-tested images corresponding to the field of view is taken as the target image corresponding to the field of view. That is, if the target sharpness level output by the fixed-point focusing model is less than 5, it means that no clear image is collected under the corresponding field of view, and the target image cannot be determined.

[0130] S55, determining the automatic focusing result based on the target images.

[0131] Optionally, the target images corresponding to each field of view are taken as the automatic focusing result.

[0132] The technical scheme provided in the present application first corrects the flatness of the micro flow cell before focusing, determines the target focusing position of each field of view in the micro flow cell, avoids the situation that the traditional automatic collection system needs to collect a large number of images for each field of view to find the target focusing position under the conditions of uneven process of the groove bottom, slight inclination of the instrument or platform, and even the situation that the system directly malfunctions due to the failure to find the target focusing position; then based on the target focusing position of each field of view, a certain number of sample images under each field of view are collected and a set of sample images is constructed to train the fixed-point focusing model, which improves the precision of collecting sample images, and further improves the training effect of the fixed-point focusing model and the accuracy of the fixed-point focusing model; in the focusing process, a preset number of to-be-tested images under each field of view are collected to construct a set of to-be-tested images; based on the set of to-be-tested images and the trained fixed-point focusing model, the automatic focusing result is determined, which effectively corrects the situations of large flatness change caused by process error of the flow cell under the microscope imaging, non-constant sample flow rate caused by automatic sampling, and blurred grain images caused by platform movement, and accurately and quickly realizes real-time automatic focusing, improves the accuracy of the automatic focusing result, and further realizes accurate grain analysis.

[0133] Figure 7 An architecture schematic diagram of a fixed-point focusing model provided in an embodiment of the present application. Based on the above-mentioned embodiments, the technical scheme is further described and optimized in the present embodiment. Specifically, in the present embodiment, as shown in FIG. 2, the fixed-point focusing model comprises a first input layer, a second input layer, a hidden layer and an output layer. Figure 7As shown, the fixed-point focusing model comprises a model input module, an initial feature extraction module, a multi-scale feature interaction extraction module, a classification module, and a model output module connected in sequence.

[0134] The model input module is configured to input all target images to be tested and corresponding target edge gradient maps. The target image to be tested is an image to be tested under a target field of view, and the target field of view is any field of view in a detection area of a micro flow cell.

[0135] The initial feature extraction module is configured to perform feature extraction on each target image to be tested and the corresponding target edge gradient map to obtain an initial feature map corresponding to each target image to be tested.

[0136] The multi-scale feature interaction extraction module is configured to perform multi-scale feature interaction extraction on each initial feature map to obtain a final feature map corresponding to each target image to be tested.

[0137] The classification module is configured to determine the definition level of each target image to be tested based on each final feature map, and determine the target definition level corresponding to the target field of view based on the definition level of each target image to be tested.

[0138] The model output module is configured to output the target definition level.

[0139] Optionally, the target definition level output by the fixed-point focusing model is transmitted into a software system. When the target definition level is 5 or 6, it indicates that there is a relatively clear image in the target image to be tested. The particle shape and morphology of the particles in this field of view can be directly calculated based on the target image determined by the target definition level, and the results are displayed on the interface. Then, the focusing reasoning of the next field of view is continued. If the target definition level is less than 5, the software pops up an alarm information of “no clear image collected”.

[0140] In one possible implementation manner, as shown in Figure 8 The initial feature extraction module comprises:

[0141] A global feature extraction unit and an edge feature extraction unit are respectively connected to the model input module and arranged side by side. The global feature extraction unit is configured to extract global features of each target image to be tested to obtain a global feature map of each target image to be tested. The edge feature extraction unit is configured to extract edge features of each target edge gradient map to obtain an edge feature result map corresponding to each target edge gradient map.

[0142] The feature fusion unit is connected to the global feature extraction unit and the edge feature extraction unit, and is configured to perform feature fusion on the global feature map corresponding to each target image to be tested and the edge feature result map to obtain a temporary result map corresponding to each target image to be tested.

[0143] The attention weight extraction unit is connected with the feature fusion unit and is configured to perform attention weight extraction on each temporary result image to obtain an attention weight image corresponding to each target image to be tested.

[0144] The attention adjustment unit is connected with the global feature extraction unit, the feature fusion unit, and the attention weight extraction unit, and is configured to perform attention gate dynamic adjustment on the corresponding global feature map and the edge feature result image based on each attention weight image to obtain an initial feature map corresponding to each target image to be tested.

[0145] In a possible implementation, as shown in Figure 9 The multi-scale feature interaction extraction module includes:

[0146] The first feature map extraction unit is connected with the initial feature extraction module and is configured to perform feature extraction based on each initial feature map to obtain a first feature map corresponding to each target image to be tested.

[0147] The second feature map extraction unit is connected with the initial feature extraction module and is configured to perform feature extraction based on each initial feature map to obtain a second feature map corresponding to each target image to be tested.

[0148] The third feature map extraction unit is connected with the second feature map extraction unit and is configured to perform feature extraction based on each initial feature map to obtain a third feature map corresponding to each target image to be tested.

[0149] The first feature map splicing unit is connected with the first feature map extraction unit and the third feature map extraction unit and is configured to perform feature splicing on the first feature map and the third feature map corresponding to each target image to be tested to obtain a first feature splicing map corresponding to each target image to be tested.

[0150] The fourth feature map extraction unit is connected with the first feature map splicing unit and is configured to perform feature extraction based on each first feature splicing map to obtain a fourth feature map corresponding to each target image to be tested.

[0151] The fifth feature map extraction unit is connected with the fourth feature map extraction unit and is configured to perform feature extraction based on each first feature splicing map to obtain a fifth feature map corresponding to each target image to be tested.

[0152] The second feature map splicing unit is connected with the second feature map extraction unit and the fifth feature map extraction unit and is configured to perform feature splicing on the second feature map and the fifth feature map corresponding to each target image to be tested to obtain a second feature splicing map corresponding to each target image to be tested.

[0153] The sixth feature map extraction unit is connected with the second feature map splicing unit and is configured to perform feature extraction based on each second feature splicing map to obtain a sixth feature map corresponding to each target image to be tested.

[0154] The third feature map splicing unit is connected with the fourth feature map extraction unit and the sixth feature map extraction unit, and is configured to perform feature splicing on the fourth feature map and the sixth feature map corresponding to each target image to be tested to obtain a final feature map corresponding to each target image to be tested.

[0155] As an example, as shown in Figure 10 The point focusing model uses a lightweight network, and the model backbone structure is MobileNetV3-Large. The network is forced to pay attention to the sharpening degree of the particle edge, so an edge feature fusion mechanism is added. At the same time, in order to further enhance the focusing robustness of the high-level network in complex scenes such as motion blur, sample liquid flow rate, particle stratification, and particle color transformation, a three times multi-scale feature interactive extraction is added. The specific description is as follows:

[0156] 1. The model input module inputs multiple target images to be tested image_input with a size of 224×224×3. The first Conv2D layer and the first three inverted residual blocks (Inverted Residual Block) of the original MobileNetV3-Large are retained, and the Bottleneck1, Bottleneck2, and Bottleneck3 layer structures are basically unchanged. The first Conv2D layer, Bottleneck1 layer, Bottleneck2 layer, and Bottleneck3 layer can be used as a global feature extraction unit.

[0157] 2. The model input module inputs the target edge gradient image edge_image corresponding to each target image to be tested image_input. The edge probability extraction layer is used to perform 1×1×3 convolution and sigmoid activation on the result, and then a first convolution layer with a convolution kernel size of 1×1×24 and a step of 4 is used for channel adjustment to obtain an edge feature result image. The function of this result image is to enhance the edge sensitivity of the network, and the size is 56×56×24. The edge probability extraction layer and the first convolution layer can be used as an edge feature extraction unit.

[0158] 3. The feature fusion unit is connected with the Bottleneck2 layer and the first convolution layer, and the features output by the Bottleneck2 layer are multiplied (Element-wise Multiplication) with the edge feature result image edge_features element by element to obtain a temporary result image BL2_edge with the same size.

[0159] 4、Attention weight extraction unit uses 1x1x24 convolution and sigmoid activation function on temporary result image BL2_edge, realizes dimension reduction and obtains attention weight image G, realizes dimension reduction and obtains attention weight image G, size is 56x56x1; value range is (0, 1). Attention adjustment unit adopts attention gate mechanism (Attention Gate) dynamic adjustment, realizes the fusion of temporary result image BL2_edge and bottleneck 3 layer output result, obtains initial feature image BL3_edge. Use formula BL3_edge=Gxedge_features+(1-G)x Bottleneck3 forces the network to pay more attention to the particle edge information. Figure Three

[0160] 5、The initial feature image BL3_edge with the size of 56x56x24 is sequentially passed through the second convolution layer (the convolution kernel size is 1x1x24, the channel number is 40), the first down-sampling layer and the second down-sampling layer, to obtain the first feature image C3 with the size of 14x14x40. The second convolution layer, the first down-sampling layer and the second down-sampling layer can be used as the first feature image extraction unit.

[0161] 6、The fourth, fifth and sixth inverted residual blocks Bottleneck4 layer, Bottleneck5 layer and Bottleneck6 layer of the original MobileNetV3-Large are kept unchanged. The feature image with the size of 28x28x40 output by the Bottleneck6 layer is passed through the third convolution layer (the convolution kernel size is 1x1x40, the channel number is 56) to obtain a feature image with the size of 28x28x56, and then the third down-sampling layer is used to down-sample once to obtain the second feature image C6. The Bottleneck4 layer, the Bottleneck5 layer, the Bottleneck6 layer, the third convolution layer and the third down-sampling layer can be used as the second feature image extraction unit.

[0162] 7、The seventh, eighth and ninth inverted residual blocks Bottleneck7 layer, Bottleneck8 layer and Bottleneck9 layer of the original MobileNetV3-Large are kept unchanged. The feature image with the size of 14x14x80 output by the Bottleneck9 layer is passed through the fourth convolution layer (the convolution kernel size is 1x1x80, the channel number is 40) to obtain the third feature image C9 with the size of 14x14x40. The Bottleneck7 layer, the Bottleneck8 layer, the Bottleneck9 layer and the fourth convolution layer can be used as the third feature image extraction unit.

[0163] 8、The first feature map splicing unit splices the first feature map C3 and the third feature map C9 directly, realizing the fusion of shallow detail features and middle layer information. The first feature splicing map with a size of 14x14x80 is obtained, and the feature diversity is reserved.

[0164] 9、The 10th, 11th and 12th Bottleneck blocks Bottleneck10 layer, Bottleneck11 layer and Bottleneck12 layer of the original MobileNetV3-Large are reserved unchanged. The output of the Bottleneck10 layer is subjected to channel adjustment and down-sampling by a fifth convolutional layer and a fourth down-sampling layer respectively, to obtain a fourth feature map C10. The output of the Bottleneck12 layer is subjected to channel adjustment by a sixth convolutional layer using 1x1 convolution, to obtain a fifth feature map C12 with a size of 14x14x56. The Bottleneck10 layer, the fifth convolutional layer and the fourth down-sampling layer can be taken as a fourth feature map extraction unit, and the Bottleneck11 layer, the Bottleneck12 layer and the sixth convolutional layer can be taken as a fifth feature map extraction unit.

[0165] 10、The second feature image C6 and the high-layer extracted features of the fifth feature C12 are spliced by the second feature map splicing unit, to obtain a second feature splicing map with a size of 14x14x112.

[0166] 11、The 13th, 14th and 15th Bottleneck blocks Bottleneck13 layer, Bottleneck14 layer and Bottleneck15 layer of the original MobileNetV3-Large are maintained unchanged, and the output of the Bottleneck15 layer is subjected to convolution operation by a seventh convolutional layer, to obtain a sixth feature map C15 after compression of the dimension. The Bottleneck13 layer, the Bottleneck14 layer, the Bottleneck15 layer and the seventh convolutional layer can be taken as a sixth feature map extraction unit.

[0167] 12、The fourth feature map C10 and the high-layer extracted features of the sixth feature map C15 are spliced by the third feature map splicing unit, to obtain a final feature map with a size of 7x7x160. Thus, the first feature splicing map, the second feature splicing map and the final feature map are obtained. ​ The cross-layer interaction splicing feature enables the high-level network to learn different low-level network features, and avoids the problem that the particle edge information is reduced or disappears as the network level increases.

[0168] 13、Finally, the last four layers of the MobileNetV3-Large original structure are reserved as the classification module, which are the second Conv2D layer, the pooling layer, the output processing layer (including a Conv2D and an Activation), and the third Conv2D layer. However, in the third Conv2D layer, the number of categories is adjusted from the original 1280 to 6, which is suitable for the fixed-point focusing model.

[0169] The fixed-point focusing model provided by the embodiment is a network model for multi-scale feature fusion and enhanced edge perception. Compared with traditional focusing models, the fixed-point focusing model is more suitable for the diversity of particle types and the variability of particle colors, and still has good focusing effect in the case of irregular particle morphology, large particle overlap degree, and layering.

[0170] In another aspect, the present application also provides a computer storage medium storing executable program codes; the executable program codes are used to execute any of the above automatic focusing methods based on micro flow cells.

[0171] In another aspect, the present application also provides an electronic device including a memory and a processor; the memory stores program codes executable by the processor; the program codes are used to execute any of the above automatic focusing methods based on micro flow cells.

[0172] For example, the program codes can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the program codes in the electronic device.

[0173] The electronic device can be a desktop computer, a notebook, a palm computer, a cloud server, and other computing devices. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the electronic device can also include input / output devices, network access devices, buses, etc.

[0174] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0175] The memory can be an internal storage unit of the electronic device, such as a hard disk or a memory. The memory can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory can include both the internal storage unit and the external storage device of the electronic device. The memory is used to store the program code and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0176] The computer storage medium and the electronic device based on the above method are created, and the technical effects and advantages are not repeated here. The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0177] The above-described embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the patent. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of variations and improvements can be made, which are within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. An autofocusing method based on a micro-flow cell, characterized in that, The method includes: The flatness of the micro-flow cell is corrected to determine the target focus position of each field of view in the micro-flow cell; Based on the target focus position of each field of view, a number of sample images under each field of view are acquired and a sample image set is constructed. Based on the sample image set, a fixed-point focusing model is trained, including: determining grain edge points in each sample image based on the pixels in each sample image in the sample image set; determining the edge gradient map corresponding to each sample image based on the grain edge points in each sample image; determining the sharpness score of the sample image corresponding to each edge gradient map based on the pixel values ​​of each edge gradient map; determining the sharpness level of each sample image and constructing a label data set based on the sharpness score of each sample image and a preset mapping relationship, wherein the preset mapping relationship is used to indicate the mapping relationship between the sharpness level and the sharpness score range; and training the fixed-point focusing model based on the sample image set and the label data set. Collect a preset number of images to be tested under each field of view to construct a set of images to be tested; Based on the set of images to be tested and the trained fixed-point focusing model, the autofocus result is determined, including: determining the grain edge points in each of the images to be tested based on the pixels in each image to be tested; determining the edge gradient map corresponding to each image to be tested based on the grain edge points in each image to be tested; inputting all images to be tested under the same field of view and the corresponding edge gradient maps into the trained fixed-point focusing model to determine the target sharpness level corresponding to each field of view; determining the target image corresponding to each field of view based on the set of images to be tested and the target sharpness level corresponding to each field of view; and determining the autofocus result based on each target image.

2. The method according to claim 1, characterized in that, The step of performing flatness correction on the micro-flow cell to determine the target focus position in each field of view within the micro-flow cell includes: Based on the camera parameters, the detection area of ​​the micro-flow cell is uniformly divided into several fields of view; Several original images were captured under each of the aforementioned fields of view; Based on the sharpness of each of the original images, the target focus position of each of the fields of view is determined, and the target focus position includes the coordinates of the target focus point.

3. The method according to claim 1, characterized in that, The fixed-point focusing model includes a model input module, an initial feature extraction module, a multi-scale feature interaction extraction module, a classification module, and a model output module connected in sequence. The model input module is used to input all target images to be tested and the corresponding target edge gradient maps. The target images to be tested are the images to be tested under the target field of view, and the target field of view is any of the aforementioned fields of view. The initial feature extraction module is used to extract features from each of the target images to be tested and the corresponding target edge gradient maps to obtain an initial feature map corresponding to each of the target images to be tested. The multi-scale feature interaction extraction module is used to perform multi-scale feature interaction extraction on each of the initial feature maps to obtain the final feature map corresponding to each of the target images to be tested. The classification module is used to determine the sharpness level of each target image under test based on each final feature map, and to determine the target sharpness level corresponding to the target field of view based on the sharpness level of each target image under test. The model output module is used to output the target sharpness level.

4. The method according to claim 3, characterized in that, The initial feature extraction module includes: A global feature extraction unit and an edge feature extraction unit are respectively connected to and arranged in parallel with the model input module. The global feature extraction unit is used to extract the global features of each target image to obtain a global feature map ... The feature fusion unit, connected to the global feature extraction unit and the edge feature extraction unit, is used to fuse the global feature map and the edge feature result map corresponding to each of the target test images to obtain a temporary result map corresponding to each of the target test images. An attention weight extraction unit, connected to the feature fusion unit, is used to extract attention weights from each of the temporary result maps to obtain attention weight maps corresponding to each of the target test images. The attention adjustment unit, connected to the global feature extraction unit, the feature fusion unit, and the attention weight extraction unit, is used to dynamically adjust the attention gating of the corresponding global feature map and edge feature result map based on each attention weight map to obtain the initial feature map corresponding to each target image to be tested.

5. The method according to claim 4, characterized in that, The multi-scale feature interaction extraction module includes: The first feature map extraction unit is connected to the initial feature extraction module and is used to extract features based on each of the initial feature maps to obtain the first feature map corresponding to each of the target images to be tested. The second feature map extraction unit is connected to the initial feature extraction module and is used to extract features based on each of the initial feature maps to obtain the second feature map corresponding to each of the target images to be tested. The third feature map extraction unit is connected to the second feature map extraction unit and is used to extract features based on each of the initial feature maps to obtain the third feature map corresponding to each of the target images to be tested. The first feature map stitching unit is connected to the first feature map extraction unit and the third feature map extraction unit, and is used to stitch together the first feature map and the third feature map corresponding to each of the target images to be tested, so as to obtain the first feature stitching map corresponding to each of the target images to be tested. The fourth feature map extraction unit is connected to the first feature map splicing unit and is used to extract features based on each of the first feature splicing maps to obtain the fourth feature map corresponding to each of the target images to be tested. The fifth feature map extraction unit is connected to the fourth feature map extraction unit and is used to extract features based on each of the first feature stitched images to obtain the fifth feature map corresponding to each of the target images to be tested. The second feature map stitching unit is connected to the second feature map extraction unit and the fifth feature map extraction unit, and is used to stitch the second feature map and the fifth feature map corresponding to each of the target images to be tested, so as to obtain the second feature stitching map corresponding to each of the target images to be tested. The sixth feature map extraction unit is connected to the second feature map splicing unit and is used to extract features based on each of the second feature splicing maps to obtain the sixth feature map corresponding to each of the target images to be tested; The third feature map stitching unit is connected to the fourth feature map extraction unit and the sixth feature map extraction unit, and is used to stitch together the fourth feature map and the sixth feature map corresponding to each of the target images to obtain the final feature map corresponding to each of the target images to be tested.

6. The method according to any one of claims 1 to 5, characterized in that, The process involves acquiring a preset number of images to be tested within each field of view to construct a set of images to be tested, including: Convert each of the images to be tested into a grayscale image; Image anomaly detection is performed based on each of the grayscale images, and candidate images to be tested are selected from each of the images to be tested; Based on each of the candidate images to be tested, the set of images to be tested is constructed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images