Image registration-based three-level lymphatic structure recognition model construction and recognition method
By using image registration and the YOLO v5 model, the challenge of automating the identification of three-level lymphatic structures in HE images was solved, achieving sub-pixel level accurate detection and supporting clinical diagnosis and treatment strategy formulation.
Patent Information
- Application Number
- CN202410420799.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-09
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2044-04-09
AI Technical Summary
In existing technologies, it is difficult to identify tertiary lymphoid structures in HE images, and traditional methods are costly and difficult to achieve automated and efficient sub-pixel level registration.
An image registration-based method is adopted, using edge detection algorithms for coarse and fine registration, combined with the YOLO v5 target detection model, to automatically identify the tertiary lymphoid structures in HE images.
It enables automatic and accurate identification of tertiary lymphoid structures in HE images, reducing manual intervention, improving identification efficiency and accuracy, and providing a reliable reference for clinical diagnosis.
Smart Images
Figure CN118279273B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The embodiment of the present application relates to the technical field of medical image processing, in particular to a three-level lymph structure recognition model construction and recognition method based on image registration. BACKGROUND
[0002] The three-level lymph structure is closely related to the treatment plan of patients and the treatment prognosis effect of patients in immunology. At present, it is difficult to recognize the three-level lymph structure in the HE image, and the gold standard is obtained by manual recognition of the immunofluorescence image. The immunofluorescence imaging itself has high processing cost, while the HE imaging has low cost and is easy to obtain. Therefore, an image processing method is needed to recognize the three-level lymph structure in the HE image. SUMMARY
[0003] The embodiment of the present application provides a three-level lymph structure recognition model construction and recognition method based on image registration to solve the above technical problems.
[0004] In a first aspect, the embodiment of the present application provides a three-level lymph structure recognition model construction method based on image registration, comprising:
[0005] Obtaining a plurality of HE images and corresponding fluorescence images including three-level lymph structures;
[0006] Detecting the edges of each HE image and fluorescence image by using an edge detection algorithm, and coarsely registering each HE image and the corresponding fluorescence image according to the detected edge information;
[0007] Dividing each coarsely registered HE image and fluorescence image into a plurality of image blocks respectively, finely registering each HE image block and the corresponding fluorescence image block, and solving the fine registration matrix of the HE image according to the fine registration matrices of each HE image block in the same HE image;
[0008] Cascading the coarse registration matrix and the fine registration matrix of each HE image into a total registration matrix, and registering the three-level lymph structure mask of each fluorescence image according to each total registration matrix to obtain the three-level lymph structure mask of each HE image;
[0009] Taking each HE image as input and taking the three-level lymph structure mask of each HE image as output, training a target detection model based on a neural network, and taking the trained target detection model as a three-level lymph structure recognition model.
[0010] In a second aspect, the embodiment of the present application provides a three-level lymph structure recognition method based on image registration, comprising:
[0011] Obtaining an HE image to be recognized;
[0012] The HE image to be recognized is processed by using the three-level lymph structure recognition model constructed by the method to obtain a three-level lymph structure mask of the HE image.
[0013] According to the three-level lymph structure mask and the HE image, an HE image of the three-level lymph structure is generated.
[0014] In a third aspect, an electronic device is provided, and the electronic device includes:
[0015] One or more processors;
[0016] Memory for storing one or more programs,
[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the three-level lymph structure recognition model construction method based on image registration or the three-level lymph structure recognition method based on image registration according to any embodiment.
[0018] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the program is executed by a processor to implement the three-level lymph structure recognition model construction method based on image registration or the three-level lymph structure recognition method based on image registration according to any embodiment.
[0019] The embodiments of the present application provide a three-level lymph structure recognition model construction and recognition method based on image registration, which is mainly used for sub-pixel level registration of pathological HE images and immunofluorescence images and automatic detection of three-level lymph structures, and provides algorithm and software basis for clinical diagnosis of doctors. Specifically, first, a HE image and an immunofluorescence image pairing database is constructed, then the HE and immunofluorescence images are automatically registered at a sub-pixel level from coarse to fine, and a three-level lymph structure mask in the HE image is obtained according to a registration matrix. A target detection model trained based on the mask and the HE image can automatically recognize the three-level lymph structure in the HE image, the whole process does not need manual intervention, and the efficiency and accuracy of pathological structure recognition are guaranteed. The features such as the morphology and quantity of the three-level lymph structure detected by the pathological HE image can provide reliable reference for clinical treatment effect prediction, treatment strategy formulation, prognosis effect prediction, and the like of patients.
[0020] More specifically, in order to solve the problem of time-consuming and laborious manual registration or doctor's naked eye observation for traditional fluorescence images and HE images, the embodiment proposes an automatic registration method from coarse to fine, which can realize relatively fine registration and is sufficient to cope with sub-pixel level pathological structure recognition of tertiary lymphoid structures. At the same time, according to the automatic registration result, a YOLO v5 target detection model is established to identify the tertiary lymphoid structure region from the HE image, avoiding the subjective differences and labor costs caused by manually outlining the region of interest, registration and manually extracting parameter features. BRIEF DESCRIPTION OF DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the description of the embodiments or the prior art. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.
[0022] Figure 1 is a flowchart of a tertiary lymphoid structure recognition model construction method based on image registration provided by an embodiment of the present application;
[0023] Figure 2 is a schematic diagram of HE and immunofluorescence paired database samples of the present application;
[0024] Figure 3 is a flowchart of another tertiary lymphoid structure recognition model construction method based on image registration provided by an embodiment of the present application;
[0025] Figure 4 is a structural schematic diagram of a YOLO v5 model provided by an embodiment of the present application;
[0026] Figure 5 is a flowchart of a tertiary lymphoid structure recognition method based on image registration provided by an embodiment of the present application;
[0027] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0029] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0030] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.
[0031] As described in the background, there is an urgent need for an image processing method to automatically identify the tertiary lymphoid structure in the HE image. There are several difficulties in automatically identifying the tertiary lymphoid structure from the HE image: 1) The specific position of the tertiary lymphoid structure in the HE image needs to be manually compared with the immunofluorescence image and the HE image. Even if it is manually compared, it is not possible to obtain a more detailed tertiary lymphoid structure contour on the HE image according to the immunofluorescence image. 2) The HE image and the corresponding immunofluorescence image are not naturally registered. Due to the washing of the dye and human operation, the image may be offset and dislocated. Due to the high pixel resolution of pathological images, it is very difficult to manually register, and there is a subjective difference, which is difficult to achieve the desired effect. Therefore, an automatic image registration method is needed, which can realize the registration of HE and immunofluorescence images of different modalities without relying on human guidance, so as to realize accurate detection of the tertiary lymphoid structure on the HE image.
[0032] In view of the above difficulties, Figure 1 A flowchart of a tertiary lymphoid structure identification model construction method based on image registration provided by an embodiment of the present application. The method is executed by an electronic device, and specifically includes the following steps:
[0033] S110, acquiring a plurality of HE images and corresponding fluorescence images including tertiary lymphoid structures.
[0034] The step constructs a paired database of HE images and immunofluorescence images as a training set of the three-level lymphatic structure recognition model. In a specific embodiment, the pathological sections of lung cancer patients can be collected first, and then sequentially subjected to HE staining and immunofluorescence treatment, with the staining agent being washed away in between. The integrity of the sections and the small positional offset are ensured as much as possible during the treatment, so as to facilitate the registration and image processing in subsequent operations. By treating a number of patients through the above operations, a paired database including HE images and immunofluorescence images can be obtained, and each HE image and its corresponding fluorescence image are as shown in FIG. 8. Figure 2
[0035] In S120, an edge detection algorithm is used to detect the edges of each HE image and fluorescence image, and the edges are used to coarsely register each HE image and the corresponding fluorescence image.
[0036] In this step, the edges of the HE images and immunofluorescence images are detected based on the established database using an edge detection algorithm, and the edges are used to perform a first registration between each image pair, which is referred to as coarse registration at the image level. In a specific embodiment, the coarse registration can include the following steps:
[0037] Step one, any HE image and the corresponding fluorescence image are processed into two thumbnail grayscale images. Since the pixels of the HE image and the fluorescence image are very large, in order to reduce the data processing burden, the two images are first reduced to grayscale images of a certain size.
[0038] Step two, an edge detection algorithm is used to process each thumbnail grayscale image to obtain the edge map of each thumbnail grayscale image. Optionally, the edge detection algorithm is a Sobel operator edge detection algorithm. For a given two-dimensional grayscale image I, the Sobel operator calculates the gradients in the horizontal and vertical directions through convolution operation, which are represented by G x and G y respectively. The combination of the two gradients can obtain the edge strength and direction, thereby completing the edge detection of the image. Specifically, the convolution kernel of the Sobel operator is as follows:
[0039]
[0040]
[0041] At each pixel point in the image, the horizontal and vertical gradients at the point are represented by I x and I y respectively, which are calculated as follows:
[0042] I x = I * G x
[0043] Iy = I * G y
[0044] wherein * represents a convolution operation. Then, the edge intensity E can be calculated as:
[0045]
[0046] The gradient direction θ can be calculated by the following formula:
[0047]
[0048] In this embodiment, the edge maps of the HE pathology thumbnail and the immunofluorescence pathology thumbnail are denoted as o and g, respectively.
[0049] Step three, performing registration on the two edge maps by using a rigid registration algorithm to obtain a coarse registration matrix. Optionally, first, performing registration on the two edge maps by using a rigid registration algorithm to obtain a coarse registration matrix of the two edge maps; and then, magnifying the coarse registration matrix of the two edge maps to the size of the HE image to obtain a coarse registration matrix of the HE image. In a specific embodiment, the rigid registration algorithm of the AntsPy library can be used to perform registration on o and g to obtain a coarse registration matrix denoted as M; the registered HE thumbnail O' = o@M, wherein @ represents a registration operation. The HE pathology original image is denoted as O, and according to the scaling factor a of O and O', the registered HE original image is O" = O@(M# a), wherein # represents a scaling processing of the registration matrix, and M# a is the coarse registration matrix of the HE image.
[0050] S130, dividing each coarse-registered HE image and fluorescence image into a plurality of image blocks, performing fine registration on each HE image block and the corresponding fluorescence image block, and solving a fine registration matrix of the HE image according to the fine registration matrices of each HE image block in the same HE image.
[0051] This step performs fine registration at the image block level (or sub-pixel level), which is used to extract more fine image information. Optionally, after obtaining the fine registration matrix of each image block, first, average the fine registration matrices of each HE image block in the same HE image; and then, magnify the averaged matrix to the size of the HE image as the fine registration matrix of the HE image.
[0052] In a specific embodiment, the registered HE pathology original image O" and the immunofluorescence original image G can be first divided into n small image blocks p O and p G , which are in one-to-one correspondence. Then, performing registration on each pair of small image blocks to obtain n fine registration matrices M' of the image blocks; and then, averaging the n matrices to obtain Finally, the scaling factor β is obtained according to the small image block and the HE pathological original image, and then the HE image after the second registration is O'''=O''(M''#β), wherein M''#β is the fine registration matrix of the HE image.
[0053] In S140, the coarse registration matrix and the fine registration matrix of each HE image are cascaded into a total registration matrix, and the tertiary lymphatic structure mask of each fluorescent image is registered according to the total registration matrix, so as to obtain the tertiary lymphatic structure mask of each HE image.
[0054] As can be seen from S120 and S130, O'''=O''(M''#β)=O(M#α)(M''#β), wherein M#α is the coarse registration matrix of the HE image, and M''#β is the fine registration matrix of the HE image, and the registration operation of (M#α)(M''#β) corresponds to the total registration matrix. The registration matrix can automatically register the HE image and the immunofluorescence image from coarse to fine sub-pixel level without manual intervention, thereby ensuring the accuracy of the registration. More specifically, the coarse refers to rigid registration at the full slice thumbnail level after image processing, and the fine refers to rigid registration of the image at the small image block level after rigid registration, and the registration transformation matrix of each small image block is averaged to obtain the final registration image.
[0055] Meanwhile, the tertiary lymphatic structure contour manually outlined by the doctor on the fluorescent image can be used to obtain the tertiary lymphatic structure mask of the fluorescent image. Then, the tertiary lymphatic structure mask is registered according to the total registration matrix to obtain the mask of the tertiary lymphatic structure in the HE image, which represents the position and contour of the tertiary lymphatic structure in the HE image.
[0056] In S150, the HE image is taken as the input, and the tertiary lymphatic structure mask of the HE image is taken as the output, so as to train the target detection model based on the neural network, and the trained target detection model is taken as the tertiary lymphatic structure recognition model.
[0057] In this step, the position and contour of the tertiary lymphatic structure in the HE image obtained in S140 are taken as the label, and the target detection model based on the neural network is trained, so that when any HE image to be recognized is input into the trained target detection model, the model can output the position and contour (i.e., the mask) of the tertiary lymphatic structure in the HE image, and the whole process is as shown in Figure 3 Optionally, the target detection model is a YOLO v5 model, and the trained YOLO v5 model can automatically detect the tertiary lymphatic structure in the HE structure, thereby ensuring efficient and accurate recognition of the pathological structure.
[0058] In a specific embodiment, the specific structure of the YOLO v5 model is as shown in Figure 4As shown, the full slice data can be cut into smaller image data blocks during training, and the data blocks of the HE image are inputted, and the tertiary lymphoid structure mask in the data blocks is outputted to train the target detection model. The blocked data facilitates the tertiary lymphoid structure to be recognized by the target detection network, and the full slice detection result can be obtained after the blocked detection result is summarized.
[0059] Based on the tertiary lymphoid structure recognition model, Figure 5 is a flowchart of a tertiary lymphoid structure recognition method based on image registration provided by an embodiment of the present application. As shown, Figure 5 the method comprises:
[0060] S210, obtaining an HE image to be recognized.
[0061] The image includes tertiary lymphoid structures and other tissues, and the tertiary lymphoid structure in the image will be accurately recognized in this embodiment.
[0062] S220, using the tertiary lymphoid structure recognition model constructed by the above method to process the HE image to be recognized, and obtaining a tertiary lymphoid structure mask in the HE image.
[0063] If the training stage is to train the full slice data cut into smaller image data blocks, the HE image is also cut into smaller data blocks in this step, which are respectively sent to the tertiary lymphoid structure recognition model for recognition, and the full slice recognition result can be obtained after the blocked recognition result is summarized.
[0064] S230, obtaining a HE image of the tertiary lymphoid structure according to the tertiary lymphoid structure mask and the HE image.
[0065] The finally obtained HE image has removed other tissues in the original image, and can more clearly display the morphology of the tertiary lymphoid structure.
[0066] In summary, the embodiment of the present application discloses a three-level lymph structure recognition model construction and recognition method based on image registration, which is mainly used for sub-pixel level registration of pathological HE images and immunofluorescence images and automatic detection of three-level lymph structures, and provides algorithm and software basis for clinical diagnosis of doctors. Specifically, first, an HE image and an immunofluorescence image pairing database is constructed, then the HE and immunofluorescence images are automatically registered from coarse to fine at the sub-pixel level, and the three-level lymph structure mask in the HE image is obtained according to the registration matrix. The target detection model trained based on the mask and the HE image can automatically identify the three-level lymph structure in the HE image. The whole process does not need manual intervention, and the efficiency and accuracy of pathological structure recognition are guaranteed. The three-level lymph structure morphology, quantity and other characteristics detected by the pathological HE image can provide reliable reference for clinical treatment effect prediction, treatment strategy formulation and prognosis effect prediction of patients.
[0067] More specifically, to solve the problem of time-consuming and laborious manual registration of traditional fluorescence images and HE images or visual observation by doctors, the embodiment proposes a coarse-to-fine automatic registration method, which can realize relatively fine registration and is sufficient to cope with sub-pixel level pathological structure recognition such as three-level lymph structure. At the same time, according to the automatic registration result, a YOLO v5 target detection model is established to identify the three-level lymph structure region from the HE image, avoiding subjective differences and human labor caused by manually outlining the region of interest, registration and manual extraction of parameter features.
[0068] Figure 6 A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 1, Figure 6 The device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more, Figure 6 and the processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected through a bus or other means, Figure 6 and the connection through the bus is taken as an example.
[0069] The memory 61 is a kind of computer readable storage medium, which can be used to store software programs, computer executable programs and modules, such as the program instructions / modules of the three-level lymph structure recognition model construction method based on image registration or the three-level lymph structure recognition method based on image registration in the embodiment of the present application. The processor 60 executes the software programs, instructions and modules stored in the memory 61, thereby performing various functional applications and data processing of the device, i.e. implementing the three-level lymph structure recognition model construction method based on image registration or the three-level lymph structure recognition method based on image registration.
[0070] The memory 61 can include a program storage area that can store an operating system, application programs required for at least one function, and a data storage area that can store data created based on a use of the terminal, among others. In addition, the memory 61 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-volatile solid state memory device. In some examples, the memory 61 can further include a memory disposed remotely from the processor 60, which can be connected to the device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0071] The input device 62 can be used to receive input digital or character information, and to generate key signal inputs related to user settings of the device and function controls. The output device 63 can include a display device such as a display screen.
[0072] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image registration based three-level lymphatic structure recognition model construction method or the image registration based three-level lymphatic structure recognition method.
[0073] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, device or apparatus.
[0074] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program code. Program code embodied on a computer readable signal medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, R.F, etc., or any suitable combination of the foregoing.
[0075] Computer readable program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, R.F, etc., or any suitable combination of the foregoing.
[0076] Computer program code for carrying out operations for aspects of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). These implementations can provide for a great deal of flexibility, portability, and adaptability as well as the advantage that a new version of the program code can be deployed quickly as part of an upgrade procedure.
[0077] Finally, it should be noted that the above-mentioned embodiments are merely intended for describing and illustrating, not limiting the technical solutions of the present application; even though the technical solutions of the present application have been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above-mentioned embodiments, or equivalently replace some or all of the technical features thereof; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present application.
Claims
1. A method for constructing a three-level lymphatic structure recognition model based on image registration, characterized in that, include: Multiple HE images and corresponding fluorescence images, including tertiary lymphoid structures, were acquired. An edge detection algorithm is used to detect the edges of each HE image and fluorescence image, and coarse registration is performed on each HE image and the corresponding fluorescence image based on the detected edge information; Each coarsely registered HE image and fluorescence image is divided into multiple image blocks. Fine registration is performed on each HE image block and its corresponding fluorescence image block. The fine registration matrix of the HE image is then solved based on the fine registration matrix of each HE image block in the same HE image. The coarse and fine registration matrices of each HE image are concatenated into a total registration matrix, and the tertiary lymphoid structure mask of each fluorescence image is registered according to the total registration matrix to obtain the tertiary lymphoid structure mask of each HE image. Using each HE image as input and the three-level lymphoid structure mask of each HE image as output, a neural network-based target detection model is trained, and the trained target detection model is used as a three-level lymphoid structure recognition model.
2. The method according to claim 1, characterized in that, The step involves using an edge detection algorithm to detect the edges of each HE image and fluorescence image, and performing coarse registration between each HE image and its corresponding fluorescence image based on the detected edge information. Each HE image and its corresponding fluorescence image are processed into two thumbnail grayscale images; Each thumbnail grayscale image is processed using an edge detection algorithm to obtain an edge map. The two edge maps are registered using a rigid registration algorithm to obtain a coarse registration matrix.
3. The method according to claim 2, characterized in that, The method of registering the two edge maps using a rigid registration algorithm to obtain a coarse registration matrix includes: The two edge maps are registered using a rigid registration algorithm to obtain a coarse registration matrix for the two edge maps; The coarse registration matrix of the two edge maps is enlarged to the size of the HE image to obtain the coarse registration matrix of the HE image.
4. The method according to claim 1, characterized in that, The step of solving the fine registration matrix of the HE image based on the fine registration matrix of each HE image block in the same HE image includes: Averaging of the fine registration matrices for each HE image patch within the same HE image; The averaged matrix is enlarged to the size of the HE image to serve as the fine registration matrix for the HE image.
5. The method according to claim 1, characterized in that, Before registering the third-order lymphoid structure mask of each fluorescence image according to each overall registration matrix, the method further includes: The tertiary lymphoid structure mask of the fluorescence image is obtained based on the manually drawn outline of the tertiary lymphoid structure in any fluorescence image.
6. The method according to claim 1, characterized in that, The target detection model is the YOLO v5 model.
7. The method according to claim 1, characterized in that, The edge detection algorithm is the Sobel operator edge detection algorithm.
8. A method for identifying three-level lymphatic structures based on image registration, characterized in that, include: Obtain the HE image to be identified; Using the three-level lymphatic structure recognition model constructed by any one of the methods described in claims 1-7, the HE image to be identified is processed to obtain the three-level lymphatic structure mask of the HE image; Based on the three-level lymphoid structure mask and the HE image, an HE image of the three-level lymphoid structure is generated.
9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image registration-based three-level lymphatic structure recognition model construction method according to any one of claims 1-7, or the image registration-based three-level lymphatic structure recognition method according to claim 8.
10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the method for constructing a three-level lymphatic structure recognition model based on image registration as described in any one of claims 1-7, or the method for recognizing a three-level lymphatic structure based on image registration as described in claim 8.