Multiple fluorescence immunohistochemistry image recognition method and related device

Abstract

The multiple fluorescent immunohistochemical image recognition method provided by the embodiment of the present invention includes: obtaining a multiple fluorescent immunohistochemical image to be detected, wherein the resolution of the multiple fluorescent immunohistochemical image to be detected is a first resolution; using a pre-trained multiple fluorescent immunohistochemical image recognition model to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected, and obtaining each tertiary lymphatic structure of the multiple fluorescent immunohistochemical image to be detected, wherein the resolution of the multiple fluorescent immunohistochemical image used for training the multiple fluorescent immunohistochemical image recognition model is equal to the first resolution. The tertiary lymphatic structures in the multiple fluorescent immunohistochemical image to be detected are quickly identified by using the pre-trained multiple fluorescent immunohistochemical image recognition model. It can be seen that the multiple fluorescent immunohistochemical image recognition method provided by the embodiment of the present application can improve the recognition efficiency of tertiary lymphatic structures in multiple fluorescent immunohistochemical images.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of medical information technology, and in particular to a multiplex fluorescence immunohistochemistry image recognition method and related devices. Background Art

[0002] Pathology image recognition technology is crucial for clinical diagnosis and prognosis. However, traditional pathology image recognition technology, based on hematoxylin and eosin (HE)-stained pathology images, cannot distinguish between different lymphocyte subtypes and, consequently, cannot identify specific pathological structures, such as tertiary lymphoid structures (TLS).

[0003] TLS refers to ectopic lymphoid organs that form in non-lymphoid organs during disease states. TLS are typically induced in tissues experiencing chronic inflammation. In most tumors, TLS are positively correlated with a favorable prognosis. For example, in breast cancer, colorectal cancer (CRC), and lung cancer, a greater number of TLS is associated with a better prognosis. Therefore, the efficient and accurate identification of TLS is crucial for patient prognosis.

[0004] Since HE staining cannot distinguish different types of lymphocyte subtypes well, the identification of TLS is very inconvenient. Currently, the identification and interpretation of TLS mainly rely on manual work, which results in low TLS identification efficiency and accuracy due to subjective differences in the identification and judgment results of TLS.

[0005] Therefore, how to improve the recognition efficiency of tertiary lymphatic structures in multiple fluorescence immunohistochemistry images has become a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The technical problem solved by the embodiments of the present invention is how to improve the recognition efficiency of tertiary lymphatic structures in multiple fluorescent immunohistochemical images.

[0007] To solve the above problems, an embodiment of the present invention provides a multiplex fluorescence immunohistochemistry image recognition method, comprising:

[0008] Acquiring a multiplex fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemistry image to be detected is a first resolution;

[0009] A pre-trained multiple fluorescent immunohistochemistry image recognition model is used to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemistry image to be detected to obtain recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition model is equal to the first resolution.

[0010] Optionally, the multiplex fluorescent immunohistochemistry image recognition method further includes:

[0011] performing image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks;

[0012] The step of using a pre-trained multiple fluorescent immunohistochemical image recognition model to perform three-level lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected to obtain the recognition results of each three-level lymphatic structure of the multiple fluorescent immunohistochemical image to be detected includes:

[0013] Using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain each three-level lymphatic structure recognition result of each of the multiple fluorescence immunohistochemistry image blocks;

[0014] The tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemical image to be detected is obtained according to each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemical image block.

[0015] Optionally, after the step of obtaining the identification results of each tertiary lymphatic structure of each of the multiple fluorescent immunohistochemistry image blocks, the step further includes:

[0016] The area of ​​each of the tertiary lymphatic structure recognition results is filtered according to a preset area threshold of the tertiary lymphatic structure, and the tertiary lymphatic structure recognition results with area values ​​less than the area threshold are removed to obtain the area true positive tertiary lymphatic structure recognition results.

[0017] Optionally, the multiplex fluorescent immunohistochemistry image recognition method further includes:

[0018] Acquiring a multiple fluorescent immunohistochemical image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemical image to be detected, wherein the second resolution is greater than the first resolution, and the multiple fluorescent immunohistochemical image to be detected at the second resolution includes identification results of each of the identified tertiary lymphoid structures;

[0019] After the step of obtaining the identification results of each tertiary lymphatic structure in each of the multiple fluorescent immunohistochemistry image blocks, the following steps are performed:

[0020] Determining the corresponding second-resolution third-level lymphatic structure recognition results according to the positions of each of the third-level lymphatic structure recognition results obtained at the first resolution in the second-resolution multiple fluorescence immunohistochemistry image to be detected, and performing second-resolution image segmentation based on the second-resolution third-level lymphatic structure recognition results to obtain a second-resolution multiple fluorescence immunohistochemistry image block containing the second-resolution third-level lymphatic structure recognition results;

[0021] A pre-trained multiple fluorescence immunohistochemistry image recognition filtering model is used to filter each of the second-resolution three-level lymphatic structure recognition results, and the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle are removed to obtain each of the second-resolution multiple fluorescence immunohistochemistry image blocks. The resolution of the multiple fluorescence immunohistochemistry image used for training the multiple fluorescence immunohistochemistry image recognition filtering model is equal to the second resolution.

[0022] Optionally, the step of determining the corresponding second-resolution third-level lymphatic structure identification result in the second-resolution multiple fluorescence immunohistochemistry image to be detected according to each of the third-level lymphatic structure identification results includes:

[0023] Obtaining coordinate position information of each of the three-level lymphatic structure identification results;

[0024] The coordinate position information is converted to a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution, and the third-level lymphatic structure recognition result at the second resolution is determined.

[0025] Optionally, the step of acquiring the multiple fluorescent immunohistochemistry image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemistry image to be detected includes:

[0026] Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected;

[0027] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical image;

[0028] Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

[0029] Optionally, after the step of obtaining the identification results of each tertiary lymphatic structure of each of the multiple fluorescent immunohistochemistry image blocks, the step further includes:

[0030] The contour coordinates of each three-level lymphatic structure recognition result of each multiple fluorescent immunohistochemistry image block are obtained, and each of the contour coordinates is connected to form editable three-level lymphatic structure recognition results.

[0031] In the multiplex fluorescent immunohistochemistry image recognition method as described in any of the aforementioned embodiments, optionally, the image cutting includes overlapping cutting.

[0032] In the multiplex fluorescent immunohistochemistry image recognition method as described in any of the aforementioned embodiments, optionally, the step of acquiring the multiplex fluorescent immunohistochemistry image to be detected includes:

[0033] Obtain original multiplex fluorescence immunohistochemistry tissue samples;

[0034] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image;

[0035] The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

[0036] Optionally, the training step of the multiplex fluorescent immunohistochemistry image recognition model includes:

[0037] Acquire a first-resolution image set, wherein the first-resolution image set includes first-resolution training multiple fluorescent immunohistochemistry image blocks, and each first-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation;

[0038] Using the multiple fluorescence immunohistochemistry image recognition model, predicting the three-level lymphatic structure of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining the three-level lymphatic structure training prediction results of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks;

[0039] The parameters of the multiple fluorescent immunohistochemistry image recognition model are adjusted according to the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition model.

[0040] Optionally, the step of acquiring the first-resolution image set includes:

[0041] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0042] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0043] Optionally, the training step of the multiple fluorescent immunohistochemistry image recognition filtering model includes:

[0044] Acquire a second-resolution image set, including each second-resolution training multiple fluorescent immunohistochemistry image block, each of which is provided with a third-level lymphatic structure training annotation;

[0045] Filtering the second-resolution tertiary lymphatic structures of each second-resolution training multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition filtering model to obtain training filtering results of each tertiary lymphatic structure of each second-resolution training multiple fluorescent immunohistochemistry image block;

[0046] The parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted according to the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations meets the filtering deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

[0047] Optionally, the step of acquiring the second-resolution image set includes:

[0048] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0049] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0050] The present application also provides a multiplex fluorescence immunohistochemistry image recognition device, comprising:

[0051] a multiple fluorescent immunohistochemical image acquisition module, adapted to acquire a multiple fluorescent immunohistochemical image to be detected, wherein the resolution of the multiple fluorescent immunohistochemical image to be detected is a first resolution;

[0052] The multiple fluorescent immunohistochemistry image recognition module is suitable for using a pre-trained multiple fluorescent immunohistochemistry image recognition model to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemistry image to be detected, and obtain the recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition model is equal to the first resolution.

[0053] Optionally, the multiplex fluorescence immunohistochemistry image recognition device further includes:

[0054] a multiple fluorescent immunohistochemistry image processing module, adapted to perform image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks;

[0055] The multiple fluorescent immunohistochemical image recognition module is adapted to perform three-level lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected using a pre-trained multiple fluorescent immunohistochemical image recognition model, and obtain recognition results of each three-level lymphatic structure of the multiple fluorescent immunohistochemical image to be detected, including:

[0056] Using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain each three-level lymphatic structure recognition result of each of the multiple fluorescence immunohistochemistry image blocks;

[0057] The tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemical image to be detected is obtained according to each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemical image block.

[0058] Optionally, the multiplex fluorescence immunohistochemistry image recognition device further includes:

[0059] The area filtering module is adapted to perform area filtering on each of the three-level lymphatic structure recognition results according to a preset area threshold of the three-level lymphatic structure, remove the three-level lymphatic structure recognition results whose area values ​​are less than the area threshold, and obtain the three-level lymphatic structure recognition results with true positive area.

[0060] Optionally, the multiplex fluorescence immunohistochemistry image recognition device further includes:

[0061] a module for acquiring a multiple fluorescent immunohistochemical image to be detected at a second resolution, adapted to acquire a multiple fluorescent immunohistochemical image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemical image to be detected, wherein the second resolution is greater than the first resolution, and the multiple fluorescent immunohistochemical image to be detected at the second resolution includes identification results of each of the three-level lymphoid structures;

[0062] a second-resolution multiple fluorescent immunohistochemistry image block acquisition module, adapted to determine the corresponding second-resolution three-level lymphatic structure recognition results based on the positions of the respective three-level lymphatic structure recognition results obtained at the first resolution in the second-resolution multiple fluorescent immunohistochemistry image to be detected, and to perform second-resolution image segmentation based on the second-resolution three-level lymphatic structure recognition results to obtain a second-resolution multiple fluorescent immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition results;

[0063] The multiple fluorescent immunohistochemistry image recognition and filtering module is suitable for filtering each of the second-resolution three-level lymphatic structure recognition results using a pre-trained multiple fluorescent immunohistochemistry image recognition and filtering model, removing the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle, and obtaining each of the second-resolution multiple fluorescent immunohistochemistry image blocks. The resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition and filtering model is equal to the second resolution.

[0064] Optionally, the second-resolution multiple fluorescence immunohistochemistry image block acquisition module is adapted to determine the corresponding second-resolution three-level lymphatic structure identification result in the second-resolution multiple fluorescence immunohistochemistry image to be detected according to each of the three-level lymphatic structure identification results, including:

[0065] Obtaining coordinate position information of each of the three-level lymphatic structure identification results;

[0066] The coordinate position information is converted to a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution, and the third-level lymphatic structure recognition result at the second resolution is determined.

[0067] Optionally, the second-resolution to-be-detected multiple fluorescent immunohistochemical image acquisition module is adapted to acquire the second-resolution to-be-detected multiple fluorescent immunohistochemical image corresponding to the to-be-detected multiple fluorescent immunohistochemical image, comprising:

[0068] Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected;

[0069] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical image;

[0070] Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

[0071] Optionally, the multiplex fluorescence immunohistochemistry image recognition device further includes:

[0072] The pathological region editing module is adapted to obtain the contour coordinates of each of the three-level lymphatic structure recognition results of each of the multiple fluorescent immunohistochemistry image blocks, and connect each of the contour coordinates to form editable three-level lymphatic structure recognition results.

[0073] In the multiplex fluorescent immunohistochemistry image recognition device described in any of the aforementioned embodiments, optionally, the image cutting includes overlapping cutting.

[0074] In the multiplex fluorescent immunohistochemistry image recognition device described in any of the aforementioned embodiments, optionally, the multiplex fluorescent immunohistochemistry image acquisition module is adapted to acquire the multiplex fluorescent immunohistochemistry image to be detected, and includes:

[0075] Obtain original multiplex fluorescence immunohistochemistry tissue samples;

[0076] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image;

[0077] The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

[0078] Optionally, the multiple fluorescent immunohistochemistry image recognition model is obtained by a multiple fluorescent immunohistochemistry image recognition model training module, and the multiple fluorescent immunohistochemistry image recognition model training module is suitable for:

[0079] Acquire a first-resolution image set, wherein the first-resolution image set includes first-resolution training multiple fluorescent immunohistochemistry image blocks, and each first-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation;

[0080] Using the multiple fluorescence immunohistochemistry image recognition model, predicting the three-level lymphatic structure of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining the three-level lymphatic structure training prediction results of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks;

[0081] The parameters of the multiple fluorescent immunohistochemistry image recognition model are adjusted according to the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition model.

[0082] Optionally, the first-resolution image set is obtained by a first-resolution image acquisition module, and the first-resolution image acquisition module is adapted to:

[0083] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0084] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0085] Optionally, the multiple fluorescent immunohistochemistry image recognition and filtering model is obtained by a multiple fluorescent immunohistochemistry image recognition and filtering training module, and the multiple fluorescent immunohistochemistry image recognition and filtering training module is suitable for:

[0086] Acquire a second-resolution image set, including each second-resolution training multiple fluorescent immunohistochemistry image block, each of which is provided with a third-level lymphatic structure training annotation;

[0087] Filtering the second-resolution tertiary lymphatic structures of each second-resolution training multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition filtering model to obtain training filtering results of each tertiary lymphatic structure of each second-resolution training multiple fluorescent immunohistochemistry image block;

[0088] The parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted according to the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations until the deviation between the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations meets the filtering deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

[0089] Optionally, the second-resolution image set is obtained by a second-resolution image acquisition module, and the second-resolution image acquisition module is adapted to:

[0090] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0091] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0092] An embodiment of the present application provides an electronic device comprising at least one memory and at least one processor; the memory stores a program, and the processor calls the program to execute the multiple fluorescent immunohistochemistry image recognition method as described in any one of the above items.

[0093] An embodiment of the present application provides a storage medium storing a program suitable for recognizing multiple fluorescent immunohistochemical images, so as to implement the multiple fluorescent immunohistochemical image recognition method as described in any of the aforementioned embodiments.

[0094] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0095] The multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present invention first obtains a multiple fluorescent immunohistochemical image to be detected at a first resolution, and then uses a pre-trained multiple fluorescent immunohistochemical image recognition model to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemical image to obtain recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemical image. Of course, the resolution of the multiple fluorescent immunohistochemical image used to train the multiple fluorescent immunohistochemical image recognition model is equal to the first resolution. It can be seen that the multiple fluorescent immunohistochemical image recognition method provided in the embodiment of the present invention realizes automatic recognition of the multiple fluorescent immunohistochemical image to be detected by using a multiple fluorescent immunohistochemical image recognition model by ensuring the resolution of the multiple fluorescent immunohistochemical image to be detected. The multiple fluorescent immunohistochemical image recognition model can unify the recognition standards of the tertiary lymphatic structure, and the multiple fluorescent immunohistochemical image recognition model is pre-trained, which can ensure the accuracy of recognition. Therefore, when actually identifying the tertiary lymphatic structure, the multiple fluorescent immunohistochemical image recognition model can directly obtain the various tertiary lymphatic structures contained in the multiple fluorescent immunohistochemical image to be detected according to the recognition standards, thereby reducing dependence on analysts, and can also assist analysts in quickly identifying and interpreting the tertiary lymphatic structures in the multiple fluorescent immunohistochemical image to be detected, thereby improving analysis efficiency.

[0096] In an optional solution, the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present invention first obtains a multiple fluorescent immunohistochemistry image to be detected with a second resolution having a resolution greater than the first resolution, and the multiple fluorescent immunohistochemistry image to be detected at the second resolution includes the identified three-level lymphatic structure recognition results; then, after obtaining each three-level lymphatic structure recognition result of each of the multiple fluorescent immunohistochemistry image blocks, the corresponding second-resolution three-level lymphatic structure recognition result is determined in the multiple fluorescent immunohistochemistry image to be detected at the second resolution according to each of the three-level lymphatic structure recognition results, and a second resolution is performed according to the second-resolution three-level lymphatic structure recognition result. The method further comprises performing image segmentation at a rate of 0.05, obtaining a second-resolution multiple fluorescent immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition result; filtering each second-resolution three-level lymphatic structure recognition result using a pre-trained multiple fluorescent immunohistochemistry image recognition filtering model, removing the corresponding three-level lymphatic structure recognition results that do not meet the pathological region identification principle, and obtaining each true-positive three-level lymphatic structure recognition result for each second-resolution multiple fluorescent immunohistochemistry image block. The resolution value of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition filtering model is equal to the second resolution. In this way, on the basis of quickly obtaining the three-level lymphatic structure recognition result of the multiple fluorescent immunohistochemistry image to be detected, further identification and screening can be performed on each three-level lymphatic structure recognition result to remove the false-positive three-level lymphatic structure recognition results that do not meet the pathological region identification principle, thereby improving the accuracy of the three-level lymphatic structure recognition result in the multiple fluorescent immunohistochemistry image to be detected, making the three-level lymphatic structure recognition result in the multiple fluorescent immunohistochemistry image to be detected more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0098] Figure 1 This is a flow chart of a multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0099] Figure 2 is a schematic diagram of multiple fluorescent immunohistochemical images of the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0100] Figure 3a is a schematic diagram of the original multiplex fluorescence immunohistochemistry image;

[0101] Figure 3b yes Figure 3a Schematic diagram of a multiplex fluorescent immunohistochemical image obtained after the original multiplex fluorescent immunohistochemical image shown in is exposed;

[0102] Figure 3c is another schematic diagram of the original multiplex fluorescence immunohistochemistry image;

[0103] Figure 3d yes Figure 3c Another schematic diagram of a multiplex fluorescent immunohistochemical image obtained after the original multiplex fluorescent immunohistochemical image shown in is exposed;

[0104] Figure 4 This is another flow chart of the multiplex fluorescent immunohistochemistry image recognition method provided in the embodiments of the present application;

[0105] Figure 5a This is a schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution;

[0106] Figure 5b is another schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution;

[0107] Figure 5c This is another schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution;

[0108] Figure 5d This is another schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution;

[0109] Figure 6 This is another schematic flow chart of the multiplex fluorescence immunohistochemistry image recognition method provided in an embodiment of the present application;

[0110] Figure 7a This is a schematic diagram of a state before the area filtering step of the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0111] Figure 7b yes Figure 7a The schematic diagram of the filtering results after area filtering is shown;

[0112] Figure 81 is another flow chart of the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0113] Figure 9a Yes Figure 5a The recognition result shown is a schematic diagram of a state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0114] Figure 9b Yes Figure 5b The recognition result shown is a schematic diagram of another state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0115] Figure 9c Yes Figure 5c The recognition result shown is another schematic diagram of a state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0116] Figure 9d Yes Figure 5d The recognition result shown is a schematic diagram of another state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0117] Figure 10 This is a schematic diagram of a training process of a multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application;

[0118] Figure 11 This is a schematic diagram of a state of labeling a first-resolution image set in the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0119] Figure 12 This is a schematic diagram of a training process of a multiple fluorescent immunohistochemistry image recognition filtering model in the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0120] Figure 13 is a schematic diagram of a first-resolution image set obtained at a first resolution in the multiplexed fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0121] Figure 14 is a schematic diagram of a second-resolution image set obtained at a second resolution in the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application;

[0122] Figure 15This is an optional block diagram of the multiplex fluorescence immunohistochemistry image recognition device provided in an embodiment of the present application;

[0123] Figure 16 This is another optional block diagram of the multiplex fluorescence immunohistochemistry image recognition device provided in an embodiment of the present application;

[0124] Figure 17 This is another optional block diagram of the multiplex fluorescence immunohistochemistry image recognition device provided in an embodiment of the present application;

[0125] Figure 18 Schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0126] As known from the background art, the efficiency of identifying tertiary lymphatic structures in multiple fluorescent immunohistochemical images is not high.

[0127] To improve the recognition efficiency of tertiary lymphatic structures in multiple fluorescence immunohistochemistry images, an embodiment of the present invention provides a multiple fluorescence immunohistochemistry image recognition method and related devices. The multiple fluorescence immunohistochemistry image recognition method includes:

[0128] Acquiring a multiplex fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemistry image to be detected is a first resolution;

[0129] A pre-trained multiple fluorescent immunohistochemistry image recognition model is used to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemistry image to be detected to obtain recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition model is equal to the first resolution.

[0130] It can be seen that the multiple fluorescent immunohistochemical image recognition method provided in the embodiment of the present invention realizes automatic recognition of the multiple fluorescent immunohistochemical image to be detected by using a multiple fluorescent immunohistochemical image recognition model by ensuring the resolution of the multiple fluorescent immunohistochemical image to be detected. The multiple fluorescent immunohistochemical image recognition model can unify the recognition standards of the tertiary lymphatic structure, and the multiple fluorescent immunohistochemical image recognition model is pre-trained, which can ensure the accuracy of recognition. Therefore, when actually identifying the tertiary lymphatic structure, the multiple fluorescent immunohistochemical image recognition model can directly obtain the various tertiary lymphatic structures contained in the multiple fluorescent immunohistochemical image to be detected according to the recognition standards, thereby reducing dependence on analysts, and can also assist analysts in quickly identifying and interpreting the tertiary lymphatic structures in the multiple fluorescent immunohistochemical image to be detected, thereby improving analysis efficiency.

[0131] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0132] It should be noted that the orientations or positional relationships indicated in this specification are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience and simplification of description, and do not indicate or imply that the device referred to must have a specific orientation or be constructed in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0133] The rapid recognition and analysis of pathological images are of great significance for clinical diagnosis. Therefore, research on pathological image recognition methods is very necessary.

[0134] Pathological images include HE-stained pathological images obtained using hematoxylin-eosin staining and multiplexed fluorescent immunohistochemistry images obtained using multiplexed fluorescent immunohistochemistry technology. This application mainly discusses multiplexed fluorescent immunohistochemistry images, and mainly describes the multiplexed fluorescent immunohistochemistry image recognition method described in this application for multiplexed fluorescent immunohistochemistry images of tertiary lymphoid structure (TLS).

[0135] Tertiary lymphoid structures (TLS) refer to ectopic lymphoid organs formed in non-lymphoid organs under disease conditions. Tertiary lymphoid structures are usually induced in tissue sites with chronic inflammation, such as tumors, persistent pathogen infection, allogeneic rejection caused by organ transplantation, and autoimmune diseases.

[0136] In autoimmune diseases, the presence of tertiary lymphoid structures is often negatively correlated with prognosis, such as rheumatoid arthritis, Sjögren's syndrome ( Syndrome) and Hashimoto's thyroiditis, in which an increase in tertiary lymphoid structures is associated with increased disease severity. However, in most tumors, tertiary lymphoid findings are generally positively correlated with good prognosis. For example, in breast cancer, colorectal cancer (CRC), and lung cancer, the greater the number of tertiary lymphoid structures, the better the patient's prognosis. Therefore, by analyzing tertiary lymphoid structures in multiplex fluorescence immunohistochemistry images, it is possible to understand the patient's prognosis.

[0137] Since the technical solutions for the identification of tertiary lymphoid structures are all based on hematoxylin-eosin (HE) stained pathological images, HE staining cannot distinguish different types of lymphocyte subtypes well. Multiple fluorescent immunohistochemistry technology can mark different cell subtypes in a tissue section by marking multiple proteins on the same tissue section, thereby distinguishing different subtypes of B lymphocytes and T lymphocytes. Therefore, multiple fluorescent immunohistochemistry technology can visualize the tertiary lymphoid structure, making it easier to identify the tertiary lymphoid structure. However, even with the use of multiple fluorescent immunohistochemistry technology for marking, the current identification and interpretation of the tertiary lymphoid structure relies on manual labor, which results in large subjective differences in the interpretation standards of different analysts for incompletely typical tertiary lymphoid structures. Even for the same analyst, there may be inconsistent standards for the identification and judgment of the tertiary lymphoid structure in the same HE stained pathological image at different times, and the recognition efficiency is low.

[0138] Therefore, in order to solve the above problems, the embodiments of the present application provide a multiple fluorescent immunohistochemical image recognition method for rapidly identifying tertiary lymphatic structures using multiple fluorescent immunohistochemical images obtained by multiple fluorescent immunohistochemical techniques.

[0139] The following is a detailed description of the multiple fluorescent immunohistochemical image recognition method provided in the embodiment of the present application. Please refer to Figure 1 , Figure 1 1 is a flow chart of the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0140] like Figure 1 As shown, the multiple fluorescent immunohistochemistry image recognition method provided in the embodiment of the present application may include the following steps:

[0141] Step S10 , obtaining a multiplex fluorescent immunohistochemical image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemical image to be detected is a first resolution.

[0142] First, a multiplex fluorescent immunohistochemical image to be identified and detected is obtained, that is, a multiplex fluorescent immunohistochemical image to be detected. The description will continue with the aforementioned multiplex fluorescent immunohistochemical image of the three-level lymphatic structure as an example.

[0143] It is easy to understand that if the directly acquired multiplex fluorescent immunohistochemical image is already a multiplex fluorescent immunohistochemical image of a tertiary lymphatic structure and has the first resolution, it can be used directly.

[0144] In addition, the first resolution described in this article is the scanning resolution set when scanning using a digital scanner. Any resolution can be selected as needed. If the resolution of the multiple fluorescent immunohistochemistry image obtained by the first digital scan is not the first resolution, it can be obtained by adjusting the scanning magnification of the digital scanner to the first resolution.

[0145] The multiplex fluorescent immunohistochemical image to be detected acquired in step S10 is the multiplex fluorescent immunohistochemical image of the tertiary lymphatic structure to be detected, and the multiplex fluorescent immunohistochemical image of the tertiary lymphatic structure to be detected is acquired by scanning at the first resolution.

[0146] In this way, it is ensured that the multiple fluorescent immunohistochemical image to be detected can be conveniently used to quickly identify the tertiary lymphatic structure using the multiple fluorescent immunohistochemical image recognition method provided in the embodiment of the present application, that is, to identify the pathological area formed by the tertiary lymphatic structure in the multiple fluorescent immunohistochemical image to be detected.

[0147] In a specific embodiment, the step of acquiring the multiple fluorescent immunohistochemistry images to be detected may include:

[0148] Obtain original multiplex fluorescence immunohistochemistry tissue samples;

[0149] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image;

[0150] The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

[0151] The first-resolution multiplex fluorescence immunohistochemistry image obtained by digital scanning of the original multiplex fluorescence immunohistochemistry tissue sample is a pathological image after the tertiary lymphatic structure is fluorescently labeled with a fluorescent agent, thereby visualizing the tertiary lymphatic structure. In this way, the tertiary lymphatic structure can be identified more conveniently and quickly.

[0152] In order to better understand the multiple fluorescent immunohistochemical images described in this application, in one embodiment, CD3 and CD20 fluorescent channels can be used for fluorescence processing, which can be referred to Figure 2 , Figure 2 Schematic diagram of multiple fluorescent immunohistochemical images of the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application.

[0153] CD3 is present only on the surface of T cells and consists of six peptide chains. It often binds tightly to the TCR to form an eight-peptide TCR-CD3 complex, which participates in T cell antigen recognition and signal transduction. The CD3 molecule connects to the T cell antigen receptor (TCR) via a salt bridge, participating in T cell signal transduction and primarily marking thymocytes, T lymphocytes, and T cell lymphomas.

[0154] CD20 is expressed on the surface of B cells at all stages of development and differentiation except plasma cells (B cells that secrete immunoglobulins). It directly acts on B cells by regulating transmembrane calcium ion flow and plays an important regulatory role in B cell proliferation and differentiation.

[0155] As shown in the figure, the area of ​​the tertiary lymphoid structure in the multiple fluorescent immunohistochemical image obtained by using CD3 and CD20 fluorescent channels to mark T lymphocytes and B lymphocytes respectively is the pathological area surrounded by the first fluorescent agent on the outside and the second fluorescent agent on the inside (the color arrangement of the fluorescent agents represents different structural areas: Figure 2 As shown in , the first fluorescent agent is light gray and the second fluorescent agent is bright white. Therefore, the fluorescent agent arrangement of the tertiary lymphatic structure is the area with light gray outside and bright white inside ( Figure 2 The other types of areas are non-pathological structural areas, such as all light gray, all bright white, and areas with bright white outside and light gray inside ( Figure 2 The structure in the left frame).

[0156] After the multiple fluorescent immunohistochemical image is obtained, it is scanned according to the first resolution to obtain the first resolution multiple fluorescent immunohistochemical image.

[0157] The first resolution is the first resolution described in step S10. The specific resolution value can be set as needed, with the purpose of adjusting it to be consistent with the resolution of the trained multiple fluorescent immunohistochemistry image of the multiple fluorescent immunohistochemistry image recognition model, so that in subsequent steps, the pre-trained multiple fluorescent immunohistochemistry image recognition model can be used to identify the tertiary lymphatic structure.

[0158] In a specific implementation, the value of the first resolution may be set to 5x resolution.

[0159] After obtaining the first-resolution fluorescent multiplex fluorescent immunohistochemical image, exposure adjustment may be performed on it to obtain the multiplex fluorescent immunohistochemical image to be detected.

[0160] In order to clearly display the tertiary lymphatic structure, the brightness can be adjusted to make the two fluorescent agents more obvious (it should be noted that the colors of the two fluorescent agents are different, and adjusting the exposure can make the colors of the two fluorescent agents more obvious and easier to distinguish), so that the tertiary lymphatic structure in the first resolution fluorescent multiplex fluorescent immunohistochemistry image can be easily identified ( Figure 2 The first light grey phosphor is shown on the outside and the second bright white phosphor is shown on the inside).

[0161] In order to facilitate the understanding of the multiple fluorescent immunohistochemical images after exposure (i.e. including the multiple fluorescent immunohistochemical images of the first resolution mentioned above), reference may be made to Figure 3a-3d , Figure 3a is a schematic diagram of the original multiplex fluorescent immunohistochemistry image. Figure 3b yes Figure 3a Schematic diagram of the multiplex fluorescent immunohistochemical image obtained after the original multiplex fluorescent immunohistochemical image shown in is exposed. Figure 3c This is another schematic diagram of the original multiplex fluorescence immunohistochemistry image. Figure 3d yes Figure 3c Another schematic diagram of a multiplex fluorescent immunohistochemical image obtained after the original multiplex fluorescent immunohistochemical image shown in is exposed.

[0162] As you can see, Figure 3a and Figure 3c The original multiplex fluorescent immunohistochemistry image shown in the figure is very unclear after being fluorescently labeled and without adjusting the exposure. Figure 3b and Figure 3d The image shown can clearly identify the tertiary lymphatic structure, which is helpful for the subsequent recognition processing of the multiple fluorescence immunohistochemistry image recognition model.

[0163] In this way, by adjusting the resolution and exposure processing of the original multiple fluorescent immunohistochemistry image, the image content of the multiple fluorescent immunohistochemistry image to be detected can be clearer, and the tertiary lymphatic structure can be identified more quickly during the subsequent multiple fluorescent immunohistochemistry image recognition, thereby improving the recognition efficiency of the tertiary lymphatic structure.

[0164] Of course, in other embodiments, depending on the different original multiple fluorescent immunohistochemistry images obtained, different methods can be used to obtain the multiple fluorescent immunohistochemistry images to be detected. For example, for multiple fluorescent immunohistochemistry images whose resolution has already met the requirements, resolution adjustment may not be performed; for multiple fluorescent immunohistochemistry images whose image brightness has already met the requirements, exposure adjustment may not be performed, etc.

[0165] Step S11 , using a pre-trained multiplex fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on the multiplex fluorescence immunohistochemistry image to be detected, and obtaining recognition results of each three-level lymphatic structure of the multiplex fluorescence immunohistochemistry image to be detected.

[0166] Of course, it is easy to understand that in order to successfully identify the tertiary lymphatic structure in the multiple fluorescent immunohistochemistry image to be detected, the resolution of the multiple fluorescent immunohistochemistry image used by the multiple fluorescent immunohistochemistry image recognition model is consistent with the resolution of the multiple fluorescent immunohistochemistry image to be detected, and is therefore equal to the first resolution.

[0167] The aforementioned “tertiary lymphatic structure identification result” refers to an area in the multiplex fluorescent immunohistochemistry image to be detected that is similar to the conventional outline of the aforementioned tertiary lymphatic structure.

[0168] It can be seen that the multiple fluorescent immunohistochemical image recognition method provided in the embodiment of the present invention realizes automatic recognition of the multiple fluorescent immunohistochemical image to be detected by using a multiple fluorescent immunohistochemical image recognition model by ensuring the resolution of the multiple fluorescent immunohistochemical image to be detected. The multiple fluorescent immunohistochemical image recognition model can unify the recognition standards of the tertiary lymphatic structure, and the multiple fluorescent immunohistochemical image recognition model is pre-trained, which can ensure the accuracy of recognition. Therefore, when actually identifying the tertiary lymphatic structure, the multiple fluorescent immunohistochemical image recognition model can directly obtain the various tertiary lymphatic structures contained in the multiple fluorescent immunohistochemical image to be detected according to the recognition standards, thereby reducing dependence on analysts, and can also assist analysts in quickly identifying and interpreting the tertiary lymphatic structures in the multiple fluorescent immunohistochemical image to be detected, thereby improving analysis efficiency.

[0169] In another specific embodiment, please refer to Figure 4 , Figure 4 This is another flow chart of the multiple fluorescent immunohistochemistry image recognition method provided in the embodiment of the present application. In order to further reduce the difficulty of training the multiple fluorescent immunohistochemistry image recognition model and improve the accuracy of the tertiary lymphatic structure recognition results, the multiple fluorescent immunohistochemistry image recognition method provided in the embodiment of the present application may further include:

[0170] Step S20 , obtaining a multiplex fluorescent immunohistochemical image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemical image to be detected is a first resolution.

[0171] For details of step S20, please refer to Figure 1 The detailed description of step S10 is omitted here.

[0172] Step S21 , performing image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks.

[0173] According to the above, the more tertiary lymphatic structures there are in the multiple fluorescent immunohistochemistry image to be detected, the better the patient's prognosis. Therefore, in order to conveniently and accurately determine the number of tertiary lymphatic structures there are in the multiple fluorescent immunohistochemistry image to be detected, the multiple fluorescent immunohistochemistry image to be detected can be segmented. By identifying each multiple fluorescent immunohistochemistry image block after segmentation, the tertiary lymphatic structure in each multiple fluorescent immunohistochemistry image block can be determined, and then the tertiary lymphatic structure in the entire multiple fluorescent immunohistochemistry image to be detected can be determined. This can improve the efficiency and accuracy of identifying all tertiary lymphatic structures in the multiple fluorescent immunohistochemistry image to be detected, and provide a reliable data basis for analysts.

[0174] Of course, it is easy to understand that in this case, the multiple fluorescent immunohistochemistry image recognition model also uses the training multiple fluorescent immunohistochemistry image blocks that have undergone image segmentation during the training process.

[0175] In order to ensure the integrity of the tertiary lymphatic structures included in the multiple fluorescent immunohistochemistry image blocks obtained by segmenting the multiple fluorescent immunohistochemistry image to be detected, in one embodiment, the image cutting includes overlapping cutting.

[0176] By overlapping and cutting, it can be ensured that the tertiary lymphatic structures contained in each multiple fluorescent immunohistochemistry image block used for multiple fluorescent immunohistochemistry image recognition are sufficiently complete, which can improve the efficiency and accuracy of subsequent multiple fluorescent immunohistochemistry image recognition.

[0177] After completing the acquisition of each multiple fluorescent immunohistochemical image block, in order to obtain each tertiary lymphatic structure of the entire multiple fluorescent immunohistochemical image to be detected, step S22, using a pre-trained multiple fluorescent immunohistochemical image recognition model to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected, the step of obtaining the recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemical image to be detected may include:

[0178] Step S220 , using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain three-level lymphatic structure recognition results for each of the multiple fluorescence immunohistochemistry image blocks.

[0179] Of course, the resolution of the training multiple fluorescent immunohistochemical image blocks of the multiple fluorescent immunohistochemical image recognition model is consistent with the resolution of the multiple fluorescent immunohistochemical image to be detected, which is equal to the first resolution.

[0180] Specifically, the three-level lymphatic structures of the multiple fluorescent immunohistochemistry image blocks can be referred to Figure 5a-5d , Figure 5a 1 is a schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution. Figure 5b is another schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at a first resolution. Figure 5c is another schematic diagram of output results obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application at the first resolution. Figure 5d This is another schematic diagram of an output result obtained by using the multiple fluorescent immunohistochemical image recognition model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application to recognize the image at the first resolution.

[0181] As shown in the figure, it can be seen that the multiple fluorescent immunohistochemistry image recognition model can quickly find recognition results that are similar to the contours of the tertiary lymphatic structures that need to be detected and identified.

[0182] Step S221 , obtaining the tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemical image to be detected according to each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemical image block.

[0183] After obtaining the tertiary lymphatic structure recognition results in each multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition model, in order to enable analysts to more conveniently and quickly obtain the interpretation results of the detected multiple fluorescent immunohistochemistry image, that is, to determine the prognostic effect based on the number of tertiary lymphatic structure recognition results in the entire multiple fluorescent immunohistochemistry image to be detected, the tertiary lymphatic structure recognition results in the entire multiple fluorescent immunohistochemistry image to be detected can be obtained by counting the tertiary lymphatic structure recognition results in each multiple fluorescent immunohistochemistry image block. The analyst can directly analyze and judge the prognostic effect of the multiple fluorescent immunohistochemistry image to be detected based on the final counted tertiary lymphatic structure recognition results, thereby improving the analysis efficiency of the analyst.

[0184] In this way, by cutting the images of the multiple fluorescent immunohistochemistry images to be detected and obtaining the tertiary lymphatic structure recognition results of each multiple fluorescent immunohistochemistry image block, on the one hand, the recognition difficulty of the multiple fluorescent immunohistochemistry image recognition model for each multiple fluorescent immunohistochemistry image block can be reduced and the accuracy can be improved. On the other hand, more training multiple fluorescent immunohistochemistry image blocks can be obtained using fewer training multiple fluorescent immunohistochemistry images, thereby increasing the training data and improving the training accuracy of the multiple fluorescent immunohistochemistry image recognition model. While reducing the difficulty of model training, the accuracy of the model is improved, thereby improving the accuracy of recognition.

[0185] Furthermore, in order to facilitate the subsequent review of the three-level lymphatic structure recognition results by the analysts, in a specific embodiment, the three-level lymphatic structure recognition results can also be edited and the three-level lymphatic structure recognition results can be visualized. For details, please continue to refer to Figure 4 .

[0186] As shown in the figure, the process may also include the following steps:

[0187] Step S23 , obtaining the contour coordinates of each of the three-level lymphatic structure recognition results of each of the multiple fluorescent immunohistochemistry image blocks, and connecting each of the contour coordinates to form editable three-level lymphatic structure recognition results.

[0188] The contour coordinates of each tertiary lymphatic structure recognition result can be obtained by various methods of obtaining contour coordinates, and then the contour coordinates are connected to form an editable tertiary lymphatic structure recognition result.

[0189] In one embodiment, the fingContours function of OpenCV can be used to search for contours of the above-mentioned three-level lymphatic structure recognition results, obtain the coordinates of the contours in each three-level lymphatic structure recognition result, connect the contour coordinates to form an editable polygon graphic, and output the final result.

[0190] In this way, each tertiary lymphatic structure identification result is edited and processed so that each tertiary lymphatic structure identification result can be visualized, which is convenient for analysts to quickly determine the location and accuracy of each tertiary lymphatic structure identification result, thereby improving analysis efficiency while ensuring the reliability of analysis results.

[0191] Of course, in the embodiment of directly obtaining the tertiary lymphatic structure recognition results of the multiple fluorescent immunohistochemistry image to be detected, the tertiary lymphatic structures identified in the entire multiple fluorescent immunohistochemistry image to be detected can also be edited, that is, the contour coordinates of each tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemistry image to be detected are obtained, and each of the contour coordinates is connected to form editable tertiary lymphatic structure recognition results, so that the identified tertiary lymphatic structures can be visualized, which is convenient for analysts to quickly determine the position and accuracy of each tertiary lymphatic structure recognition result, thereby improving the analysis efficiency and ensuring the reliability of the analysis results.

[0192] In order to improve the recognition accuracy of the multiple fluorescent immunohistochemical images to be detected, and thus improve the reliability of the analysis results of the analysts, in a specific embodiment, the results of the tertiary lymphatic structure recognition can be further screened. For details, please refer to Figure 6 , Figure 6 This is another flow chart of the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0193] As shown in the figure, the process also includes the following steps:

[0194] Step S30 , obtaining a multiplex fluorescent immunohistochemical image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemical image to be detected is a first resolution.

[0195] For step S30 , reference may be made to the description of step S10 , which will not be repeated here.

[0196] Step S31 , performing image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks.

[0197] For step S31 , reference may be made to the description of step S21 , which will not be repeated here.

[0198] Step S32 , using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain three-level lymphatic structure recognition results for each of the multiple fluorescence immunohistochemistry image blocks.

[0199] For step S32, reference may be made to the description of step S221, which will not be repeated here.

[0200] Step S33 , performing area filtering on each of the three-level lymphatic structure recognition results according to a preset area threshold of the three-level lymphatic structure, removing the three-level lymphatic structure recognition results with area values ​​smaller than the area threshold, and obtaining the three-level lymphatic structure recognition results with true positive area.

[0201] According to the foregoing content, each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemistry image block obtained by the multiple fluorescent immunohistochemistry image recognition model is an area similar to the conventional contour of the tertiary lymphatic structure to be detected. However, there will be some incomplete tertiary lymphatic structures in these tertiary lymphatic structure recognition results, that is, although the identified tertiary lymphatic structure belongs to the conventional contour of the pathological structure, its contour area has no clinical diagnostic significance when the analyst performs analysis. Therefore, further screening and judgment can be performed based on its contour area, and the tertiary lymphatic structure recognition results that are beneficial to the analysis will be retained, and the tertiary lymphatic structure recognition results that have no clinical diagnostic significance will be filtered out, thereby improving the analysis and diagnosis efficiency of the analyst.

[0202] The "area value" is the area value of the diseased tertiary lymphatic structure with clinical diagnostic significance that can be used for analysis by analysts.

[0203] For details, please refer to Figure 7a and Figure 7b , Figure 7a 1 is a schematic diagram of a state before the area filtering step of the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application. Figure 7b yes Figure 7a The figure shows the filtering results after area filtering.

[0204] It can be seen that through area filtering, the tertiary lymphatic structures that do not meet the area threshold are filtered out, and the area true positive tertiary lymphatic structure identification results that can be used to assist analysts in analysis are obtained.

[0205] Of course, the area threshold is a value set according to needs, and specifically, an empirical value obtained based on experience can be selected.

[0206] In this way, by further screening the areas of the tertiary lymphatic structure identification results, the true positive tertiary lymphatic structure identification results with areas that can help analysts analyze can be obtained, thereby improving the recognition accuracy of the multiple fluorescent immunohistochemistry images to be tested.

[0207] After area filtering, more accurate true positive tertiary lymphatic structure recognition results for each area are obtained. At this time, the number of true positive tertiary lymphatic structure recognition results for each area of ​​each multiple fluorescent immunohistochemistry image block can also be counted. For details, please continue to refer to Figure 6 .

[0208] Step S34 , determining the number of true positive tertiary lymphatic structure recognition results for each area of ​​each of the multiple fluorescent immunohistochemistry image blocks, and obtaining the number of true positive tertiary lymphatic structure recognition results for the entire area of ​​the multiple fluorescent immunohistochemistry image to be detected.

[0209] On the one hand, it can facilitate analysts to directly analyze and interpret statistical results; on the other hand, it can improve the accuracy of statistical results, thereby providing analysts with a reliable data basis.

[0210] Of course, in order to facilitate the analysis of the tertiary lymphatic structures after area filtering, in one embodiment, the contour coordinates of the area true positive tertiary lymphatic structures can also be edited. For details, please continue to refer to Figure 6 .

[0211] Step S35 , obtaining the contour coordinates of each area true positive tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemistry image block, and connecting each contour coordinate to form an editable area true positive tertiary lymphatic structure recognition result.

[0212] Of course, as described in step S23, when searching for the contour coordinates of the above-mentioned true-positive tertiary lymphatic structure recognition results of each area, the fingContours function of OpenCV can also be used to obtain the coordinates of the contours in the true-positive tertiary lymphatic structure recognition results of each area, connect the contour coordinates to form an editable polygon graphic, and output the final result.

[0213] In this way, analysts can easily analyze and judge the true positive tertiary lymphatic structure identification results of each area to obtain more accurate clinical diagnosis results.

[0214] In order to further improve the recognition accuracy of the multiple fluorescent immunohistochemical images to be detected and ensure the reliability of the analysis results of the analysts, in one embodiment, the identified tertiary lymphatic structures can be further processed by other methods. For details, please refer to Figure 8 , Figure 8 This is another flowchart of the multiplex fluorescent immunohistochemistry image recognition method provided in the embodiments of the present application.

[0215] As shown in the figure, the process may include the following steps:

[0216] Step S300 : Acquire a multiplex fluorescent immunohistochemical image to be detected, where the resolution of the multiplex fluorescent immunohistochemical image to be detected is a first resolution.

[0217] For step S300 , reference may be made to the description of step S10 , which will not be repeated here.

[0218] Step S301 : performing image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks.

[0219] For step S301 , reference may be made to the description of step S21 , which will not be repeated here.

[0220] Step S302 , using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain three-level lymphatic structure recognition results for each of the multiple fluorescence immunohistochemistry image blocks.

[0221] For step S302 , reference may be made to the description of step S22 , which will not be repeated here.

[0222] Step S303 : acquiring the multiplex fluorescent immunohistochemical image to be detected at a second resolution corresponding to the multiplex fluorescent immunohistochemical image to be detected.

[0223] Of course, the second resolution is greater than the first resolution, and the multiple fluorescent immunohistochemistry image to be detected at the second resolution includes the identification results of each of the three-level lymphatic structures.

[0224] It should be noted that the above-mentioned “identification results of each tertiary lymphatic structure” mean that the conventional outline of the tertiary lymphatic structure to be detected and identified is met, and the composition structure of the tertiary lymphatic structure to be detected and identified is met (i.e., the tertiary lymphatic structure including the first light gray fluorescent marker and the second bright white fluorescent marker). Figure 5c The composition of the fluorescent agent in the tertiary lymphatic structure is shown.

[0225] After processing the multiple fluorescent immunohistochemistry image to be detected at a first resolution, the resulting tertiary lymphatic structure recognition results are regions similar to the conventional outlines of the tertiary lymphatic structures to be detected and identified. Therefore, to further ensure that the resulting tertiary lymphatic structure recognition results with similar outlines are regions of authentic tertiary lymphatic structures, a second, higher resolution multiple fluorescent immunohistochemistry image to be detected can be obtained for further processing.

[0226] The second resolution value of the obtained second-resolution multiple fluorescent immunohistochemistry image to be detected is greater than the first resolution, which can further magnify the location of the tertiary lymphatic structure that has been determined, making it easier to identify and filter the specific fluorescent labeling forms in the identification results of each tertiary lymphatic structure.

[0227] In this way, on the basis of obtaining an identification result that is similar to the conventional outline of the diseased tertiary lymphatic structure that needs to be detected and identified, the specific fluorescent labeling form within each tertiary lymphatic structure identification result can be further filtered, so that the true positive identification tertiary lymphatic structure identification result finally obtained is the tertiary lymphatic structure required by the analyst during analysis and interpretation.

[0228] In order to facilitate further filtering of the tertiary lymphatic structures, in a specific embodiment, the second-resolution multiple fluorescent immunohistochemical image to be detected can be fluorescently labeled to mark the specific composition of the tertiary lymphatic structure to be detected. Specifically, the following steps can be included:

[0229] Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected;

[0230] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical image;

[0231] Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

[0232] It is easy to understand that since the position information of the tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected at the first resolution needs to be correspondingly marked and displayed in the multiple fluorescent immunohistochemistry image to be detected at the second resolution, the original multiple fluorescent immunohistochemistry tissue sample of the multiple fluorescent immunohistochemistry image to be detected at the second resolution is consistent with the original multiple fluorescent immunohistochemistry tissue sample of the multiple fluorescent immunohistochemistry image to be detected, that is, the multiple fluorescent immunohistochemistry image to be detected and the multiple fluorescent immunohistochemistry image to be detected at the second resolution are both obtained by processing the same original multiple fluorescent immunohistochemistry tissue sample.

[0233] It should be noted that in order to achieve corresponding annotation of the tertiary lymphoid structures between the second-resolution multiple fluorescent immunohistochemistry image to be detected and the multiple fluorescent immunohistochemistry image to be detected, the fluorescent agents used by the two are the same, that is, the CD3 fluorescent channel to mark T lymphocytes and the CD20 fluorescent channel to mark B lymphocytes as described above.

[0234] Then, a second-resolution fluorescent multiplex fluorescent immunohistochemistry image is obtained according to the second resolution.

[0235] The second resolution has a greater resolution value than the first resolution, which can make the content displayed by the second resolution fluorescent multiple fluorescent immunohistochemistry image clearer and the recognition of the tertiary lymphatic structure more accurate.

[0236] Finally, exposure adjustment is performed on the second-resolution fluorescence multiplex fluorescence immunohistochemistry image to obtain the second-resolution multiplex fluorescence immunohistochemistry image to be detected.

[0237] In this way, the subsequent recognition of the multiple fluorescent immunohistochemistry image recognition filter model can be facilitated, making the final recognition result more accurate.

[0238] After obtaining the identification results of each third-level lymphatic structure of each of the multiple fluorescent immunohistochemistry image blocks in step S302, the following steps may be further included:

[0239] Step S304: Determine the corresponding second-resolution three-level lymphatic structure recognition result based on the position of each of the three-level lymphatic structure recognition results obtained at the first resolution in the second-resolution multiple fluorescence immunohistochemistry image to be detected, and perform second-resolution image segmentation based on the second-resolution three-level lymphatic structure recognition result to obtain a second-resolution multiple fluorescence immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition result.

[0240] Each tertiary lymphatic structure identification structure identified at the first resolution is matched to the acquired second resolution multiple fluorescence immunohistochemistry image to be detected, and the corresponding determined position is the second resolution pathological area, and then the second resolution multiple fluorescence immunohistochemistry image to be detected is image segmented.

[0241] Specifically, the second resolution pathological area can be determined by:

[0242] Obtaining coordinate position information of each of the three-level lymphatic structure identification results;

[0243] The coordinate position information is converted into a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution to determine the pathological area at the second resolution.

[0244] After identifying the recognition results that are similar to the conventional contours of the tertiary lymphatic structures to be detected and identified by using the multiple fluorescent immunohistochemistry image recognition model, in order to quickly further screen the recognition results of each tertiary lymphatic structure, the contour coordinate position information of each identified tertiary lymphatic structure recognition result can be obtained. Each coordinate position information indicates that the corresponding area is similar to the conventional contour of the tertiary lymphatic structure.

[0245] It should be noted that the original multiple fluorescent immunohistochemical tissue sample of the multiple fluorescent immunohistochemical image to be detected at the first resolution and the multiple fluorescent immunohistochemical image to be detected at the second resolution is the same multiple fluorescent immunohistochemical tissue sample, except that the scanning magnifications of the tissue samples are different. For example, the multiple fluorescent immunohistochemical image to be detected at the first resolution mentioned above is obtained by scanning at a scanning magnification of 5x. Then, in one embodiment, the multiple fluorescent immunohistochemical image to be detected at the second resolution can be obtained by displaying the multiple fluorescent immunohistochemical image to be detected obtained at a scanning magnification of 5x and then displaying it at a resolution of 10x, or the same multiple fluorescent immunohistochemical image can be obtained at a scanning magnification of 5x and a scanning magnification of 10x respectively. Of course, the scanning magnification can be adjusted and set according to actual needs, as long as the resolution value of the second resolution is ensured to be greater than the resolution value of the first resolution.

[0246] In this way, by determining the position of the tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected at the first resolution on the multiple fluorescent immunohistochemistry image to be detected at the second resolution, the second resolution pathological area is obtained, and the specific image content of each tertiary lymphatic structure can be magnified, so that the composition form of the fluorescent markers in each second resolution pathological area is clearer. When further filtering is performed using the multiple fluorescent immunohistochemistry image recognition filtering model, the false positive tertiary lymphatic structure recognition results can be easily and quickly filtered out, and a more accurate identification of the true positive tertiary lymphatic structure recognition result can be obtained.

[0247] In order to improve the accuracy of the filtering results, the entire second-resolution multiple fluorescence immunohistochemistry image to be detected can be cut. Of course, in order to obtain each second-resolution multiple fluorescence immunohistochemistry image block with a complete second-resolution pathological area, the image cutting can include overlapping cutting.

[0248] Step S305: Filter each of the second-resolution three-level lymphatic structure recognition results using a pre-trained multiple fluorescence immunohistochemistry image recognition filtering model, remove the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle, and obtain each of the second-resolution multiple fluorescence immunohistochemistry image blocks. Each of the three-level lymphatic structure recognition results is a true positive.

[0249] Of course, the resolution of the multiple fluorescent immunohistochemistry images used for training the multiple fluorescent immunohistochemistry image recognition and filtering model is equal to the second resolution.

[0250] The pathological area identification principle is the area where the form of the fluorescent identification within the tertiary lymphatic structure is the same as the form of the fluorescent identification of the actual pathological structure, which can be the aforementioned "area of ​​the tertiary lymphatic structure identification result is the pathological area surrounded by the first light gray fluorescent agent on the outside and the second bright white fluorescent agent on the inside". The specific pathological area identification principle is determined according to the color form of the fluorescent agent actually used.

[0251] The multiple fluorescence immunohistochemistry image recognition filtering model can further filter the fluorescence identification form of each tertiary lymphatic structure identification result within the second resolution pathological area according to the true fluorescence identification form of the tertiary lymphatic structure, filter out the areas that do not conform to the true fluorescence identification form of the tertiary lymphatic structure, and obtain a true positive tertiary lymphatic structure identification result.

[0252] For specific recognition results, please refer to Figure 9a-9d , Figure 9a Yes Figure 5a The recognition result shown is a schematic diagram of a state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application. Figure 9b Yes Figure 5b The recognition result shown is a schematic diagram of another state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application. Figure 9c Yes Figure 5c The recognition result shown is another schematic diagram of a state in which the multiple fluorescent immunohistochemical image recognition filtering model further filters at a second resolution in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application. Figure 9d Yes Figure 5d The recognition result shown is a schematic diagram of another state in which the multiple fluorescent immunohistochemical image recognition filtering model in the multiple fluorescent immunohistochemical image recognition method provided in an embodiment of the present application further filters at a second resolution.

[0253] As shown in the figure, the multiple fluorescent immunohistochemistry image recognition filtering model Figure 5a-5d The identification results shown are further filtered to remove the areas where the fluorescence identification form does not conform to the true positive tertiary lymphatic structure fluorescence identification form, such as Figure 9d No identification was made in Figure 5d The three-level lymphatic structure in the figure shows Figure 5dThe fluorescence form of the tertiary lymphatic structure identified in the image is different from the true fluorescence identification form of the tertiary lymphatic structure, that is, this area is a false positive tertiary lymphatic structure recognition result. Therefore, when further using the multiple fluorescence immunohistochemistry image recognition filtering model to filter and identify the various tertiary lymphatic structure recognition results obtained by the multiple fluorescence immunohistochemistry image recognition model, Figure 5d The recognition area in is filtered out and the obtained Figure 9d An image in which no tertiary lymphatic structure is identified is considered to be a true positive tertiary lymphatic structure recognition result.

[0254] In this way, the reliability of the identification results of true positive tertiary lymphatic structures can be improved, providing analysts with reliable analysis data.

[0255] It is easy to understand that for the solution that does not obtain multiple fluorescent immunohistochemistry image blocks, the pre-trained multiple fluorescent immunohistochemistry image recognition filtering model can also be used for filtering. However, when filtering, the corresponding processing object is the second-resolution multiple fluorescent immunohistochemistry image that has not undergone image cutting. The specific method is similar and will not be repeated here.

[0256] Of course, in order to further ensure the reliability and accuracy of the recognition results, in one embodiment, the true positive tertiary lymphatic structure recognition results can be further screened by area. Figure 8 .

[0257] Step S306, determining whether area filtering is required, if yes, executing step S307, if not, executing step S310.

[0258] When it is determined to perform area filtering, the final recognition result can be directly subjected to area filtering, that is, step S307. When area filtering is not required, the number of true positive tertiary lymphatic structures identified as a whole can be directly obtained, that is, step S308.

[0259] Step S307 , performing area filtering on each of the true-positive tertiary lymphatic structure recognition results according to a preset area threshold of the tertiary lymphatic structure, removing the true-positive tertiary lymphatic structure recognition results whose areas are smaller than the area threshold, and obtaining the area true-positive tertiary lymphatic structure recognition results.

[0260] It is easy to understand that the area is the area value of the general overall structure of the tertiary lymphatic structure.

[0261] Since the identified true-positive tertiary lymphatic structure recognition result is obtained based on the tertiary lymphatic structure recognition result, only the tertiary lymphatic structure recognition results that do not meet the requirements in the fluorescent identification form within each tertiary lymphatic structure recognition result are filtered out, and the contour area of ​​the identified true-positive tertiary lymphatic structure recognition result and the contour area of ​​the tertiary lymphatic structure recognition result do not change, so the preset area threshold of the tertiary lymphatic structure can continue to be used as the basis for area filtering.

[0262] In this way, the accuracy of the obtained recognition results can be further ensured.

[0263] Step S308, determining the number of true positive tertiary lymphatic structure recognition results for each area of ​​each second-resolution multiple fluorescence immunohistochemistry image block, and obtaining the number of true positive tertiary lymphatic structure recognition results for the entire area of ​​the second-resolution multiple fluorescence immunohistochemistry image to be detected.

[0264] After area filtering, the number of true-positive tertiary lymphatic structure identification results based on the overall area can be counted, allowing analysts to directly determine the number of tertiary lymphatic structures and thus determine the prognostic effect.

[0265] Step S309 , obtaining the contour coordinates of each area true positive tertiary lymphatic structure recognition result of each second resolution multiplex fluorescence immunohistochemistry image block, and connecting each of the contour coordinates to form editable areas of each true positive tertiary lymphatic structure.

[0266] For details of step S309, please refer to Figure 6 The step S35 shown is not repeated here.

[0267] On the one hand, it can facilitate analysts to directly analyze and interpret statistical results; on the other hand, it can improve the accuracy of statistical results, thereby providing analysts with a reliable data basis.

[0268] When it is determined that area filtering is not required, in order to facilitate the pathological diagnosis by the analyst, the number of true positive identification results of each marker may be counted, that is, step S310.

[0269] Step S310, determining the number of each identified true positive tertiary lymphatic structure recognition result of each second-resolution multiple fluorescence immunohistochemistry image block, and obtaining the number of the overall identified true positive tertiary lymphatic structure recognition results of the second-resolution multiple fluorescence immunohistochemistry image to be detected.

[0270] Step S311 , obtaining the contour coordinates of each identified true positive tertiary lymphatic structure of each second resolution multiplex fluorescence immunohistochemistry image block, and connecting each of the contour coordinates to form editable identified true positive tertiary lymphatic structures.

[0271] In this way, analysts can easily analyze and judge the identified true positive tertiary lymphatic structures in each area to obtain more accurate clinical diagnosis results.

[0272] It is easy to understand that the execution order of step S305 and step S306 can be adjusted as needed. Area filtering can be performed first, and then filtering based on the pathological region identification principle can be performed.

[0273] In order to accurately identify the multiple fluorescent immunohistochemical images to be detected, it is necessary to pre-train a usable multiple fluorescent immunohistochemical image recognition model. In a specific embodiment, please refer to Figure 10 , Figure 10 This is a schematic diagram of a training process of a multiple fluorescent immunohistochemistry image recognition model in the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0274] As shown in the figure, the process of training a multiplex fluorescent immunohistochemistry image recognition model may include:

[0275] Step S40: Acquire a first resolution image set.

[0276] It should be noted that the first-resolution image set includes various first-resolution training multiple fluorescent immunohistochemistry image blocks, and each of the first-resolution training multiple fluorescent immunohistochemistry image blocks has a three-level lymphatic structure training annotation.

[0277] For details, please refer to Figure 11 , Figure 11 This is a schematic diagram of a state of labeling a first-resolution image set in the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0278] The three-level lymphatic structure training annotation can be manually annotated. In order to ensure the recognition performance of the constructed multiple fluorescent immunohistochemistry image recognition model, in one embodiment, Figure 11 As shown, the first resolution training multiple fluorescent immunohistochemistry image blocks can be used to annotate the tertiary lymphatic structure and background, enriching the learning samples of the multiple fluorescent immunohistochemistry image recognition model, so that the multiple fluorescent immunohistochemistry image recognition model has better effect.

[0279] In some embodiments, the step of acquiring the first resolution image set includes:

[0280] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0281] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0282] Of course, the original multiplex fluorescent immunohistochemical images for training are also obtained by digitally scanning the original multiplex fluorescent immunohistochemical tissue samples.

[0283] Of course, the image cutting method may include overlapping cutting.

[0284] After obtaining the first-resolution image set, in order to ensure the training effect, in one embodiment, a part of the first-resolution image set can be selected as the first-resolution training set, and the other part can be selected as the first-resolution verification set.

[0285] For example, the first-resolution image set can be divided into a first-resolution training set and a first-resolution verification set according to a ratio of 8:2 or 7:3. It is easy to understand that in order to ensure the training effect, the number of images in the first-resolution training set must be greater than the number of images in the first-resolution verification set.

[0286] By dividing the first resolution image set into a predetermined ratio, the output stability of the multiple fluorescent immunohistochemistry image recognition model can be ensured on the basis of being able to train the multiple fluorescent immunohistochemistry image recognition model.

[0287] In some embodiments, in order to quickly obtain a large number of image blocks in the first resolution image set, each training first resolution training multiple fluorescence immunohistochemistry pathology image may be horizontally flipped, rotated, transformed in color space, and horizontally translated.

[0288] In this way, a sufficient number of first-resolution multiplex fluorescent immunohistochemistry image blocks with different contents can be quickly obtained.

[0289] Of course, when the number of multiplex fluorescent immunohistochemical images is sufficient, it is not necessary to perform image amplification on each of the labeled multiplex fluorescent immunohistochemical images with the first resolution.

[0290] Step S41 , using the multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure prediction on each of the first resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining each three-level lymphatic structure training prediction result of each of the first resolution training multiple fluorescence immunohistochemistry image blocks.

[0291] Step S42: determine whether the difference between the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold. If yes, execute step S43; otherwise, execute step S44.

[0292] If the difference between the three-level lymphatic structure training prediction result and the three-level lymphatic structure training annotation meets the prediction deviation threshold, that is, the content contained in the three-level lymphatic structure training prediction result is the same as the content of the three-level lymphatic structure training annotation, and it contains an area that conforms to the actual three-level lymphatic structure conventional outline, then the multiple fluorescent immunohistochemistry image recognition model is a trained model, that is, step S43.

[0293] Step S43: obtaining the trained multiple fluorescence immunohistochemistry image recognition model.

[0294] If the prediction deviation threshold is not met, it means that the recognition result of the multiple fluorescent immunohistochemistry image recognition model is inaccurate. At this time, the multiple fluorescent immunohistochemistry image recognition model needs to be further adjusted, that is, step S44.

[0295] Step S44: adjusting parameters of the multiplex fluorescence immunohistochemistry image recognition model.

[0296] The trained multiple fluorescent immunohistochemistry image recognition model is obtained by adjusting parameters of the multiple fluorescent immunohistochemistry image recognition model and continuing to execute step S41 until the prediction deviation threshold is met.

[0297] The prediction deviation threshold can be determined by the difference between the number of the three-level lymphatic structure training prediction results identified in all the first-resolution image set and the number of all the three-level lymphatic structure training annotations marked in the first-resolution image set, or it can be determined by the ratio of the number of the three-level lymphatic structure prediction results identified in all the first-resolution training multiple fluorescent immunohistochemistry image blocks in the first-resolution image set to the number of the three-level lymphatic structure training annotations marked in all the first-resolution training multiple fluorescent immunohistochemistry image blocks.

[0298] In this way, the reliability of the output recognition results can be ensured on the basis of improving the recognition efficiency of the multiple fluorescent immunohistochemical images to be detected.

[0299] In addition, it should be noted that if the multiple fluorescent immunohistochemistry image recognition method provided in the embodiment of the present application directly identifies each tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected, then the first resolution image set used in the training of the multiple fluorescent immunohistochemistry image recognition model includes each first resolution training multiple fluorescent immunohistochemistry image, and there is no need for image cutting.

[0300] When further processing the three-level lymphatic structures, a multiple fluorescent immunohistochemical image recognition and filtering model is required. In order to obtain accurate recognition and filtering results, in one embodiment, the multiple fluorescent immunohistochemical image recognition and filtering model can be trained. For details, please refer to Figure 12 , Figure 12 This is a schematic diagram of a training process of a multiple fluorescent immunohistochemistry image recognition filtering model in the multiple fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0301] As shown in the figure, the process may include the following steps:

[0302] Step S50: Acquire a second resolution image set.

[0303] Of course, the second-resolution image set includes each second-resolution training multiple fluorescent immunohistochemistry image block, and each second-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation.

[0304] Similarly, the second resolution training multiple fluorescent immunohistochemistry image blocks can be used to annotate the tertiary lymphatic structure and background, enriching the learning samples of the multiple fluorescent immunohistochemistry image recognition and filtering model, so that the multiple fluorescent immunohistochemistry image recognition and filtering model is more effective.

[0305] Of course, in order to achieve further identification and filtering of each of the tertiary lymphatic structures, the images in the second resolution image set need to be subjected to the same fluorescence processing as the images in the first resolution image set, that is, the CD3 and CD20 fluorescence channels are also used for fluorescence processing.

[0306] In some specific embodiments, the step of acquiring the second resolution image set includes:

[0307] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0308] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0309] To facilitate understanding of the display results of multiple fluorescent immunohistochemistry images obtained at different resolutions, please refer to Figure 13 and Figure 14 , Figure 13is a schematic diagram of a first-resolution image set obtained at a first resolution in the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application, Figure 14 Schematic diagram of a second-resolution image set obtained at a second resolution in the multiplex fluorescent immunohistochemistry image recognition method provided in an embodiment of the present application.

[0310] As shown in the figure, the image set obtained at the second resolution has a better display effect than the image set at the first resolution, which facilitates subsequent recognition and filtering of pathological areas.

[0311] Of course, the image cutting method can be overlapping cutting.

[0312] It should be noted that in order to be able to further filter and identify the output results of the multiple fluorescent immunohistochemistry image recognition model, namely, each tertiary lymphatic structure, it is necessary to keep the original multiple fluorescent immunohistochemistry images trained by obtaining the second resolution training image set consistent with the original multiple fluorescent immunohistochemistry images trained by obtaining the first resolution image set.

[0313] By annotating the background and tertiary lymphatic structures of the original multiple fluorescent immunohistochemistry images for training, rich training samples can be provided for the training of the multiple fluorescent immunohistochemistry image recognition and filtering model, making the training effect of the multiple fluorescent immunohistochemistry image recognition and filtering model better.

[0314] After obtaining the second resolution image set, in order to ensure the training effect, in one embodiment, a part of the second resolution image set can be selected as the second resolution training set, and the other part can be selected as the second resolution verification set.

[0315] For example, the second resolution image set can be divided into a second resolution training set and a second resolution verification set according to the ratio of 8:2 or 7:3. It is easy to understand that in order to ensure the training effect, the number of images in the second resolution training set must be greater than the number of images in the second resolution verification set.

[0316] By dividing the second resolution image set into a predetermined proportion, the output stability of the multiple fluorescent immunohistochemistry image recognition and filtering model can be ensured on the basis of being able to train the multiple fluorescent immunohistochemistry image recognition and filtering model.

[0317] In some embodiments, a sufficient number of second-resolution multiple fluorescent immunohistochemistry image blocks with different contents can be quickly obtained by image horizontal flipping, image rotation, image color space conversion, and image horizontal translation.

[0318] Step S51, using the multiple fluorescence immunohistochemistry image recognition filtering model to filter the second resolution tertiary lymphatic structures of each second resolution training multiple fluorescence immunohistochemistry image block, to obtain each tertiary lymphatic structure training filtering result of each second resolution training multiple fluorescence immunohistochemistry image block.

[0319] Step S52: Determine whether the deviation between the three-level lymphatic structure training filtering result and the three-level lymphatic structure training annotation meets the filtering deviation threshold. If so, execute step S53; if not, execute step S54.

[0320] If the filtering deviation threshold is met, that is, the content contained in each image in the three-level lymphatic structure training filtering result is the same as the content identified by the three-level lymphatic structure training annotation, and it contains an area that conforms to the regular outline and fluorescence form of the actual three-level lymphatic structure, then the multiple fluorescence immunohistochemistry image recognition filtering model is a trained model, that is, step S53.

[0321] Step S53: obtaining the trained multiple fluorescence immunohistochemistry image recognition filtering model.

[0322] If the filtering deviation threshold is not met, it means that the recognition and filtering result of the multiple fluorescent immunohistochemistry image recognition and filtering model is inaccurate. At this time, the pathological image recognition and filtering model needs to be further adjusted, that is, step S54.

[0323] Step S54 , adjusting parameters of the multiplex fluorescent immunohistochemistry image recognition and filtering model.

[0324] When the filtering deviation threshold is not met, the parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted, and step S51 is continued until the filtering deviation threshold is met, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

[0325] The constructed multiple fluorescent immunohistochemistry image recognition and filtering model is used to identify and filter the second-resolution training multiple fluorescent immunohistochemistry image blocks in each second-resolution image set, and the parameters of the multiple fluorescent immunohistochemistry image recognition and filtering model are adjusted according to the training output results until the filtering deviation threshold is met, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition and filtering model.

[0326] The filtering deviation threshold can refer to the setting of the prediction deviation threshold in the multiple fluorescent immunohistochemistry image recognition model, which will not be repeated here.

[0327] In this way, the accuracy of recognition of the multiple fluorescent immunohistochemical images to be detected can be improved, and the reliability of the recognition results finally output can be ensured.

[0328] To address the aforementioned issues, the present invention also provides a multiplex fluorescence immunohistochemistry image recognition device. This device can be considered the functional module required to implement the multiplex fluorescence immunohistochemistry image recognition method provided in the present invention. The device described below can be used in conjunction with the method described above.

[0329] As an optional implementation, Figure 15 This is an optional block diagram of the multiple fluorescence immunohistochemistry image recognition device provided in the embodiment of the present application. Figure 15 As shown, the device is suitable for identifying multiple fluorescent immunohistochemical images to be detected, and may include:

[0330] The multiplex fluorescent immunohistochemical image acquisition module 800 is adapted to acquire the multiplex fluorescent immunohistochemical image to be detected, where the resolution of the multiplex fluorescent immunohistochemical image to be detected is a first resolution.

[0331] In some embodiments, the multiplex fluorescent immunohistochemistry image acquisition module 800 is adapted to acquire the multiplex fluorescent immunohistochemistry image to be detected, including:

[0332] Obtain original multiplex fluorescence immunohistochemistry tissue samples;

[0333] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image;

[0334] The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

[0335] The multiple fluorescent immunohistochemistry image recognition module 801 is suitable for using a pre-trained multiple fluorescent immunohistochemistry image recognition model to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemistry image to be detected, and obtain the recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition model is equal to the first resolution.

[0336] It can be seen that the multiple fluorescent immunohistochemical image recognition device provided by the embodiment of the present invention first obtains a multiple fluorescent immunohistochemical image to be detected with a resolution of a first resolution, and then uses a pre-trained multiple fluorescent immunohistochemical image recognition model to identify the tertiary lymphatic structure of each of the multiple fluorescent immunohistochemical images to obtain the recognition results of each tertiary lymphatic structure of each of the multiple fluorescent immunohistochemical images. Of course, the resolution of the training multiple fluorescent immunohistochemical image used to train the multiple fluorescent immunohistochemical image recognition model is equal to the first resolution. It can be seen that the multiple fluorescent immunohistochemistry image recognition device provided in the embodiment of the present invention realizes automatic recognition of the multiple fluorescent immunohistochemistry image to be detected by using a multiple fluorescent immunohistochemistry image recognition model by ensuring the resolution of the multiple fluorescent immunohistochemistry image to be detected. The multiple fluorescent immunohistochemistry image recognition model can unify the recognition standards of the tertiary lymphatic structure, and the multiple fluorescent immunohistochemistry image recognition model is pre-trained, which can ensure the accuracy of recognition. Therefore, when actually identifying the tertiary lymphatic structure, the multiple fluorescent immunohistochemistry image recognition model can directly obtain the various tertiary lymphatic structures contained in the multiple fluorescent immunohistochemistry image to be detected according to the recognition standards, thereby reducing dependence on analysts, and can also assist analysts in quickly identifying and interpreting the tertiary lymphatic structures in the multiple fluorescent immunohistochemistry image to be detected, thereby improving analysis efficiency.

[0337] In some embodiments, in order to obtain more accurate identification results, the multiple fluorescent immunohistochemical images to be detected can also be processed. Figure 16 , Figure 16 This is another optional block diagram of the multiplex fluorescence immunohistochemistry image recognition device provided in an embodiment of the present application.

[0338] like Figure 16 As shown, the multiple fluorescence immunohistochemistry image recognition device may further include:

[0339] Multiplex fluorescence immunohistochemistry image acquisition module 800.

[0340] The multiple fluorescent immunohistochemistry image processing module 901 is adapted to perform image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks;

[0341] Multiplex fluorescent immunohistochemistry image recognition module 801.

[0342] At this time, the multiple fluorescence immunohistochemistry image recognition module 801 can identify the tertiary lymphatic structure on each multiple fluorescence immunohistochemistry image block.

[0343] In some embodiments, the multiple fluorescent immunohistochemistry image recognition model of the multiple fluorescent immunohistochemistry image recognition module 801 is obtained by a multiple fluorescent immunohistochemistry image recognition model training module, and the multiple fluorescent immunohistochemistry image recognition model training module is suitable for:

[0344] Acquire a first-resolution image set, wherein the first-resolution image set includes first-resolution training multiple fluorescent immunohistochemistry image blocks, and each first-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation;

[0345] Using the multiple fluorescence immunohistochemistry image recognition model, predicting the three-level lymphatic structure of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining the three-level lymphatic structure training prediction results of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks;

[0346] The parameters of the multiple fluorescent immunohistochemistry image recognition model are adjusted according to the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition model.

[0347] Of course, when the multiple fluorescent immunohistochemistry image recognition model is used to recognize the entire multiple fluorescent immunohistochemistry image to be detected, the first resolution training set of the acquired first resolution image set contains the unsegmented complete first resolution training multiple fluorescent immunohistochemistry images.

[0348] In some embodiments, the first-resolution image set is obtained by a first-resolution image acquisition module, which is adapted to:

[0349] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0350] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0351] The overall tertiary lymphatic structure determination module 902 is adapted to determine the number of each tertiary lymphatic structure in each of the multiple fluorescent immunohistochemical image blocks, and obtain the number of overall tertiary lymphatic structure recognition results of the multiple fluorescent immunohistochemical image to be detected.

[0352] At this time, the pathological area is the result of the third-level lymphatic structure recognition, and the total number of the third-level lymphatic structure recognition results is counted.

[0353] The tertiary lymphatic structure editing module 903 is adapted to obtain the contour coordinates of each tertiary lymphatic structure recognition result of each of the multiple fluorescent immunohistochemistry image blocks, and connect each of the contour coordinates to form editable tertiary lymphatic structures.

[0354] At this time, the contour coordinates of the tertiary lymphatic structure recognition results are edited.

[0355] In other embodiments, in order to improve the accuracy of the identification results of the three-level lymphatic structures, each three-level lymphatic structure can be further processed. For details, please continue to refer to Figure 16 .

[0356] As shown in the figure, the multiplex fluorescence immunohistochemistry image recognition device may further include:

[0357] The area filtering module 904 is adapted to perform area filtering on each of the tertiary lymphatic structure recognition results according to a preset area threshold of the tertiary lymphatic structure, remove the tertiary lymphatic structure recognition results whose areas are smaller than the area threshold, and obtain a true positive area tertiary lymphatic structure recognition result, where the area is the area value of the tertiary lymphatic structure.

[0358] At this time, the area of ​​the tertiary lymphatic structure recognition results is filtered.

[0359] Of course, in one embodiment, the area true positive three-level lymphatic structure recognition results obtained after being processed by the area filtering module 904 can be further statistically analyzed and edited. For details, please continue to refer to Figure 16 .

[0360] As shown in the figure, it can also include:

[0361] The overall tertiary lymphatic structure determination module 902 is adapted to determine the number of each tertiary lymphatic structure recognition result of each of the multiple fluorescent immunohistochemistry image blocks, and obtain the number of overall tertiary lymphatic structure recognition results of the multiple fluorescent immunohistochemistry image to be detected.

[0362] At this time, the number of true positive tertiary lymphatic structure identification results by area is counted.

[0363] The tertiary lymphatic structure editing module 903 is adapted to obtain the contour coordinates of each tertiary lymphatic structure recognition result of each of the multiple fluorescent immunohistochemistry image blocks, and connect each of the contour coordinates to form editable tertiary lymphatic structure recognition results.

[0364] At this time, the contour coordinates of the area true positive tertiary lymphatic structure recognition results are edited.

[0365] In another embodiment, in order to improve the reliability of the recognition results, the recognition results of each three-level lymphatic structure can also be screened. For details, please refer to Figure 17 , Figure 17 This is another optional block diagram of the multiplex fluorescence immunohistochemistry image recognition device provided in an embodiment of the present application.

[0366] As shown in the figure, the device may include:

[0367] Multiple fluorescence immunohistochemistry image acquisition module 800;

[0368] Multiple fluorescence immunohistochemistry image processing module 901;

[0369] Multiplex fluorescent immunohistochemistry image recognition module 801.

[0370] The second-resolution multiple fluorescent immunohistochemistry image acquisition module 110 is suitable for acquiring the multiple fluorescent immunohistochemistry image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemistry image to be detected, wherein the second resolution is greater than the first resolution, and the second-resolution multiple fluorescent immunohistochemistry image to be detected includes the identified results of each tertiary lymphatic structure.

[0371] In some embodiments, the second-resolution multiple fluorescence immunohistochemistry image block acquisition module 110 is adapted to determine the corresponding second-resolution three-level lymphatic structure identification result in the second-resolution multiple fluorescence immunohistochemistry image to be detected according to each of the three-level lymphatic structure identification results, including:

[0372] Obtaining coordinate position information of each of the three-level lymphatic structure identification results;

[0373] The coordinate position information is converted to a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution, and the third-level lymphatic structure recognition result at the second resolution is determined.

[0374] In a specific embodiment, the second-resolution to-be-detected multiple fluorescent immunohistochemical image acquisition module 110 is adapted to acquire the second-resolution to-be-detected multiple fluorescent immunohistochemical image corresponding to the to-be-detected multiple fluorescent immunohistochemical image, including:

[0375] Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected;

[0376] digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical image;

[0377] Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

[0378] The multiple fluorescence immunohistochemistry image recognition module 801 is adapted to obtain each tertiary lymphatic structure of each multiple fluorescence immunohistochemistry image block, and includes:

[0379] a second-resolution multiple fluorescent immunohistochemistry image block acquisition module 111 adapted to determine, based on the position of each of the three-level lymphatic structure recognition results obtained at the first resolution in the second-resolution multiple fluorescent immunohistochemistry image to be detected, a corresponding second-resolution three-level lymphatic structure recognition result, and perform second-resolution image segmentation based on the second-resolution three-level lymphatic structure recognition result to obtain a second-resolution multiple fluorescent immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition result;

[0380] The multiple fluorescent immunohistochemistry image recognition and filtering module 112 is suitable for filtering each of the second-resolution three-level lymphatic structure recognition results using a pre-trained multiple fluorescent immunohistochemistry image recognition and filtering model, removing the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle, and obtaining each of the second-resolution multiple fluorescent immunohistochemistry image blocks. The resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition and filtering model is equal to the second resolution.

[0381] In some embodiments, the multiple fluorescent immunohistochemistry image recognition and filtering model is obtained by a multiple fluorescent immunohistochemistry image recognition and filtering training module, and the multiple fluorescent immunohistochemistry image recognition and filtering training module is suitable for:

[0382] Acquire a second-resolution image set, including each second-resolution training multiple fluorescent immunohistochemistry image block, each of which is provided with a third-level lymphatic structure training annotation;

[0383] Filtering the second-resolution tertiary lymphatic structures of each second-resolution training multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition filtering model to obtain training filtering results of each tertiary lymphatic structure of each second-resolution training multiple fluorescent immunohistochemistry image block;

[0384] The parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted according to the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations meets the filtering deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

[0385] In some embodiments, the second resolution image set is obtained by a second resolution image acquisition module, and the second resolution image acquisition module is adapted to:

[0386] Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image;

[0387] Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

[0388] After obtaining the true positive tertiary lymphatic structure recognition results for each marker, statistics and editing can be performed on them. Please continue to refer to Figure 17 .

[0389] As shown in the figure, it can also include:

[0390] In the overall tertiary lymphatic structure determination module 902 , at this time, the pathological region is the result of the true positive tertiary lymphatic structure identification, and the total number of the true positive tertiary lymphatic structure identification results is counted.

[0391] The third-level lymphatic structure editing module 903 edits the contour coordinates of the true positive third-level lymphatic structure recognition result.

[0392] Of course, in some embodiments, before performing statistics and editing on each of the true positive three-level lymphatic structure recognition results, each of the true positive three-level lymphatic structure recognition results can also be subjected to area filtering. For details, please continue to refer to Figure 17 .

[0393] As shown in the figure, the device may also include:

[0394] The area filtering module 904 performs area filtering on the areas that identify the true positive tertiary lymphatic structure recognition results.

[0395] An embodiment of the present application further provides an electronic device, comprising at least one memory 100 and at least one processor 101; the memory 100 stores a program, and the processor 101 calls the program to execute the variation interpretation acquisition method as described in any of the aforementioned embodiments.

[0396] like Figure 18 As shown, Figure 18 Schematic diagram of an electronic device provided in an embodiment of the present application.

[0397] It can be understood that the device may also include at least one communication interface 102 and at least one communication bus 103; the processor 101 and the memory 100 may be located in the same electronic device, for example, the processor 100 and the memory 101 may be located in a service unit device or a terminal device; the processor 100 and the memory 101 may also be located in different electronic devices.

[0398] In the embodiment provided in the present application, the number of the processor 101, the communication interface 102, the memory 100, and the communication bus 103 is at least one, and the processor 101, the communication interface 102, and the memory 100 communicate with each other through the communication bus 103; obviously, Figure 10 The communication connection diagram of the processor 101, the communication interface 102, the memory 100 and the communication bus 103 shown is only an optional manner.

[0399] Optionally, the communication interface 102 can be an interface of a communication module, such as an interface of a GSM module; the processor 101 can be a central processing unit CPU, or a specific integrated circuit ASIC, or one or more integrated circuits configured to implement the embodiments of the present application; the memory 100 can include a high-speed RAM memory, or can include a non-volatile memory, such as at least one disk storage.

[0400] It should be noted that the above-mentioned device may also include other devices (not shown) that may not be necessary for understanding the contents disclosed in the embodiments of the present invention; since these other devices may not be necessary for understanding the contents disclosed in the embodiments of the present invention, the embodiments of the present invention will not introduce them one by one.

[0401] An embodiment of the present application further provides a storage medium storing a program suitable for recognizing multiple fluorescent immunohistochemical images, so as to implement the multiple fluorescent immunohistochemical image recognition method as described in any of the aforementioned embodiments.

[0402] Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A multiplex fluorescence immunohistochemistry image recognition method, characterized in that: include: Acquiring a multiplex fluorescent immunohistochemistry image to be detected, wherein the resolution of the multiplex fluorescent immunohistochemistry image to be detected is a first resolution; A pre-trained multiple fluorescent immunohistochemistry image recognition model is used to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemistry image to be detected, and each tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemistry image to be detected is obtained. The multiple fluorescent immunohistochemistry image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemistry image to be detected is obtained, wherein the second resolution is greater than the first resolution, and the multiple fluorescent immunohistochemistry image to be detected at the second resolution includes each identified tertiary lymphatic structure recognition result. According to the position of each tertiary lymphatic structure recognition result obtained at the first resolution in the multiple fluorescent immunohistochemistry image to be detected at the second resolution, the corresponding second resolution tertiary lymphatic structure recognition result is determined, wherein the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition model is equal to the first resolution.

2. The multiplex fluorescent immunohistochemistry image recognition method according to claim 1, wherein: The step of obtaining the tertiary lymphatic structure identification result of the multiple fluorescent immunohistochemistry image to be detected further includes: Performing image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks; The step of using a pre-trained multiple fluorescent immunohistochemical image recognition model to perform three-level lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected to obtain the recognition results of each three-level lymphatic structure of the multiple fluorescent immunohistochemical image to be detected includes: Using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain each three-level lymphatic structure recognition result of each of the multiple fluorescence immunohistochemistry image blocks; The tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemical image to be detected is obtained according to each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemical image block.

3. The multiplex fluorescent immunohistochemistry image recognition method according to claim 2, wherein: After the step of obtaining the identification results of each of the three-level lymphatic structures in each of the multiple fluorescent immunohistochemistry image blocks, the method further includes: The area of ​​each of the tertiary lymphatic structure recognition results is filtered according to a preset area threshold of the tertiary lymphatic structure, and the tertiary lymphatic structure recognition results with area values ​​less than the area threshold are removed to obtain the area true positive tertiary lymphatic structure recognition results.

4. The multiplex fluorescent immunohistochemistry image recognition method according to claim 2, wherein: After determining the corresponding second-resolution third-level lymphatic structure recognition result, the following steps are also included: Performing second-resolution image segmentation according to the second-resolution three-level lymphatic structure recognition result to obtain a second-resolution multiple fluorescent immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition result; A pre-trained multiple fluorescence immunohistochemistry image recognition filtering model is used to filter each of the second-resolution three-level lymphatic structure recognition results, and the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle are removed to obtain each of the second-resolution multiple fluorescence immunohistochemistry image blocks. The resolution of the multiple fluorescence immunohistochemistry image used for training the multiple fluorescence immunohistochemistry image recognition filtering model is equal to the second resolution.

5. The multiplex fluorescent immunohistochemistry image recognition method according to claim 4, wherein: The step of determining the corresponding second-resolution third-level lymphatic structure recognition result in the second-resolution multiple fluorescence immunohistochemistry image to be detected according to each of the third-level lymphatic structure recognition results comprises: Obtaining coordinate position information of each of the three-level lymphatic structure identification results; The coordinate position information is converted into a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution to determine the pathological area at the second resolution.

6. The multiplex fluorescent immunohistochemistry image recognition method according to claim 4, wherein: The step of acquiring the second-resolution multiple fluorescent immunohistochemistry image to be detected corresponding to the multiple fluorescent immunohistochemistry image to be detected comprises: Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected; digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical image; Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

7. The multiplex fluorescent immunohistochemistry image recognition method according to claim 2, wherein: After the step of obtaining the identification results of each of the three-level lymphatic structures in each of the multiple fluorescent immunohistochemistry image blocks, the method further includes: The contour resolution coordinates included in each of the three-level lymphatic structure recognition results of each of the multiple fluorescent immunohistochemistry image blocks are obtained, and the contour resolution coordinates are connected to form editable three-level lymphatic structure recognition results.

8. The multiplex fluorescent immunohistochemistry image recognition method according to any one of claims 2 to 7, wherein: The image cutting includes overlapping cutting.

9. The multiplex fluorescent immunohistochemistry image recognition method according to any one of claims 1 to 6, wherein: The step of acquiring the multiple fluorescent immunohistochemical images to be detected includes: Obtain original multiplex fluorescence immunohistochemistry tissue samples; digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image; The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

10. The multiplex fluorescent immunohistochemistry image recognition method according to claim 2, wherein: The training steps of the multiple fluorescent immunohistochemistry image recognition model include: Acquire a first-resolution image set, wherein the first-resolution image set includes first-resolution training multiple fluorescent immunohistochemistry image blocks, and each first-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation; Using the multiple fluorescence immunohistochemistry image recognition model, predicting the three-level lymphatic structure of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining the three-level lymphatic structure training prediction results of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks; The parameters of the multiple fluorescent immunohistochemistry image recognition model are adjusted according to the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition model.

11. The multiplex fluorescent immunohistochemistry image recognition method according to claim 10, wherein: The step of acquiring the first-resolution image set includes: Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image; Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

12. The multiplex fluorescent immunohistochemistry image recognition method according to claim 4, wherein: The training steps of the multiple fluorescent immunohistochemistry image recognition filtering model include: Acquire a second-resolution image set, including each second-resolution training multiple fluorescent immunohistochemistry image block, each of which is provided with a third-level lymphatic structure training annotation; Filtering the second-resolution tertiary lymphatic structures of each second-resolution training multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition filtering model to obtain training filtering results of each tertiary lymphatic structure of each second-resolution training multiple fluorescent immunohistochemistry image block; The parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted according to the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations meets the filtering deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

13. The multiplex fluorescent immunohistochemistry image recognition method according to claim 12, wherein: The step of acquiring the second-resolution image set includes: Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image; Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

14. A multiplex fluorescence immunohistochemistry image recognition device, characterized in that: include: a multiple fluorescent immunohistochemical image acquisition module, adapted to acquire a multiple fluorescent immunohistochemical image to be detected, wherein the resolution of the multiple fluorescent immunohistochemical image to be detected is a first resolution; a multiple fluorescent immunohistochemical image recognition module, adapted to perform tertiary lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected using a pre-trained multiple fluorescent immunohistochemical image recognition model, and obtain recognition results of each tertiary lymphatic structure of the multiple fluorescent immunohistochemical image to be detected; A second-resolution multiple fluorescent immunohistochemical image acquisition module to be detected is suitable for acquiring a multiple fluorescent immunohistochemical image to be detected at a second resolution corresponding to the multiple fluorescent immunohistochemical image to be detected, wherein the second resolution is greater than the first resolution, and the multiple fluorescent immunohistochemical image to be detected at the second resolution includes the identified results of each three-level lymphatic structure; a second-resolution multiple fluorescent immunohistochemical image block acquisition module is suitable for determining the corresponding second-resolution three-level lymphatic structure identification result according to the position of each three-level lymphatic structure identification result obtained at the first resolution in the multiple fluorescent immunohistochemical image to be detected at the second resolution, wherein the resolution of the multiple fluorescent immunohistochemical image used for training the multiple fluorescent immunohistochemical image recognition model is equal to the first resolution.

15. The multiplex fluorescent immunohistochemistry image recognition device according to claim 14, wherein: Also includes: a multiple fluorescent immunohistochemistry image processing module, adapted to perform image segmentation on the multiple fluorescent immunohistochemistry image to be detected to obtain multiple fluorescent immunohistochemistry image blocks; The multiple fluorescent immunohistochemical image recognition module is adapted to perform three-level lymphatic structure recognition on the multiple fluorescent immunohistochemical image to be detected using a pre-trained multiple fluorescent immunohistochemical image recognition model, and obtain recognition results of each three-level lymphatic structure of the multiple fluorescent immunohistochemical image to be detected, including: Using a pre-trained multiple fluorescence immunohistochemistry image recognition model to perform three-level lymphatic structure recognition on each of the multiple fluorescence immunohistochemistry image blocks, to obtain each three-level lymphatic structure recognition result of each of the multiple fluorescence immunohistochemistry image blocks; The tertiary lymphatic structure recognition result of the multiple fluorescent immunohistochemical image to be detected is obtained according to each tertiary lymphatic structure recognition result of each multiple fluorescent immunohistochemical image block.

16. The multiplex fluorescent immunohistochemistry image recognition device according to claim 15, wherein: Also includes: The area filtering module is adapted to perform area filtering on each of the three-level lymphatic structure recognition results according to a preset area threshold of the three-level lymphatic structure, remove the three-level lymphatic structure recognition results whose area values ​​are less than the area threshold, and obtain the three-level lymphatic structure recognition results with true positive area.

17. The multiplex fluorescent immunohistochemistry image recognition device according to claim 15, wherein: The second-resolution multiple fluorescent immunohistochemistry image block acquisition module is also suitable for performing second-resolution image cutting according to the second-resolution three-level lymphatic structure recognition result to obtain a second-resolution multiple fluorescent immunohistochemistry image block containing the second-resolution three-level lymphatic structure recognition result; the multiple fluorescent immunohistochemistry image recognition device also includes a multiple fluorescent immunohistochemistry image recognition filtering module, which is suitable for using a pre-trained multiple fluorescent immunohistochemistry image recognition filtering model to filter each second-resolution three-level lymphatic structure recognition result, remove the corresponding three-level lymphatic structure recognition results that do not meet the pathological area identification principle, and obtain each identified true positive three-level lymphatic structure recognition result of each second-resolution multiple fluorescent immunohistochemistry image block, and the resolution of the multiple fluorescent immunohistochemistry image used for training the multiple fluorescent immunohistochemistry image recognition filtering model is equal to the second resolution.

18. The multiplex fluorescent immunohistochemistry image recognition device according to claim 17, wherein: The second-resolution multiple fluorescence immunohistochemistry image block acquisition module is adapted to determine the corresponding second-resolution three-level lymphatic structure identification result in the second-resolution multiple fluorescence immunohistochemistry image to be detected according to each of the three-level lymphatic structure identification results, including: Obtaining coordinate position information of each of the three-level lymphatic structure identification results; The coordinate position information is converted to a corresponding position of the multiple fluorescent immunohistochemistry image to be detected at the second resolution, and the third-level lymphatic structure recognition result at the second resolution is determined.

19. The multiplex fluorescent immunohistochemistry image recognition device according to claim 17, wherein: The second-resolution multiple fluorescent immunohistochemical image acquisition module is adapted to acquire the second-resolution multiple fluorescent immunohistochemical image corresponding to the multiple fluorescent immunohistochemical image, and includes: Acquiring an original multiple fluorescent immunohistochemical tissue sample corresponding to the multiple fluorescent immunohistochemical image to be detected; digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a second-resolution multiple fluorescent immunohistochemical tissue sample; Exposure adjustment is performed on the second-resolution multiplex fluorescent immunohistochemistry image to obtain the second-resolution multiplex fluorescent immunohistochemistry image to be detected corresponding to the multiplex fluorescent immunohistochemistry image to be detected.

20. The multiplex fluorescent immunohistochemistry image recognition device according to claim 15, wherein: Also includes: The pathological region editing module is adapted to obtain the contour coordinates of each tertiary lymphatic structure recognition result of each of the multiple fluorescent immunohistochemistry image blocks, and connect each of the contour coordinates to form editable tertiary lymphatic structure recognition results.

21. The multiplex fluorescent immunohistochemistry image recognition device according to any one of claims 15 to 20, characterized in that: The image cutting includes overlapping cutting.

22. The multiplex fluorescent immunohistochemistry image recognition device according to any one of claims 14 to 19, wherein: The multiplex fluorescent immunohistochemistry image acquisition module is suitable for acquiring multiplex fluorescent immunohistochemistry images to be detected, and includes: Obtain original multiplex fluorescence immunohistochemistry tissue samples; digitally scanning the original multiple fluorescent immunohistochemical tissue sample to obtain a first-resolution multiple fluorescent immunohistochemical image; The exposure of the first-resolution multiplex fluorescent immunohistochemistry image is adjusted to obtain the multiplex fluorescent immunohistochemistry image to be detected.

23. The multiplex fluorescent immunohistochemistry image recognition device according to claim 15, wherein: The multiple fluorescent immunohistochemistry image recognition model is obtained by a multiple fluorescent immunohistochemistry image recognition model training module, and the multiple fluorescent immunohistochemistry image recognition model training module is suitable for: Acquire a first-resolution image set, wherein the first-resolution image set includes first-resolution training multiple fluorescent immunohistochemistry image blocks, and each first-resolution training multiple fluorescent immunohistochemistry image block has a three-level lymphatic structure training annotation; Using the multiple fluorescence immunohistochemistry image recognition model, predicting the three-level lymphatic structure of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks, and obtaining the three-level lymphatic structure training prediction results of each of the first-resolution training multiple fluorescence immunohistochemistry image blocks; The parameters of the multiple fluorescent immunohistochemistry image recognition model are adjusted according to the three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training prediction results and the three-level lymphatic structure training annotations meets the prediction deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition model.

24. The multiplex fluorescent immunohistochemistry image recognition device according to claim 23, wherein: The first-resolution image set is obtained by a first-resolution image acquisition module, which is adapted to: Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image; Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the first resolution to obtain the first resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

25. The multiplex fluorescent immunohistochemistry image recognition device according to claim 17, wherein: The multiple fluorescent immunohistochemistry image recognition and filtering model is obtained by a multiple fluorescent immunohistochemistry image recognition and filtering training module, and the multiple fluorescent immunohistochemistry image recognition and filtering training module is adapted to: obtain a second-resolution image set, including respective second-resolution training multiple fluorescent immunohistochemistry image blocks, each of which is provided with a three-level lymphatic structure training annotation; Filtering the second-resolution tertiary lymphatic structures of each second-resolution training multiple fluorescent immunohistochemistry image block using the multiple fluorescent immunohistochemistry image recognition filtering model to obtain training filtering results of each tertiary lymphatic structure of each second-resolution training multiple fluorescent immunohistochemistry image block; The parameters of the multiple fluorescent immunohistochemistry image recognition filtering model are adjusted according to the three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations until the deviation between the obtained three-level lymphatic structure training filtering results and the three-level lymphatic structure training annotations meets the filtering deviation threshold, thereby obtaining the trained multiple fluorescent immunohistochemistry image recognition filtering model.

26. The multiplex fluorescent immunohistochemistry image recognition device according to claim 25, wherein: The second-resolution image set is obtained by a second-resolution image acquisition module, and the second-resolution image acquisition module is adapted to: Acquiring an original training multiplex fluorescent immunohistochemistry image, and annotating the three-level lymphatic structure and background of the original training multiplex fluorescent immunohistochemistry image; Under the first fluorescent channel and the second fluorescent channel, the original multiple fluorescent immunohistochemistry image is cut according to the second resolution to obtain the second resolution image set, and the first fluorescent channel and the second fluorescent channel are used to perform fluorescent identification of the background and the tertiary lymphoid structure in the original multiple fluorescent immunohistochemistry image.

27. An electronic device, characterized in that: The system comprises at least one memory and at least one processor; the memory stores a program, and the processor calls the program to execute the multiple fluorescent immunohistochemistry image recognition method according to any one of claims 1 to 13.

28. A storage medium, characterized in that The storage medium stores a program suitable for recognizing multiple fluorescent immunohistochemical images, so as to implement the multiple fluorescent immunohistochemical image recognition method according to any one of claims 1 to 13.

Images