Method and system for tissue region image recognition based on tile division

Abstract

The present invention relates to the technical field of data processing, and discloses a method and system for tissue region image recognition based on tile division. The method comprises: performing tile division on a biological tissue image to be recognized to obtain a plurality of image tiles; inputting the plurality of image tiles into a region recognition model to obtain a plurality of tissue regions and corresponding region types; screening for at least one region combination from the plurality of tissue regions according to a preset type adjacency rule, the region combination comprising a plurality of mutually adjacent tissue regions; and inputting a region image corresponding to the region combination and the corresponding region types into a region boundary update model to obtain a corresponding region update result, the region update result being used for re-indicating region boundaries of different tissue regions in the region image. Therefore, the present invention can improve the accuracy and efficiency of tissue region recognition, and provide an accurate data basis for disease treatment or research.

Description

Tissue Region Image Recognition Method and System Based on Patch Segmentation Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for recognizing tissue regions based on tile division. Background Technology

[0002] With the development of image processing technology and the popularization of medical imaging technology, more and more biological tissue images are being used for disease research or treatment. How to effectively identify different types of tissue regions in biological tissue images has become a core technical problem. However, existing tissue region image recognition technologies generally only rely on algorithm models trained on labeled images for region identification, without further incorporating region connectivity features related to region type for more refined region edge determination. Therefore, their efficiency and accuracy in region identification are lacking. It is clear that existing technologies have shortcomings that urgently need to be addressed. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for tissue region image recognition based on patch division, which can make full use of the characteristics of the border region types to accurately identify the region boundary, improve the accuracy and efficiency of tissue region recognition, and provide an accurate data foundation for disease treatment or research.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses a method for tissue region image recognition based on patch segmentation, the method comprising:

[0005] The biological tissue image to be identified is divided into multiple image patches;

[0006] The multiple image blocks are input into the region recognition model to obtain multiple tissue regions and their corresponding region types;

[0007] According to a preset type-adjacency rule, at least one region combination is selected from the plurality of organizational regions; the region combination includes a plurality of mutually adjacent organizational regions;

[0008] The region image corresponding to the region combination and the corresponding region type are input into the region boundary update model to obtain the corresponding region update result; the region update result is used to re-indicate the region boundary of different tissue regions in the region image.

[0009] As an optional implementation, in the first aspect of the invention, the region identification model is trained through the following steps:

[0010] Determine the pre-defined type region segmentation labels corresponding to the training tissue images;

[0011] The training tissue image is segmented to obtain multiple training image blocks;

[0012] For each training image block, calculate the intersection area between the training image block and any type of partitioning region in the type region partitioning annotation;

[0013] The region type corresponding to the type division region with the highest intersection area is determined as the type label corresponding to the training image block;

[0014] Based on all the training image patches and the corresponding type labels, the model of the preset CNN network architecture is trained iteratively. In each round of training, the average classification accuracy corresponding to each type label is calculated to obtain the training performance parameters; the classification accuracy is the F1 score or the F2 score.

[0015] The weights of the model obtained in the training round with the highest training performance parameters are determined as the final model weights, thus obtaining the region recognition model.

[0016] As an optional implementation, in the first aspect of the present invention, the step of inputting the plurality of image patches into a region recognition model to obtain a plurality of tissue regions and corresponding region types includes:

[0017] Each image block is input into the region recognition model to obtain the predicted region type corresponding to each image block;

[0018] All the image blocks are stitched together based on the segmentation rules to obtain a stitched tissue image;

[0019] Morphological operations are performed on the stitched tissue image to obtain a contour-smoothed image; the contour-smoothed image includes multiple tissue regions and corresponding region types; each tissue region includes multiple corresponding image blocks of the same predicted region type; the region type is the predicted region type corresponding to the image block in the tissue region;

[0020] For each tissue region in the contour smoothing image, identify the image block in the tissue region whose predicted region type is different from the region type of the tissue region, and obtain at least one abnormal image block;

[0021] Calculate the ratio of the number of all the abnormal image patches to the total number of all the image patches in the tissue region;

[0022] Determine whether the ratio of the quantities is less than a preset ratio threshold;

[0023] If so, modify the predicted region type of all the abnormal image blocks to the region type of the tissue region;

[0024] If not, each of the image blocks in the tissue region is input into the prediction model corresponding to the prediction region type of any of the abnormal image blocks to obtain the other type prediction probability corresponding to each image block;

[0025] For each of the abnormal image patches, the average of the other type prediction probabilities of all the image patches in the tissue region corresponding to that prediction region type is calculated;

[0026] The predicted region type with the highest average value is determined as the region type of the organization region.

[0027] As an optional implementation, in the first aspect of the invention, the step of selecting at least one region combination from the plurality of organizational regions according to a preset type bordering rule includes:

[0028] For any two tissue regions that have an adjacent boundary, calculate the difference between the region images of the two tissue regions;

[0029] Calculate the border ratio of the two said tissue regions; the border ratio is the ratio of the length of the common boundary of the two said tissue regions to the total boundary length of the two said tissue regions;

[0030] The product of the difference and the border ratio is calculated to obtain the border parameters of the two tissue regions;

[0031] Based on the border parameters, the multiple organizational regions are grouped according to a dynamic programming algorithm to obtain at least one region combination.

[0032] As an optional implementation, in the first aspect of the invention, the step of grouping the plurality of organizational regions according to the border parameters using a dynamic programming algorithm to obtain at least one region combination includes:

[0033] The objective function is set to minimize the number of all said region combinations and the number of said organizational regions in said region combinations;

[0034] Setting restrictions includes:

[0035] The region types of any two of the organizational regions in the region combination satisfy the preset type conditions;

[0036] The border parameter of any two tissue regions in the region combination is greater than a preset first parameter threshold.

[0037] The border parameter of any two tissue regions belonging to different combinations of the said regions is less than a preset second parameter threshold; the second parameter threshold is less than the first parameter threshold.

[0038] Based on the objective function and the constraints, the multiple organizational regions are iteratively grouped according to the dynamic programming algorithm to obtain at least one region combination.

[0039] As an optional implementation, in the first aspect of the present invention, the region type is normal, necrosis, tumor, infiltration, or background; the region types of any two adjacent tissue regions in the region combination satisfy a preset type condition; the type condition is used to limit the region types of the two adjacent tissue regions to be different, and the region type combination of the two tissue regions is normal-necrosis, normal-tumor, necrosis-tumor, infiltration-tumor, or infiltration-necrosis.

[0040] As an optional implementation, in the first aspect of the present invention, the region boundary update model is trained using a training dataset that includes multiple training region combination images and corresponding region combination type annotations and region boundary adjustment annotations; the region boundary adjustment annotations include the original region boundary and the modified region boundary.

[0041] As an optional implementation, in the first aspect of the invention, the region boundary update model is trained through the following steps:

[0042] The loss function is determined to be the weighted sum of the first cross-entropy and the second cross-entropy; the first cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the original region; the second cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the modified region; wherein, the weight of the first cross-entropy is less than that of the second cross-entropy; the weight of the first cross-entropy is the product of the first weight and the second weight; the first weight is proportional to the boundary length of the original region boundary; the second weight is proportional to the total number of different region types in the corresponding training region combination image; the weight of the second cross-entropy is related to the corresponding region combination type label, and when the region combination type labels are normal-tumor, normal-necrosis, infiltrative-necrosis, necrosis-tumor, or infiltrative-tumor, the weight of the second cross-entropy increases sequentially;

[0043] The training dataset is input into a model with a preset CNN architecture for iterative training. During training, parameters are updated based on the loss function and gradient descent algorithm until training converges, thus obtaining the region boundary update model.

[0044] A second aspect of this invention discloses a tissue region image recognition system based on patch segmentation, the system comprising:

[0045] The segmentation module is used to divide the image of the biological tissue to be identified into multiple image blocks;

[0046] The first recognition module is used to input the multiple image blocks into the region recognition model to obtain multiple tissue regions and corresponding region types;

[0047] The filtering module is used to filter at least one region combination from the plurality of organizational regions according to a preset type bordering rule; the region combination includes a plurality of mutually bordering organizational regions;

[0048] The second identification module is used to input the region image corresponding to the region combination and the corresponding region type into the region boundary update model to obtain the corresponding region update result; the region update result is used to re-indicate the region boundary of different tissue regions in the region image.

[0049] As an optional implementation, in a second aspect of the invention, the region identification model is trained through the following steps:

[0050] Determine the pre-defined type region segmentation labels corresponding to the training tissue images;

[0051] The training tissue image is segmented to obtain multiple training image blocks;

[0052] For each training image block, calculate the intersection area between the training image block and any type of partitioning region in the type region partitioning annotation;

[0053] The region type corresponding to the type division region with the highest intersection area is determined as the type label corresponding to the training image block;

[0054] Based on all the training image patches and the corresponding type labels, the model of the preset CNN network architecture is trained iteratively. In each round of training, the average classification accuracy corresponding to each type label is calculated to obtain the training performance parameters; the classification accuracy is the F1 score or the F2 score.

[0055] The weights of the model obtained in the training round with the highest training performance parameters are determined as the final model weights, thus obtaining the region recognition model.

[0056] As an optional implementation, in a second aspect of the present invention, the specific method by which the first recognition module inputs the plurality of image blocks into a region recognition model to obtain a plurality of tissue regions and corresponding region types includes:

[0057] Each image block is input into the region recognition model to obtain the predicted region type corresponding to each image block;

[0058] All the image blocks are stitched together based on the segmentation rules to obtain a stitched tissue image;

[0059] Morphological operations are performed on the stitched tissue image to obtain a contour-smoothed image; the contour-smoothed image includes multiple tissue regions and corresponding region types; each tissue region includes multiple corresponding image blocks of the same predicted region type; the region type is the predicted region type corresponding to the image block in the tissue region;

[0060] For each tissue region in the contour smoothing image, identify the image block in the tissue region whose predicted region type is different from the region type of the tissue region, and obtain at least one abnormal image block;

[0061] Calculate the ratio of the number of all the abnormal image patches to the total number of all the image patches in the tissue region;

[0062] Determine whether the ratio of the quantities is less than a preset ratio threshold;

[0063] If so, modify the predicted region type of all the abnormal image blocks to the region type of the tissue region;

[0064] If not, each of the image blocks in the tissue region is input into the prediction model corresponding to the prediction region type of any of the abnormal image blocks to obtain the other type prediction probability corresponding to each image block;

[0065] For each of the abnormal image patches, the average of the other type prediction probabilities of all the image patches in the tissue region corresponding to that prediction region type is calculated;

[0066] The predicted region type with the highest average value is determined as the region type of the organization region.

[0067] As an optional implementation, in a second aspect of the invention, the specific method by which the filtering module filters at least one combination of regions from the plurality of tissue regions according to a preset type boundary rule includes:

[0068] For any two tissue regions that have an adjacent boundary, calculate the difference between the region images of the two tissue regions;

[0069] Calculate the border ratio of the two said tissue regions; the border ratio is the ratio of the length of the common boundary of the two said tissue regions to the total boundary length of the two said tissue regions;

[0070] The product of the difference and the border ratio is calculated to obtain the border parameters of the two tissue regions;

[0071] Based on the border parameters, the multiple organizational regions are grouped according to a dynamic programming algorithm to obtain at least one region combination.

[0072] As an optional implementation, in a second aspect of the invention, the screening module, based on the border parameters, groups the plurality of tissue regions according to a dynamic programming algorithm to obtain at least one region combination in the following specific manner:

[0073] The objective function is set to minimize the number of all said region combinations and the number of said organizational regions in said region combinations;

[0074] Setting restrictions includes:

[0075] The region types of any two of the organizational regions in the region combination satisfy the preset type conditions;

[0076] The border parameter of any two tissue regions in the region combination is greater than a preset first parameter threshold.

[0077] The border parameter of any two tissue regions belonging to different combinations of the said regions is less than a preset second parameter threshold; the second parameter threshold is less than the first parameter threshold.

[0078] Based on the objective function and the constraints, the multiple organizational regions are iteratively grouped according to the dynamic programming algorithm to obtain at least one region combination.

[0079] As an optional implementation, in the second aspect of the present invention, the region type is a normal type, a necrosis type, a tumor type, an infiltrative type, or a background type; the region types of any two adjacent tissue regions in the region combination satisfy a preset type condition; the type condition is used to limit the region types of the two adjacent tissue regions to be different, and the region type combination of the two tissue regions is normal-necrosis, normal-tumor, necrosis-tumor, infiltrative-tumor, or infiltrative-necrosis.

[0080] As an optional implementation, in a second aspect of the present invention, the region boundary update model is trained using a training dataset that includes multiple training region combination images and corresponding region combination type annotations and region boundary adjustment annotations; the region boundary adjustment annotations include the original region boundary and the modified region boundary.

[0081] As an optional implementation, in a second aspect of the invention, the region boundary update model is trained through the following steps:

[0082] The loss function is determined to be the weighted sum of the first cross-entropy and the second cross-entropy; the first cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the original region; the second cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the modified region; wherein, the weight of the first cross-entropy is less than that of the second cross-entropy; the weight of the first cross-entropy is the product of the first weight and the second weight; the first weight is proportional to the boundary length of the original region boundary; the second weight is proportional to the total number of different region types in the corresponding training region combination image; the weight of the second cross-entropy is related to the corresponding region combination type label, and when the region combination type labels are normal-tumor, normal-necrosis, infiltrative-necrosis, necrosis-tumor, or infiltrative-tumor, the weight of the second cross-entropy increases sequentially;

[0083] The training dataset is input into a model with a preset CNN architecture for iterative training. During training, parameters are updated based on the loss function and gradient descent algorithm until training converges, thus obtaining the region boundary update model.

[0084] A third aspect of the present invention discloses another tissue region image recognition system based on patch segmentation, the system comprising:

[0085] Memory containing executable program code;

[0086] A processor coupled to the memory;

[0087] The processor calls the executable program code stored in the memory to execute some or all of the steps in the tissue region image recognition method based on patch division disclosed in the first aspect of the present invention.

[0088] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the tissue region image recognition method based on patch division disclosed in the first aspect of the present invention.

[0089] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0090] This invention can first identify preliminary tissue regions and region types from tissue images after they have been divided into blocks based on a region recognition model. Then, after filtering out the region combinations that match the types, the region boundaries are identified and updated based on the bordering regions and types through a further region boundary update model. This allows for full utilization of the characteristics of bordering region types to accurately identify region boundaries, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 is a flowchart illustrating a tissue region image recognition method based on patch division disclosed in an embodiment of the present invention.

[0093] Figure 2 is a schematic diagram of the structure of an organization region image recognition system based on patch division disclosed in an embodiment of the present invention.

[0094] Figure 3 is a schematic diagram of another tissue region image recognition system based on patch division disclosed in an embodiment of the present invention.

[0095] Figure 4 is a schematic diagram of the image recognition result of an organization region image recognition scheme disclosed in an embodiment of the present invention.

[0096] Figure 5 shows the image recognition result of another tissue region image recognition scheme disclosed in an embodiment of the present invention. Embodiments of the present invention

[0097] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0098] The terms "first," "second," etc., used in the specification and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0099] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0100] This invention discloses a method and system for tissue region image recognition based on patch division. It first identifies preliminary tissue regions and region types from the patched tissue image using a region recognition model. Then, it filters out region combinations with suitable types and, based on adjacent regions and their types, uses a further region boundary update model to identify and update the region boundaries. This fully utilizes the characteristics of adjacent region types to accurately identify region boundaries, improving the accuracy and efficiency of tissue region recognition and providing an accurate data foundation for disease treatment or research. Detailed explanations follow.

[0101] Example 1

[0102] Please refer to Figure 1, which is a flowchart illustrating a method for tissue region image recognition based on tile division according to an embodiment of the present invention. The method for tissue region image recognition based on tile division described in Figure 1 can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As shown in Figure 1, the method for tissue region image recognition based on tile division may include the following operations:

[0103] 101. Divide the biological tissue image to be identified into multiple image blocks.

[0104] 102. Input multiple image patches into the region recognition model to obtain multiple tissue regions and their corresponding region types.

[0105] 103. Based on the preset type bordering rules, select at least one region combination from multiple organizational regions.

[0106] Optionally, the regional combination may include multiple contiguous organizational regions.

[0107] 104. Input the region image and the corresponding region type corresponding to the region combination into the region boundary update model to obtain the corresponding region update result.

[0108] Optionally, the region update results are used to re-indicate the region boundaries of different tissue regions in the region image.

[0109] As can be seen, the above-described embodiments of the invention can first identify the preliminary tissue regions and region types based on the region recognition model after dividing the tissue image into blocks, and then select the region combinations that match the types. Based on the bordering regions and types, the region boundaries are identified and updated through a further region boundary update model. This can make full use of the characteristics of the bordering region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0110] As an optional embodiment, the region identification model in the above steps is trained through the following steps:

[0111] Determine the pre-defined type region segmentation labels corresponding to the training tissue images;

[0112] The training tissue image is segmented to obtain multiple training image patches;

[0113] For each training image patch, calculate the intersection area between the training image patch and any type partitioning region in the type region partitioning annotation;

[0114] The region type corresponding to the type division with the highest intersection area is determined as the type label corresponding to the training image patch;

[0115] Based on all training image patches and their corresponding type labels, the model of the preset CNN network architecture is trained iteratively. In each round of training, the average classification accuracy corresponding to each type label is calculated to obtain the training performance parameters; optionally, the classification accuracy is the F1 score or the F2 score.

[0116] The weights of the model obtained in the training round with the highest performance parameters are determined as the final model weights, thus obtaining the region recognition model.

[0117] In one specific implementation, a region identification model is used to identify tumor regions within a tissue and surrounding invasive areas for subsequent boundary updates. Specifically, the region identification model needs to identify tumor regions, normal epithelial regions, and necrotic regions. The normal epithelial and necrotic regions are primarily used to distinguish them from the tumor regions and surrounding invasive areas.

[0118] Specifically, the digital pathology slides scanned from the entire slide were randomly divided into a training set, a validation set, and a test set. The training set consisted of 414 slides, the validation set of 60 slides, and the test set of 250 slides. Each digital pathology slide had a region outline manually annotated by a pathologist, with the annotation categories being "normal epithelium," "tumor," and "necrosis." It also had a corresponding region for machine recognition, as shown in Figure 4. The red outline represents the tumor region identified by the machine, and the yellow area represents the normal epithelial region identified by the machine.

[0119] Specifically, due to the enormous size of digital pathology slides, it's impossible to load the entire image into a GPU for training. Therefore, a slice-based training method is used, where a 512x512 window is used to traverse the large image, dividing it into smaller 512x512 tiles. If the proportion of non-tissue region pixels (black pixels in Figure 4) in a small tile is less than 5% of the total pixels in the tile, the tile is retained to ensure that the final small tiles contain sufficient tissue regions. A total of 1.46 million and 670,000 tiles were collected from the training and validation sets, respectively. Specifically, the category of a small tile is determined based on its intersection with the contours annotated by the doctor, and the determination method has been disclosed in the above embodiment.

[0120] Specifically, the region identification model adopts the EfficientNetV2 architecture and trains a classification model based on the generated small image patches. The classification categories are four: tumor, normal epithelium, necrosis, and background. Optionally, various classification model architectures could be used, but considering both model performance and prediction efficiency, the classic EfficientNetV2 architecture was chosen.

[0121] Specifically, the input image is first preprocessed, including: channel extraction, which extracts only the DAPI and PanCK+ channels from the original four-channel image and adds an extra channel with all zeros, so that the final input image becomes a three-channel image, which can then load the ImageNet pre-trained weights; and data compression, which compresses the original image data of each channel to 8 bits, thereby improving the efficiency of training and prediction.

[0122] Specifically, during training, each training round consists of 7000 random iterations, and each validation round consists of 3000 random iterations, for a total of 400 training rounds. Since the number of tiles in the four categories is not balanced, class-balanced sampling is used in each training round to ensure that the tile data for the four categories does not differ too much. Simultaneously, random data augmentation is used during training, with a certain probability of rotating, flipping, scaling, and changing the color of the tiles, thereby increasing the model's generalization ability. After each training round, the mean F1 score of the three categories (tumor, normal epithelial, and necrotic) on the validation set is calculated, and the weights of the round with the largest mean are selected as the optimal weights for the final classification model.

[0123] As can be seen, through the above optional embodiments, the entire training tissue image can be divided and labeled based on the judgment of physicians or the intervention of technicians. Then, the type label is automatically determined based on the intersection area of ​​the image block after segmentation and different segmented regions. This simplifies the labeling process on the one hand and ensures the labeling accuracy on the other. The CNN network is then trained using the segmented training tissue image and type label, and the model weights are selected based on the training performance related to classification accuracy to train a more accurate region recognition model. This helps to fully utilize the characteristics of the border region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region recognition, and provide an accurate data foundation for disease treatment or research.

[0124] As an optional embodiment, the above steps, in which multiple image patches are input into a region recognition model to obtain multiple tissue regions and corresponding region types, include:

[0125] Each image patch is input into the region recognition model to obtain the predicted region type corresponding to each image patch;

[0126] All image blocks are stitched together based on the segmentation rules to obtain the stitched tissue image;

[0127] Morphological operations are performed on the stitched tissue images to obtain a contour-smoothed image; optionally, the contour-smoothed image includes multiple tissue regions and corresponding region types; each tissue region includes multiple corresponding image patches with the same predicted region type; the region type is the predicted region type corresponding to the image patch in the tissue region;

[0128] For each tissue region in the contour smoothing image, identify the image patch in the tissue region whose predicted region type is different from the region type of the tissue region, and obtain at least one abnormal image patch.

[0129] Calculate the ratio of the number of all anomalous image patches to the total number of all image patches in the tissue region;

[0130] Determine whether the quantity ratio is less than a preset ratio threshold;

[0131] If so, change the predicted region type of all abnormal image patches to the region type of the tissue region;

[0132] If not, input each image block in the tissue region into the prediction model corresponding to the prediction region type of any abnormal image block to obtain the other type prediction probability for each image block;

[0133] For each anomalous image patch's predicted region type, calculate the average of the other predicted probabilities for all image patches in that tissue region corresponding to that predicted region type;

[0134] The region type with the highest average predicted region type is determined as the region type of the organization.

[0135] Specifically, in the above implementation scheme, the scheme in this embodiment was implemented. Specifically, when the region recognition model predicts, it also uses 512×512 tiles to traverse the entire image organization range and obtains the classification result of each tile. The classified tiles are then stitched together to restore the large image, and morphological operations, such as dilation followed by erosion, are performed to make the contours of different categories of regions smoother.

[0136] Specifically, the solution based on the above embodiments can be implemented such that if there is a tumor area within a normal tissue area, these tumor areas are converted into normal tissue areas, thereby eliminating falsely detected tumor areas within the normal tissue area; if there is a normal epithelial area within a tumor area, these normal epithelial areas are converted into tumor areas, thereby eliminating falsely detected normal epithelial areas within the tumor area.

[0137] As can be seen, through the above optional embodiments, image blocks can be predicted based on the region recognition model, and then the image blocks can be re-stitched and the regions can be judged to achieve more accurate preliminary region type recognition. This helps to make full use of the characteristics of the border region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region recognition, and provide an accurate data foundation for disease treatment or research.

[0138] As an optional embodiment, the step described above, selecting at least one region combination from multiple organizational regions according to a preset type boundary rule, includes:

[0139] For any two tissue regions that have an adjacent boundary, calculate the difference between the region images of the two tissue regions;

[0140] Calculate the confluence ratio of the boundaries of the two organizational regions; optionally, the confluence ratio is the ratio of the length of the common boundary of the two organizational regions to the total length of the boundary of the two organizational regions.

[0141] The product of the degree of difference and the proportion of bordering is used to obtain the bordering parameters of the two tissue regions;

[0142] Based on the border parameters, multiple organizational regions are grouped according to the dynamic programming algorithm to obtain at least one region combination.

[0143] As can be seen, through the above optional embodiments, the degree of border sufficiency can be obtained by calculating the image difference between the adjacent tissue regions and the proportion of the border boundary. Then, based on the dynamic programming algorithm, the regions can be divided and screened to obtain more differentiated combinations of regions, which facilitates subsequent regional boundary identification. This helps to make full use of the characteristics of the border region types to accurately identify regional boundaries, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0144] As an optional embodiment, the step described above, grouping multiple organizational regions based on border parameters using a dynamic programming algorithm to obtain at least one region combination, includes:

[0145] The objective function is set to minimize the number of all region combinations and the number of organizational regions within each region combination.

[0146] Setting restrictions includes:

[0147] The region types of any two organizational regions in the region combination meet the preset type conditions;

[0148] The border parameter between any two organizational regions in the region combination is greater than the preset first parameter threshold.

[0149] The border parameter of any two organizational regions belonging to different regional combinations is less than the preset second parameter threshold; optionally, the second parameter threshold is less than the first parameter threshold.

[0150] Based on the objective function and constraints, multiple organizational regions are iteratively grouped using a dynamic programming algorithm to obtain at least one region combination.

[0151] As can be seen, through the above optional embodiments, based on the preset objective function and constraints, the dynamic programming algorithm can be used to iteratively group and filter to obtain more differentiated and accurate combinations of regions, which facilitates subsequent regional boundary identification, helps to make full use of the characteristics of the bordering region types to accurately identify regional boundaries, improves the accuracy and efficiency of tissue region identification, and provides an accurate data foundation for disease treatment or research.

[0152] As an optional embodiment, in the above steps, the region type is normal, necrosis, tumor, infiltration, or background; the region types of any two adjacent tissue regions in the region combination meet the preset type conditions; the type conditions are used to limit the region types of the two adjacent tissue regions to be different, and the region type combination of the two tissue regions is normal-necrosis, normal-tumor, necrosis-tumor, infiltration-tumor, or infiltration-necrosis.

[0153] As can be seen, the above optional embodiments limit the type conditions of tissue regions in the region combination, which facilitates subsequent region boundary identification, helps to make full use of the characteristics of the border region types to accurately identify the region boundaries, improves the accuracy and efficiency of tissue region identification, and provides an accurate data foundation for disease treatment or research.

[0154] As an optional embodiment, in the above steps, the region boundary update model is trained using a training dataset that includes multiple training region combination images and corresponding region combination type annotations and region boundary adjustment annotations; the region boundary adjustment annotations include the original region boundary and the modified region boundary.

[0155] As can be seen, the training dataset details of the regional boundary update model are defined through the above optional embodiments, so that after training, it can be effectively used to refine the boundaries of specific types of regions, making full use of the characteristics of the bordering region types to accurately identify the regional boundaries, improving the accuracy and efficiency of tissue region identification, and providing an accurate data foundation for disease treatment or research.

[0156] As an optional embodiment, the region boundary update model in the above steps is trained through the following steps:

[0157] The loss function is determined as a weighted sum of the first cross-entropy and the second cross-entropy. Optionally, the first cross-entropy is the cross-entropy between the recognition boundary output by the model and the original region boundary; the second cross-entropy is the cross-entropy between the recognition boundary output by the model and the modified region boundary. The weight of the first cross-entropy is less than that of the second cross-entropy. The weight of the first cross-entropy is the product of the first weight and the second weight. The first weight is proportional to the boundary length of the original region boundary. The second weight is proportional to the total number of different region types in the corresponding training region combination image. The weight of the second cross-entropy is related to the corresponding region combination type label. When the region combination type labels are normal-tumor, normal-necrosis, infiltrative-necrosis, necrosis-tumor, or infiltrative-tumor, the weight of the second cross-entropy increases sequentially.

[0158] The training dataset is input into a model with a pre-defined CNN architecture for iterative training. During training, parameters are updated based on the loss function and gradient descent algorithm until the training converges, resulting in a region boundary update model.

[0159] Specifically, a recognition example of the trained region boundary update model can be seen in Figure 5, where the left side is the region boundary before the update and the right side is the region boundary after the update (the green outline is the tumor region, the purple outline is the normal epithelial region, and the pink outline is the necrotic region).

[0160] In one experiment, the DICE metric was used on the entire image to evaluate the consistency with human annotations for each of the final generated tissue regions. The observed DICE values ​​of the main target tumor regions were all greater than 0.8 on the training, validation, and test sets, demonstrating the model's good generalization ability.

[0161] As can be seen, through the above optional embodiments, the priority of different boundary labels when updating the boundary can be effectively adjusted based on the preset loss function, so as to train a regional boundary update model that can update the boundary of the tissue region more accurately, realize the full utilization of the characteristics of the border region type to accurately identify the regional boundary, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0162] Example 2

[0163] Please refer to Figure 2, which is a schematic diagram of a tissue region image recognition system based on tile division disclosed in an embodiment of the present invention. The tissue region image recognition system based on tile division described in Figure 2 can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As shown in Figure 2, the tissue region image recognition system based on tile division may include:

[0164] The segmentation module 201 is used to segment the biological tissue image to be identified into multiple image blocks.

[0165] The first recognition module 202 is used to input multiple image blocks into the region recognition model to obtain multiple tissue regions and corresponding region types.

[0166] The filtering module 203 is used to filter at least one combination of regions from multiple organizational regions according to preset type boundary rules.

[0167] Optionally, the regional combination may include multiple contiguous organizational regions.

[0168] The second recognition module 204 is used to input the region image corresponding to the region combination and the corresponding region type into the region boundary update model to obtain the corresponding region update result.

[0169] Optionally, the region update results are used to re-indicate the region boundaries of different tissue regions in the region image.

[0170] As can be seen, the above-described embodiments of the invention can first identify the preliminary tissue regions and region types based on the region recognition model after dividing the tissue image into blocks, and then select the region combinations that match the types. Based on the bordering regions and types, the region boundaries are identified and updated through a further region boundary update model. This can make full use of the characteristics of the bordering region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0171] As an optional implementation, the region identification model is trained through the following steps:

[0172] Determine the pre-defined type region segmentation labels corresponding to the training tissue images;

[0173] The training tissue image is segmented to obtain multiple training image patches;

[0174] For each training image patch, calculate the intersection area between the training image patch and any type partitioning region in the type region partitioning annotation;

[0175] The region type corresponding to the type division with the highest intersection area is determined as the type label corresponding to the training image patch;

[0176] Based on all training image patches and their corresponding type labels, the model of the preset CNN network architecture is trained iteratively. In each round of training, the average classification accuracy corresponding to each type label is calculated to obtain the training performance parameters; optionally, the classification accuracy is the F1 score or the F2 score.

[0177] The weights of the model obtained in the training round with the highest performance parameters are determined as the final model weights, thus obtaining the region recognition model.

[0178] As can be seen, through the above optional embodiments, the entire training tissue image can be divided and labeled based on the judgment of physicians or the intervention of technicians. Then, the type label is automatically determined based on the intersection area of ​​the image block after segmentation and different segmented regions. This simplifies the labeling process on the one hand and ensures the labeling accuracy on the other. The CNN network is then trained using the segmented training tissue image and type label, and the model weights are selected based on the training performance related to classification accuracy to train a more accurate region recognition model. This helps to fully utilize the characteristics of the border region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region recognition, and provide an accurate data foundation for disease treatment or research.

[0179] As an optional embodiment, the first recognition module inputs multiple image patches into the region recognition model to obtain multiple tissue regions and their corresponding region types in the following specific ways:

[0180] Each image patch is input into the region recognition model to obtain the predicted region type corresponding to each image patch;

[0181] All image blocks are stitched together based on the segmentation rules to obtain the stitched tissue image;

[0182] Morphological operations are performed on the stitched tissue images to obtain a contour-smoothed image; optionally, the contour-smoothed image includes multiple tissue regions and corresponding region types; each tissue region includes multiple corresponding image patches with the same predicted region type; the region type is the predicted region type corresponding to the image patch in the tissue region;

[0183] For each tissue region in the contour smoothing image, identify the image patch in the tissue region whose predicted region type is different from the region type of the tissue region, and obtain at least one abnormal image patch.

[0184] Calculate the ratio of the number of all anomalous image patches to the total number of all image patches in the tissue region;

[0185] Determine whether the quantity ratio is less than a preset ratio threshold;

[0186] If so, change the predicted region type of all abnormal image patches to the region type of the tissue region;

[0187] If not, input each image block in the tissue region into the prediction model corresponding to the prediction region type of any abnormal image block to obtain the other type prediction probability for each image block;

[0188] For each anomalous image patch's predicted region type, calculate the average of the other predicted probabilities for all image patches in that tissue region corresponding to that predicted region type;

[0189] The region type with the highest average predicted region type is determined as the region type of the organization.

[0190] As can be seen, through the above optional embodiments, image blocks can be predicted based on the region recognition model, and then the image blocks can be re-stitched and the regions can be judged to achieve more accurate preliminary region type recognition. This helps to make full use of the characteristics of the border region types to accurately identify the region boundaries, improve the accuracy and efficiency of tissue region recognition, and provide an accurate data foundation for disease treatment or research.

[0191] As an optional embodiment, the filtering module selects at least one combination of regions from multiple organizational regions according to preset type boundary rules, including:

[0192] For any two tissue regions that have an adjacent boundary, calculate the difference between the region images of the two tissue regions;

[0193] Calculate the confluence ratio of the boundaries of the two organizational regions; optionally, the confluence ratio is the ratio of the length of the common boundary of the two organizational regions to the total length of the boundary of the two organizational regions.

[0194] The product of the degree of difference and the proportion of bordering is used to obtain the bordering parameters of the two tissue regions;

[0195] Based on the border parameters, multiple organizational regions are grouped according to the dynamic programming algorithm to obtain at least one region combination.

[0196] As can be seen, through the above optional embodiments, the degree of border sufficiency can be obtained by calculating the image difference between the adjacent tissue regions and the proportion of the border boundary. Then, based on the dynamic programming algorithm, the regions can be divided and screened to obtain more differentiated combinations of regions, which facilitates subsequent regional boundary identification. This helps to make full use of the characteristics of the border region types to accurately identify regional boundaries, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0197] As an optional embodiment, the screening module, based on the border parameters, groups multiple organizational regions according to a dynamic programming algorithm to obtain at least one specific method for combining regions, including:

[0198] The objective function is set to minimize the number of all region combinations and the number of organizational regions within each region combination.

[0199] Setting restrictions includes:

[0200] The region types of any two organizational regions in the region combination meet the preset type conditions;

[0201] The border parameter between any two organizational regions in the region combination is greater than the preset first parameter threshold.

[0202] The border parameter of any two organizational regions belonging to different regional combinations is less than the preset second parameter threshold; optionally, the second parameter threshold is less than the first parameter threshold.

[0203] Based on the objective function and constraints, multiple organizational regions are iteratively grouped using a dynamic programming algorithm to obtain at least one region combination.

[0204] As can be seen, through the above optional embodiments, based on the preset objective function and constraints, the dynamic programming algorithm can be used to iteratively group and filter to obtain more differentiated and accurate combinations of regions, which facilitates subsequent regional boundary identification, helps to make full use of the characteristics of the bordering region types to accurately identify regional boundaries, improves the accuracy and efficiency of tissue region identification, and provides an accurate data foundation for disease treatment or research.

[0205] As an optional embodiment, the region type is normal, necrosis, tumor, infiltration, or background; the region types of any two adjacent tissue regions in the region combination meet the preset type conditions; the type conditions are used to limit the region types of the two adjacent tissue regions to be different, and the region type combination of the two tissue regions is normal-necrosis, normal-tumor, necrosis-tumor, infiltration-tumor, or infiltration-necrosis.

[0206] As can be seen, the above optional embodiments limit the type conditions of tissue regions in the region combination, which facilitates subsequent region boundary identification, helps to make full use of the characteristics of the border region types to accurately identify the region boundaries, improves the accuracy and efficiency of tissue region identification, and provides an accurate data foundation for disease treatment or research.

[0207] As an optional embodiment, the region boundary update model is trained using a training dataset that includes multiple training region combination images and corresponding region combination type annotations and region boundary adjustment annotations; the region boundary adjustment annotations include the original region boundary and the modified region boundary.

[0208] As can be seen, the training dataset details of the regional boundary update model are defined through the above optional embodiments, so that after training, it can be effectively used to refine the boundaries of specific types of regions, making full use of the characteristics of the bordering region types to accurately identify the regional boundaries, improving the accuracy and efficiency of tissue region identification, and providing an accurate data foundation for disease treatment or research.

[0209] As an optional implementation, the region boundary update model is trained through the following steps:

[0210] The loss function is determined as a weighted sum of the first cross-entropy and the second cross-entropy. Optionally, the first cross-entropy is the cross-entropy between the recognition boundary output by the model and the original region boundary; the second cross-entropy is the cross-entropy between the recognition boundary output by the model and the modified region boundary. The weight of the first cross-entropy is less than that of the second cross-entropy. The weight of the first cross-entropy is the product of the first weight and the second weight. The first weight is proportional to the boundary length of the original region boundary. The second weight is proportional to the total number of different region types in the corresponding training region combination image. The weight of the second cross-entropy is related to the corresponding region combination type label. When the region combination type labels are normal-tumor, normal-necrosis, infiltrative-necrosis, necrosis-tumor, or infiltrative-tumor, the weight of the second cross-entropy increases sequentially.

[0211] The training dataset is input into a model with a pre-defined CNN architecture for iterative training. During training, parameters are updated based on the loss function and gradient descent algorithm until the training converges, resulting in a region boundary update model.

[0212] As can be seen, through the above optional embodiments, the priority of different boundary labels when updating the boundary can be effectively adjusted based on the preset loss function, so as to train a regional boundary update model that can update the boundary of the tissue region more accurately, realize the full utilization of the characteristics of the border region type to accurately identify the regional boundary, improve the accuracy and efficiency of tissue region identification, and provide an accurate data foundation for disease treatment or research.

[0213] Example 3

[0214] Please refer to Figure 3, which illustrates another tissue region image recognition system based on tile division according to an embodiment of the present invention. The tissue region image recognition system based on tile division described in Figure 3 is applied in a data processing system / data processing device / data processing server (wherein, the server includes a local processing server or a cloud processing server). As shown in Figure 3, the tissue region image recognition system based on tile division may include:

[0215] Memory 301 storing executable program code;

[0216] Processor 302 coupled to memory 301;

[0217] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the tissue region image recognition method based on patch division described in Embodiment 1.

[0218] Example 4

[0219] This invention discloses a computer read storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps of the tissue region image recognition method based on tile division described in Embodiment 1.

[0220] Example 5

[0221] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the tissue region image recognition method based on tile division described in Embodiment 1.

[0222] The foregoing has described specific embodiments of this specification. In some cases, the described actions or steps may be performed in a different order than those shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0223] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0224] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0225] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0226] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0227] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0228] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0229] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0230] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0231] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0232] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0233] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0234] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0235] Finally, it should be noted that the method and system for tissue region image recognition based on patch division disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing tissue regions in images based on patch segmentation, characterized in that, The method includes: The biological tissue image to be identified is divided into multiple image patches; The multiple image blocks are input into the region recognition model to obtain multiple tissue regions and their corresponding region types; According to a preset type-adjacency rule, at least one region combination is selected from the plurality of organizational regions; the region combination includes a plurality of mutually adjacent organizational regions; The region image corresponding to the region combination and the corresponding region type are input into the region boundary update model to obtain the corresponding region update result; the region update result is used to re-indicate the region boundary of different tissue regions in the region image.

2. The method for tissue region image recognition based on patch segmentation according to claim 1, characterized in that, The region identification model is trained through the following steps: Determine the pre-defined type region segmentation labels corresponding to the training tissue images; The training tissue image is segmented to obtain multiple training image blocks; For each training image block, calculate the intersection area between the training image block and any type of partitioning region in the type region partitioning annotation; The region type corresponding to the type division region with the highest intersection area is determined as the type label corresponding to the training image block; Based on all the training image patches and the corresponding type labels, the model of the preset CNN network architecture is trained iteratively. In each round of training, the average classification accuracy corresponding to each type label is calculated to obtain the training performance parameters; the classification accuracy is the F1 score or the F2 score. The weights of the model obtained in the training round with the highest training performance parameters are determined as the final model weights, thus obtaining the region recognition model.

3. The tissue region image recognition method based on patch segmentation according to claim 2, characterized in that, The step of inputting the multiple image patches into the region recognition model to obtain multiple tissue regions and corresponding region types includes: Each image block is input into the region recognition model to obtain the predicted region type corresponding to each image block; All the image blocks are stitched together based on the segmentation rules to obtain a stitched tissue image; Morphological operations are performed on the stitched tissue image to obtain a contour-smoothed image; the contour-smoothed image includes multiple tissue regions and corresponding region types; each tissue region includes multiple corresponding image blocks of the same predicted region type; the region type is the predicted region type corresponding to the image block in the tissue region; For each tissue region in the contour smoothing image, identify the image block in the tissue region whose predicted region type is different from the region type of the tissue region, and obtain at least one abnormal image block; Calculate the ratio of the number of all the abnormal image patches to the total number of all the image patches in the tissue region; Determine whether the ratio of the quantities is less than a preset ratio threshold; If so, modify the predicted region type of all the abnormal image blocks to the region type of the tissue region; If not, each of the image blocks in the tissue region is input into the prediction model corresponding to the prediction region type of any of the abnormal image blocks to obtain the other type prediction probability corresponding to each image block; For each of the abnormal image patches, the average of the other type prediction probabilities of all the image patches in the tissue region corresponding to that prediction region type is calculated; The predicted region type with the highest average value is determined as the region type of the organization region.

4. The method for tissue region image recognition based on patch segmentation according to claim 1, characterized in that, The step of selecting at least one region combination from the multiple organizational regions according to preset type bordering rules includes: For any two tissue regions that have an adjacent boundary, calculate the difference between the region images of the two tissue regions; Calculate the border ratio of the two said tissue regions; the border ratio is the ratio of the length of the common boundary of the two said tissue regions to the total boundary length of the two said tissue regions; The product of the difference and the border ratio is calculated to obtain the border parameters of the two tissue regions; Based on the border parameters, the multiple organizational regions are grouped according to a dynamic programming algorithm to obtain at least one region combination.

5. The tissue region image recognition method based on patch segmentation according to claim 4, characterized in that, Based on the border parameters, the multiple organizational regions are grouped according to a dynamic programming algorithm to obtain at least one region combination, including: The objective function is set to minimize the number of all said region combinations and the number of said organizational regions in said region combinations; Setting restrictions includes: The region types of any two of the organizational regions in the region combination satisfy the preset type conditions; The border parameter of any two tissue regions in the region combination is greater than a preset first parameter threshold. The border parameter of any two tissue regions belonging to different combinations of the said regions is less than a preset second parameter threshold; the second parameter threshold is less than the first parameter threshold. Based on the objective function and the constraints, the multiple organizational regions are iteratively grouped according to the dynamic programming algorithm to obtain at least one region combination.

6. The method for tissue region image recognition based on patch segmentation according to claim 1, characterized in that, The region type is normal, necrotic, tumor, infiltrative, or background; the region types of any two adjacent tissue regions in the region combination meet the preset type conditions; The type condition is used to define that the two adjacent tissue regions have different region types, and that the region types of the two tissue regions are combined as normal-necrosis, normal-tumor, necrosis-tumor, infiltration-tumor, or infiltration-necrosis.

7. The method for tissue region image recognition based on patch segmentation according to claim 1, characterized in that, The region boundary update model is trained using a training dataset that includes multiple training region combination images and corresponding region combination type annotations and region boundary adjustment annotations; the region boundary adjustment annotations include the original region boundary and the modified region boundary.

8. The method for tissue region image recognition based on patch segmentation according to claim 7, characterized in that, The region boundary update model is trained through the following steps: The loss function is determined to be the weighted sum of the first cross-entropy and the second cross-entropy; the first cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the original region; the second cross-entropy is the cross-entropy between the recognition boundary output by the model and the boundary of the modified region; wherein, the weight of the first cross-entropy is less than that of the second cross-entropy; the weight of the first cross-entropy is the product of the first weight and the second weight; the first weight is proportional to the boundary length of the original region boundary; the second weight is proportional to the total number of different region types in the corresponding training region combination image; the weight of the second cross-entropy is related to the corresponding region combination type label, and when the region combination type labels are normal-tumor, normal-necrosis, infiltrative-necrosis, necrosis-tumor, or infiltrative-tumor, the weight of the second cross-entropy increases sequentially; The training dataset is input into a model with a preset CNN architecture for iterative training. During training, parameters are updated based on the loss function and gradient descent algorithm until training converges, thus obtaining the region boundary update model.

9. A tissue region image recognition system based on patch segmentation, characterized in that, The system includes: The segmentation module is used to divide the image of the biological tissue to be identified into multiple image blocks; The first recognition module is used to input the multiple image blocks into the region recognition model to obtain multiple tissue regions and corresponding region types; The filtering module is used to filter at least one region combination from the plurality of organizational regions according to a preset type bordering rule; the region combination includes a plurality of mutually bordering organizational regions; The second identification module is used to input the region image corresponding to the region combination and the corresponding region type into the region boundary update model to obtain the corresponding region update result; the region update result is used to re-indicate the region boundary of different tissue regions in the region image.

10. A tissue region image recognition system based on patch segmentation, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the tissue region image recognition method based on patch division as described in any one of claims 1-8.

Images