Pathological image cancer area automatic identification system and method based on large pathological model

Abstract

The invention provides a pathological image cancer area automatic identification system and method based on a large pathological model, and the system comprises a large pathological model construction module which is used for carrying out the architecture optimization of a pathological image basic model according to a multi-head self-attention mechanism of deep learning and multi-scale feature fusion, and obtaining the large pathological model; the analysis module is used for capturing multi-level pathological characteristics from the cellular morphology to the tissue structure of the pancreatic cancer according to the large pathological model, and analyzing an input target pathological image according to the multi-level pathological characteristics; and the identification result output module is used for outputting an identification result of the pancreatic cancer area in the target pathological image based on the analysis result according to the large pathological model. Through a deep learning technology, a tumor area of pancreatic cancer is automatically identified and displayed in a thermodynamic diagram form, auxiliary pathological diagnosis and quantitative analysis of pathological features are realized, and pancreatic cancer pathological diagnosis efficiency and precision are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a pathological image cancer region automatic recognition system and method based on a pathological large model. Background Art

[0002] Currently, the prognosis of pancreatic cancer is extremely poor, and the incidence and mortality rates are continuously rising. Pathology is the "gold standard" for the diagnosis of pancreatic cancer, running through the diagnosis and treatment process and providing an important basis for the treatment decision-making of pancreatic cancer.

[0003] However, the pathological diagnosis of pancreatic cancer faces many challenges and it is difficult to guarantee the quality. First of all, the pathological features of pancreatic cancer are unique. The stroma accounts for more than 80% of the tumor, and the tumor intersects with normal pancreas and pancreatitis. It is difficult to accurately evaluate the size with the naked eye. Secondly, the pathological morphology of neoplastic epithelium and reactive epithelium of chronic pancreatitis is very similar, and pancreatic cancer is often accompanied by pancreatitis. The differential diagnosis between the two is very difficult, especially in biopsies with small and fragmented tissue and frozen sections with changed tissue morphology. It is more challenging to distinguish the two. In addition, the evaluation of lymph node invasion is time-consuming and laborious, and junior doctors are very likely to miss small cancer foci. Therefore, the precise evaluation of pancreatic cancer requires extremely high professionalism and rigor. In recent years, the technologies of artificial intelligence (AI) and whole-slide imaging (WSI) have developed rapidly. AI-assisted diagnosis systems have been widely used in multiple cancer types such as prostate cancer and cervical cancer. However, the diagnosis and annotation of pancreatic cancer are extremely difficult, and there is no clinically available development of an auxiliary diagnosis model for pancreatic cancer.

[0004] Therefore, in order to overcome the above technical problems, the present invention provides a pathological image cancer region automatic recognition system and method based on a pathological large model. Summary of the Invention

[0005] The present invention provides a pathological image cancer region automatic recognition system and method based on a pathological large model, which are used to automatically recognize the tumor region of pancreatic cancer through deep learning technology and display it in the form of a heat map, so as to realize auxiliary pathological diagnosis, quantitatively analyze pathological features, and improve the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0006] The present invention provides a pathological image cancer region automatic recognition system based on a pathological large model, including:

[0007] A pathological large model construction module, which is used to optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathological large model;

[0008] A parsing module, configured to capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to a pathological large model, and parse the input target pathological image according to the multi-level pathological features;

[0009] An identification result output module, configured to output an identification result of the pancreatic cancer area in the target pathological image based on the parsing result according to the pathological large model.

[0010] Optionally, the pathological large model construction module includes:

[0011] A requirement parsing unit, configured to obtain the business requirements of pancreatic cancer based on a management terminal;

[0012] A model library acquisition unit, configured to acquire a model library, where the model library includes a plurality of basic models;

[0013] A basic model selection unit, configured to screen out models that meet the business requirements from a preset model library according to the business requirements.

[0014] Optionally, the pathological large model construction module includes:

[0015] A model optimization preparation unit, configured to:

[0016] When identifying the cancer area of a pathological image based on a preset business system, obtain the attention dimensions of the input image, and at the same time, obtain the input features of each attention dimension, and convert the input features of each attention dimension into corresponding feature vector sequences;

[0017] Generate a self-attention matrix based on the feature vector sequences, and synthesize the self-attention matrices of each attention dimension to obtain a multi-head self-attention mechanism;

[0018] At the same time, based on the preset business system, determine the feature extraction scales when identifying the cancer area of a pathological image, and determine the association and dependency relationships between different feature extraction scales; determine a multi-scale feature fusion strategy between different feature extraction scales based on the hierarchical association relationship;

[0019] An architecture optimization unit, configured to:

[0020] Obtain the structural parameters of the basic model of the pathological image, and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the basic model of the pathological image based on the structural parameters;

[0021] Add the multi-head self-attention mechanism and the multi-scale feature fusion strategy to the basic model of the pathological image based on the insertion positions to complete the architecture optimization of the basic model of the pathological image and obtain a pathological large model.

[0022] Optionally, the pathological image cancer area automatic recognition system based on the pathological large model further includes a training unit, and the training unit includes:

[0023] A result acquisition subunit, configured to acquire the architecture optimization result of the obtained pathological image basic model, and select a loss function for the architecture optimization result from a preset function library based on the task type of the pathological large model;

[0024] A model test subunit, configured to acquire test data, input the test data into the architecture optimization result of the pathological image basic model for analysis and recognition, and determine a loss value based on the loss function for the analysis and recognition result;

[0025] A model optimization and deployment subunit, configured to, when the loss value is greater than a preset threshold, re-optimize the architecture of the architecture optimization result of the pathological image basic model, and deploy the obtained pathological large model in an application environment after the architecture optimization ends.

[0026] Optionally, the parsing module includes:

[0027] An image acquisition unit, configured to:

[0028] Collect pathological hierarchies including the cell morphology to tissue structure of pancreatic cancer;

[0029] Acquire a pathological image sample set of pancreatic cancer at different stages and subtypes according to the pathological hierarchy, and perform image processing on the pathological sample image set to obtain a standard pathological image sample set;

[0030] An image sample set division unit, configured to divide the standard pathological image sample set according to the pathological hierarchy to obtain a sub-standard pathological image sample set corresponding to each pathological layer;

[0031] A labeling unit, configured to label the sub-standard pathological image sample set corresponding to each pathological layer;

[0032] A learning unit, configured to learn according to the pathological large model for the labeling result to obtain pathological features corresponding to each pathological layer, and at the same time, fuse the pathological features corresponding to each pathological layer to obtain multi-level pathological features;

[0033] An analysis unit, configured to input the currently acquired target pathological image into the pathological large model, and parse the target pathological image according to the multi-level pathological features based on the pathological large model.

[0034] Optionally, the image acquisition unit includes:

[0035] An image processing subunit, configured to read the acquired pathological image sample set and determine the image pixel point distribution state of each pathological sample image;

[0036] A feature acquisition subunit, configured to acquire irrelevant region pixel features and key region pixel features;

[0037] A feature matching subunit, configured to perform a first match between the image pixel distribution state of each pathological sample image and the irrelevant region pixel features respectively, and a second match with the key region pixel features;

[0038] A localization subunit, configured to perform a first localization in each pathological sample image according to the first match result to obtain the irrelevant region of each pathological sample image, and perform a second localization in each pathological sample image according to the second match result to obtain the key region of each pathological sample image;

[0039] A region analysis subunit, configured to:

[0040] Stitch the irrelevant region and the key region of each pathological sample image, and determine whether the irrelevant region and the key region of each pathological sample image overlap according to the stitching result;

[0041] If the irrelevant region and the key region of the pathological sample image overlap, use the contour line of the key region as the cropping boundary line for the pathological sample image;

[0042] If the irrelevant region and the key region of the pathological sample image do not overlap, use the contour line of the irrelevant region as the cropping boundary line for the pathological sample image;

[0043] An image processing subunit, configured to determine the cropping boundary line of each pathological sample image according to the judgment result, and crop each pathological sample image according to the cropping boundary line of each pathological sample image to obtain the key region of each pathological sample image;

[0044] A synthesis subunit, configured to synthesize the key regions of each pathological sample image to obtain a standard pathological image sample set.

[0045] Optionally, the annotation unit includes:

[0046] A color annotation subunit, configured to read the sub-standard pathological image sample set corresponding to each pathological layer, determine the probability that each pixel point in the sub-standard pathological sample image set is cancer, and convert the probability into different colors for annotation;

[0047] A display subunit, configured to display the sub-standard pathological sample image set in the form of a heat map according to the annotation result.

[0048] Optionally, the recognition result output module includes:

[0049] A result determination unit, configured to obtain the analysis result of the target pathological image by the large pathological model, and determine the pancreatic cancer area existing in the target pathological image based on the analysis result;

[0050] A marking unit, configured to circle and mark the pancreatic cancer area, and determine the regional characteristics and distribution characteristics of each pancreatic cancer area based on the circled marking result;

[0051] A result output unit, configured to generate a pathological image cancer area recognition report based on the regional characteristics and distribution characteristics, bind the pathological image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathological image, and output the binding result.

[0052] This application also provides a method for automatically recognizing cancer areas in pathological images based on a large pathological model, including:

[0053] Step 1: Optimize the architecture of the basic pathological image model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathological model;

[0054] Step 2: Capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the large pathological model, and analyze the input target pathological image according to the multi-level pathological features;

[0055] Step 3: Output the recognition result of the pancreatic cancer area in the target pathological image based on the analysis result according to the large pathological model.

[0056] Optionally, in Step 1, optimizing the architecture of the basic pathological image model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning includes:

[0057] Obtain business requirements based on the management terminal;

[0058] Obtain a model library, where the model library includes multiple basic models;

[0059] Screen out the models that meet the business requirements in the preset model library according to the business requirements.

[0060] Determine the amount of computation required for the business requirements based on the business composition and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the amount of computation to obtain a set of available basic models;

[0061] Extract the basic configurations of each basic model in the set of available basic models, obtain the self-performance parameters of each basic model based on the basic configurations, and select the basic model with the best self-performance as the basic pathological image model.

[0062] The present invention provides a method for automatically recognizing cancer areas in pathological images based on a large pathological model, including:

[0063] Step 1: Optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathological model;

[0064] Step 2: Capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the large pathological model, and analyze the input target pathological image according to the multi-level pathological features;

[0065] Step 3: Output the recognition result of the pancreatic cancer area in the target pathological image based on the analysis result according to the large pathological model.

[0066] Preferably, for an automatic recognition method of the cancer area of a pathological image based on a large pathological model, in Step 1, optimizing the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning includes:

[0067] Obtain business requirements based on the management terminal, and analyze the business requirements to obtain the business composition and business execution purpose;

[0068] Determine the amount of computation required for the business requirements based on the business composition and business execution purpose, and perform conditional traversal of each basic model in the preset model library based on the amount of computation to obtain a set of available basic models;

[0069] Extract the basic configurations of each basic model in the set of available basic models, obtain the own performance parameters of each basic model based on the basic configurations, and select the basic model with the best own performance as the pathological image basic model.

[0070] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0071] Through deep learning technology, automatically identify the tumor area of pancreatic cancer and display it in the form of a heat map, realizing auxiliary pathological diagnosis and quantitative analysis of pathological features, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0072] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0073] The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments. Description of the Drawings

[0074] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0075] Figure 1 This is the structural diagram of a pathological image cancer area automatic recognition system based on a pathological large model in an embodiment of the present invention;

[0076] Figure 2 This is the structural diagram of the pathological large model construction module in a pathological image cancer area automatic recognition system based on a pathological large model in an embodiment of the present invention;

[0077] Figure 3 This is the flowchart of a pathological image cancer area automatic recognition method based on a pathological large model in an embodiment of the present invention.

[0078] Figure 4 This is the heat map of using a pathological large model to recognize a pancreatic cancer section in an embodiment of the present invention. Specific Embodiments

[0079] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0080] In one embodiment, a pathological image cancer area automatic recognition system based on a pathological large model is provided, as Figure 1 shown, including:

[0081] A pathological large model construction module, which is used to optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathological large model;

[0082] The core idea of the multi-head self-attention mechanism (Multi-Head Attention) is to perform parallel calculations through multiple different attention heads (heads). Each attention head independently learns different aspects of the input data, so as to capture more context information. Specifically, the multi-head self-attention mechanism is implemented through the following steps:

[0083] Linear transformation of the input: The input query (Query), key (Key), and value (Value) are respectively passed through multiple different linear transformations to obtain multiple "heads".

[0084] Independent self-attention calculation: Each head has its own self-attention calculation to obtain multiple outputs.

[0085] Concatenation and linear transformation of the output: The outputs of multiple heads are concatenated and passed through a linear transformation to obtain the final output representation.

[0086] The core formula is:

[0087] Multi-View Attention(Q,K,V) = Merge(subview1, subview2, …, subview h )A O

[0088] Where:

[0089]

[0090] Here, is the linear transformation matrix for each head, and A O is the linear transformation matrix of the output. Through the multi-head self-attention mechanism, the model can learn different context relationships from multiple subspaces, thereby capturing more information.

[0091] The core idea of Multi-Scale Feature Fusion is to capture the multi-scale information of the image by fusing feature maps of different scales, thereby improving the feature extraction ability and robustness of the model. Specifically, multi-scale feature fusion is achieved through the following steps:

[0092] 1. Feature extraction: Use feature maps of different scales to extract features at different levels.

[0093] 2. Feature fusion: Through the Dynamic Feature Fusion (DFF) mechanism, adaptively learn to fuse multi-scale features while reducing redundant information.

[0094] 3. Output: Use the fused feature map for the final classification or segmentation task.

[0095] The core formula is:

[0096]

[0097] Where F i are feature maps of different scales, and α i are the weights learned through the dynamic mechanism, used to adjust the importance of feature maps at different scales. Through multi-scale feature fusion, the model can better capture the multi-scale information of the image and improve the processing ability for complex data.

[0098] The parsing module is used to capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the pathological large model, and parse the input target pathological image according to the multi-level pathological features;

[0099] The recognition result output module is used to output the recognition result of the pancreatic cancer area in the target pathological image based on the parsing result according to the pathological large model.

[0100] In this embodiment, the multi-head self-attention mechanism refers to a mechanism that can focus on the features of pathological images from different aspects.

[0101] In this embodiment, multi-scale feature fusion refers to a mechanism for extracting (such as local features and global features, etc.) and fusing features of pathological images at different scales.

[0102] In this embodiment, the pathological image basic model refers to the basic model selected from the model framework according to business requirements and capable of adapting to the current business needs, such as a convolutional model, etc.

[0103] In this embodiment, the pathological large model refers to a tool that can effectively identify the pancreatic cancer area in pathological images.

[0104] In this embodiment, multi-level pathological features refer to the features presented under different dimensions of pancreatic cancer, so as to facilitate the effective identification of the pancreatic cancer area.

[0105] In this embodiment, the target pathological image refers to the image that needs to be analyzed currently, that is, the pathological image containing pancreatic cancer.

[0106] In this embodiment, through large model technology, combined with self-supervised pre-training, weakly-supervised pre-training, and supervised fine-tuning, while reducing the annotation burden, the model performance is greatly improved, enabling accurate identification of cancer areas in various different types of pathological sections, and greatly improving the diagnosis efficiency and accuracy of pathologists.

[0107] The working principle and beneficial effects of the above technical solution are: through deep learning technology, automatically identify the tumor area of pancreatic cancer and display it in the form of a heat map, realizing auxiliary pathological diagnosis and quantitative analysis of pathological features, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0108] In one embodiment, a pathological image cancer area automatic identification system based on a pathological large model is provided, as Figure 2 shown, the pathological large model construction module includes:

[0109] A requirement analysis unit, configured to obtain business requirements based on a management terminal and analyze the business requirements to obtain the business composition and business execution purpose; in this embodiment, the business execution purpose includes:

[0110] Classification task: Classify pathological images into different categories, such as cancerous or non-cancerous.

[0111] Detection task: Detect specific regions or features in pathological images, such as cancerous regions.

[0112] Segmentation task: Segment different tissues or cells in pathological images.

[0113] Analyze the business requirements to obtain the business composition and the purpose of business execution;

[0114] A basic model selection unit for:

[0115] Determine the amount of computation required for the business requirements based on the business composition and the purpose of business execution, and perform conditional traversal on each basic model in the preset model library based on the amount of computation to obtain a set of available basic models. For example, when processing 1000 high-resolution pathological images, it is necessary to evaluate the processing time of each high-resolution image, and determine the amount of computation based on the total time. In the preset model library, filter out the models that meet the amount of computation;

[0116] Specifically, extract the basic configurations of each basic model in the set of available basic models, obtain the self-performance parameters of each basic model based on the basic configurations, and select the basic model with the best self-performance as the pathological image basic model based on the self-performance parameters.

[0117] In this embodiment, the business requirements are set in advance to characterize the business composition and the purpose of business execution. Among them, the business composition is the type of business currently executed by the business requirements, and the purpose of business execution is the result corresponding to the type of business execution.

[0118] In this embodiment, the basic model set can be the set of basic models corresponding to when the amount of computation is reached. Among them, the preset model library stores basic models corresponding to different amounts of computation.

[0119] In this embodiment, the basic configuration is the model parameters and model composition of each basic model, etc.

[0120] In this embodiment, selecting the basic model with the best self-performance based on the self-performance parameters means selecting the best basic model from the set of basic models through the self-performance parameters.

[0121] The working principle and beneficial effects of the above technical solution are as follows: Obtain the business requirements through the management terminal and analyze them to clarify the business composition and the purpose of execution. Determine the amount of computation according to the analyzed business composition and the purpose of execution, and find the set of available basic models by performing conditional traversal on the basic models in the preset model library; then extract the basic configurations of each basic model in the set, obtain the self-performance parameters based on this, and finally select the one with the best self-performance as the pathological image basic model; it can specifically select the most suitable basic model according to the specific business requirements, improve the applicability and accuracy of the model; by considering the amount of computation and the model performance, optimize resource utilization and improve work efficiency; contribute to building a more accurate and efficient pathological large model.

[0122] In one embodiment, a pathological image cancer area automatic recognition system based on a pathological large model is provided. The pathological large model construction module includes:

[0123] A model optimization preparation unit for:

[0124] In our preset business system, it is clear how to extract features from input data, how to process these features, and how to use these features for the final business goal. When identifying the cancerous area of a pathological image based on the preset business system, obtain the attention dimensions of the input image, and at the same time, obtain the input features of each attention dimension, and convert the input features of each attention dimension into corresponding feature vector sequences;

[0125] Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrices of each attention dimension to obtain a multi-head self-attention mechanism;

[0126] At the same time, based on the preset business system, determine the feature extraction scale for identifying the cancerous area of the pathological image, and determine the hierarchical association relationship between different feature extraction scales. Features at the cellular level may be associated with features at the tissue level, and features at the tissue level may be associated with features at the image level, which is the hierarchical association relationship;

[0127] Determine a multi-scale feature fusion strategy between different feature extraction scales based on the hierarchical association relationship;

[0128] An architecture optimization unit for:

[0129] Obtain the structural parameters of the pathological image base model (the number of layers of the model, the size of the feature map of each layer, the size of the convolution kernel, etc.), and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the pathological image base model based on the structural parameters. The multi-head self-attention mechanism can be inserted into the middle layer of some residual blocks to enhance the model's attention to local features; the multi-scale feature fusion strategy can be inserted at the skip connection of the model to fuse features of different scales and improve the model's feature extraction ability;

[0130] Add the multi-head self-attention mechanism and the multi-scale feature fusion strategy to the pathological image base model based on the insertion positions to complete the architecture optimization of the pathological image base model and obtain a large pathological model.

[0131] In one embodiment, a pathological image cancer area automatic recognition system based on a large pathological model is provided. The pathological image cancer area automatic recognition system based on the large pathological model further includes a training unit, and the training unit includes:

[0132] A result acquisition subunit, configured to acquire the architecture optimization result of the obtained pathological image basic model, and select a loss function for the architecture optimization result from a preset function library based on the business requirements of the pathological large model (such as image classification, object detection, segmentation, generation, retrieval, etc. These task types determine the types of loss functions required for the model during training, such as cross-entropy for classification, Dice or IoU for segmentation, etc.);

[0133] A model testing subunit, configured to acquire test data, input the test data into the architecture optimization result of the pathological image basic model for analysis and recognition, and determine a loss value based on the loss function for the analysis and recognition result;

[0134] A model optimization and deployment subunit, configured to, when the loss value is greater than a preset threshold, re-optimize the architecture optimization result of the pathological image basic model (such as adjusting the number of network layers or structure, adding or reducing convolutional layers, modifying the connection method, improving residual connections, attention mechanisms, skip connections, etc.), then perform re-training and evaluation, and finally deploy the obtained pathological large model in the application environment.

[0135] In this embodiment, the preset threshold is set in advance and is used as a standard to measure whether the architecture optimization result of the pathological image basic model needs to be re-optimized.

[0136] The working principle and beneficial effects of the above technical solution are: by determining the architecture optimization result and determining the loss value according to the loss function, it is possible to effectively measure whether the architecture optimization result of the pathological image basic model needs to be re-optimized, thereby ensuring that the obtained pathological large model is more accurate and improving the accuracy of the pathological large model when applied in the application environment.

[0137] In one embodiment, a pathological image cancer area automatic recognition system based on a pathological large model is provided, and the parsing module includes:

[0138] An image acquisition unit, configured to:

[0139] Collect the pathological levels including the cell morphology to tissue structure of pancreatic cancer;

[0140] According to the pathological levels, collect pathological image sample sets of pancreatic cancer at different stages and subtypes, and perform image processing on the pathological sample image sets to obtain a standard pathological image sample set;

[0141] An image sample set partitioning unit, configured to partition the standard pathological image sample set according to the pathological levels to obtain a sub-standard pathological image sample set corresponding to each pathological layer;

[0142] A labeling unit, configured to label the sub-standard pathological image sample set corresponding to each pathological layer;

[0143] A learning unit for learning the annotation results according to a large pathological model to obtain pathological features corresponding to each pathological layer. At the same time, the pathological features corresponding to each pathological layer are fused to obtain multi-level pathological features;

[0144] An analysis unit for inputting the currently acquired target pathological image into the large pathological model and parsing the target pathological image according to the large pathological model based on the multi-level pathological features.

[0145] The working principle and beneficial effects of the above technical solution are as follows: By capturing multi-level pathological features from cell morphology to tissue structure, the pathological image is comprehensively analyzed, providing technical support for accurate diagnosis; Combining multi-perspective data processing and masked modeling, multi-level information extraction is achieved, adapting to the image analysis requirements of high resolution and complex structures, and reducing the dependence on labeled data.

[0146] In one embodiment, a pathological image cancer area automatic recognition system based on a large pathological model is provided. The image acquisition unit includes:

[0147] An image processing sub-unit for reading the collected pathological image sample set and determining the image pixel point distribution state of each pathological sample image;

[0148] A feature acquisition sub-unit for acquiring irrelevant region pixel point features and acquiring key region pixel point features;

[0149] A feature matching sub-unit for performing a first match between the image pixel point distribution state of each pathological sample image and the irrelevant region pixel point features respectively and a second match with the key region pixel point features;

[0150] A positioning sub-unit for performing a first positioning in each pathological sample image according to the first match result to obtain the irrelevant region of each pathological sample image, and performing a second positioning in each pathological sample image according to the second match result to obtain the key region of each pathological sample image;

[0151] A region analysis sub-unit for:

[0152] Stitching the irrelevant region and the key region of each pathological sample image and judging whether the irrelevant region and the key region of each pathological sample image overlap according to the stitching result;

[0153] If the irrelevant region and the key region of the pathological sample image overlap, then use the contour line of the key region as the cropping boundary line for the pathological sample image;

[0154] If the irrelevant region and the key region of the pathological sample image do not overlap, then use the contour line of the irrelevant region as the cropping boundary line for the pathological sample image;

[0155] An image processing subunit, configured to determine a cropping boundary line for each pathological sample image according to the judgment result, and crop each pathological sample image according to the cropping boundary line of each pathological sample image to obtain a key region of each pathological sample image;

[0156] An integration subunit, configured to integrate the key regions of each pathological sample image to obtain a standard pathological image sample set.

[0157] In this embodiment, in the region analysis subunit, determining whether the irrelevant region and the key region of each pathological sample image overlap according to the stitching result includes:

[0158] Obtaining the point (x p , y p ) of the irrelevant region A in the coordinate system. At the same time, obtaining the point (x q , y q ) of the key region B in the coordinate system, and according to the point (x p , y p ) of the irrelevant region A in the coordinate system and the point (x q , y q ) of the key region B in the coordinate system, constructing a first support function corresponding to the irrelevant region A and a second support function of the key region B;

[0159] h A (θ) = max p∈A (x p cosθ + y p sinθ);

[0160] h B (θ) = max q∈B (x q cosθ + y q sinθ);

[0161] Wherein, h A (θ) represents the first support function corresponding to the irrelevant region A; θ represents the rotation angle with respect to the x-axis; max(·) represents the maximum value function; p represents a point in the irrelevant region A; x p represents the abscissa value of the point in the irrelevant region A; cosθ represents the cosine value of the rotation angle θ with respect to the x-axis; y p represents the ordinate value of the point in the irrelevant region A; sinθ represents the sine value of the rotation angle θ with respect to the x-axis; h B (θ) represents the second support function of the key region B; q represents a point in the key region B; x q represents the abscissa value of the point in the key region B; y q represents the ordinate value of the point in the key region B;

[0162] Calculate the target difference between the first support function corresponding to the computationally irrelevant region A and the second support function of the critical region B;

[0163] d(θ) = h A (θ) - h B (θ);

[0164] where d(θ) represents the target difference;

[0165] Determine the minimum value of the target difference for θ in [0, 2π];

[0166] Compare the minimum value of the target difference with 0 to determine whether the irrelevant region and the critical region overlap;

[0167] If the minimum value of the target difference is less than or equal to 0, it is determined that the irrelevant region and the critical region overlap;

[0168] Otherwise, it is determined that the irrelevant region and the critical region do not overlap.

[0169] The beneficial effects of the above technical solution are: By calculating the first support function and the second support function respectively, the target difference can be effectively determined. Furthermore, by finding the minimum value of the target difference for θ in [0, 2π], it is possible to effectively determine whether the irrelevant region and the critical region overlap based on the minimum value, and it is possible to effectively implement the determination of regions with different shapes, improving the effectiveness and accuracy of determining overlap.

[0170] In one embodiment, a pathological image cancer region automatic recognition system based on a pathological large model is provided. The pathological image cancer region automatic recognition system based on the pathological large model includes an annotation unit, where the annotation unit includes:

[0171] A color annotation sub-unit for reading the sub-standard pathological image sample set corresponding to each pathological layer, determining the probability that each pixel point in the sub-standard pathological sample image set is cancerous, and converting the probability into different colors for annotation;

[0172] A display sub-unit for displaying the sub-standard pathological sample image set in the form of a heat map according to the annotation result.

[0173] See Figure 4 , in this embodiment, the probability that each pixel point is cancerous is converted into different colors, the greater the probability, the redder the color, and vice versa, the bluer. After filtering out the pixel points below the threshold, it is displayed in the form of a heat map.

[0174] The beneficial effects of the above technical solution are: It helps doctors quickly locate the cancer region, improves the doctor's diagnosis efficiency, and reduces the missed diagnosis rate.

[0175] In one embodiment, a pathological image cancer area automatic recognition system based on a pathological large model is provided. The pathological image cancer area automatic recognition system based on the pathological large model includes a recognition result output module. Among them, the recognition result output module includes:

[0176] A result determination unit, configured to obtain the analysis result of the target pathological image by the pathological large model, and determine the pancreatic cancer area existing in the target pathological image based on the analysis result;

[0177] A marking unit, configured to circle and mark the pancreatic cancer area, and determine the regional characteristics and distribution characteristics of each pancreatic cancer area based on the circled marking result;

[0178] A result output unit, configured to generate a pathological image cancer area recognition report based on the regional characteristics and distribution characteristics, bind the pathological image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathological image, and output the binding result.

[0179] In this embodiment, the circled marking refers to annotating the shape of the pancreatic cancer area and its distribution in the whole area, so as to determine the regional characteristics and distribution characteristics.

[0180] The working principle and beneficial effects of the above technical solution are: by circling and marking the pancreatic cancer area, the regional characteristics and distribution characteristics of the pancreatic cancer area are effectively realized, so as to accurately generate a pathological image cancer area recognition report. By binding and outputting the pathological image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathological image, it is beneficial to ensure the convenience and interpretability of report reading.

[0181] In one embodiment, a method for automatically recognizing a cancer area of a pathological image based on a pathological large model is provided, as Figure 3 shown, including:

[0182] Step 1: Optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathological large model;

[0183] Step 2: Capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the pathological large model, and analyze the input target pathological image according to the multi-level pathological features;

[0184] Step 3: Output the recognition result of the pancreatic cancer area in the target pathological image based on the analysis result according to the pathological large model.

[0185] The working principle and beneficial effects of the above technical solution are: through deep learning technology, the tumor area of pancreatic cancer is automatically recognized and displayed in the form of a heat map, so as to realize auxiliary pathological diagnosis and quantitative analysis of pathological features, and improve the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0186] In one embodiment, an automatic recognition method for cancer regions in pathological images based on a pathological large model is provided. In step 1, the architecture of the pathological image basic model is optimized according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning, including:

[0187] Obtain business requirements based on the management terminal, and analyze the business requirements to obtain the business composition and business execution purpose;

[0188] Determine the amount of computation required for the business requirements based on the business composition and business execution purpose, and perform conditional traversal of each basic model in the preset model library based on the amount of computation to obtain a set of available basic models;

[0189] Extract the basic configurations of each basic model in the set of available basic models, obtain the self-performance parameters of each basic model based on the basic configurations, and select the basic model with the best self-performance as the pathological image basic model.

[0190] The working principle and beneficial effects of the above technical solution are as follows: Obtain business requirements through the management terminal and analyze them to clarify the business composition and execution purpose. Determine the amount of computation according to the analyzed business composition and execution purpose, and find a set of available basic models by performing conditional traversal of the basic models in the preset model library; then extract the basic configurations of each basic model in the set, obtain the self-performance parameters based on this, and finally select the one with the best self-performance as the pathological image basic model; it can specifically select the most suitable basic model according to the specific business requirements, improve the applicability and accuracy of the model; optimize resource utilization and improve work efficiency by considering the amount of computation and model performance; contribute to building a more accurate and efficient pathological large model.

[0191] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A pathological image cancer area automatic recognition system based on a large pathological model, characterized in that Including: A pathological large model construction module, which is used to optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathological large model; An analysis module, which is used to capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the pathological large model, and analyze the input target pathological image according to the multi-level pathological features; An identification result output module, which is used to output the identification result of the pancreatic cancer area in the target pathological image based on the analysis result according to the pathological large model.

2. The automatic cancer region recognition system for pathological images based on a large pathological model according to claim 1, characterized in that, The pathological large model construction module includes: A requirement analysis unit, which is used to obtain the business requirements of pancreatic cancer based on the management terminal; A model library acquisition unit, which is used to acquire a model library, and the model library includes multiple basic models; A basic model selection unit, which is used to screen out the models that meet the business requirements in the preset model library according to the business requirements.

3. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 1, characterized in that, The pathological large model construction module includes: A model optimization preparation unit, which is used for: When identifying the cancer area of the pathological image based on the preset business system, obtain the attention dimensions of the input image, and at the same time, obtain the input features of each attention dimension, and convert the input features of each attention dimension into corresponding feature vector sequences; Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrix of each attention dimension to obtain a multi-head self-attention mechanism; At the same time, based on the preset business system, determine the feature extraction scales when identifying the cancer area of the pathological image, and determine the association and dependence relationships between different feature extraction scales; determine the multi-scale feature fusion strategy between different feature extraction scales based on the hierarchical association relationship; An architecture optimization unit, which is used for: Obtain the structural parameters of the pathological image basic model, and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the pathological image basic model based on the structural parameters; Add the multi-head self-attention mechanism and the multi-scale feature fusion strategy to the pathological image basic model based on the insertion positions to complete the architecture optimization of the pathological image basic model and obtain a pathological large model.

4. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 3, characterized in that, The pathological image cancer area automatic identification system based on the pathological large model further includes a training unit, and the training unit includes: A result acquisition subunit, which is used to obtain the architecture optimization result of the obtained pathological image basic model, and select the loss function of the architecture optimization result from the preset function library based on the business requirements of the pathological large model; A model test subunit, which is used to obtain test data, input the test data into the architecture optimization result of the pathological image basic model for analysis and identification, and determine the loss value based on the loss function for the analysis and identification result; A model optimization and deployment subunit, which is used to re-optimize the architecture optimization result of the pathological image basic model when the loss value is greater than the preset threshold, then perform re-training and evaluation, and finally deploy the obtained pathological large model in the application environment.

5. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 1, characterized in that, The analysis module includes: An image acquisition unit, which is used for: Collect the pathological levels including the cell morphology to the tissue structure of pancreatic cancer; Collect a set of pathological image samples of pancreatic cancer at different stages and subtypes according to pathological levels, and perform image processing on the pathological sample image set to obtain a standard pathological image sample set; An image sample set division unit for dividing the standard pathological image sample set according to pathological levels to obtain a sub-standard pathological image sample set corresponding to each pathological layer; A labeling unit for labeling the sub-standard pathological image sample set corresponding to each pathological layer; A learning unit for learning the labeling results according to a pathological large model to obtain pathological features corresponding to each pathological layer. At the same time, fuse the pathological features corresponding to each pathological layer to obtain multi-level pathological features; An analysis unit for inputting the currently obtained target pathological image into the pathological large model and parsing the target pathological image according to the pathological large model based on the multi-level pathological features.

6. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 5, characterized in that, The image acquisition unit includes: An image processing sub-unit for reading the collected pathological image sample set and determining the image pixel point distribution state of each pathological sample image; A feature acquisition sub-unit for acquiring irrelevant region pixel point features and acquiring key region pixel point features; A feature matching sub-unit for respectively performing a first match between the image pixel point distribution state of each pathological sample image and the irrelevant region pixel point features and a second match with the key region pixel point features; A positioning sub-unit for performing a first positioning in each pathological sample image according to the first match result to obtain the irrelevant region of each pathological sample image, and performing a second positioning in each pathological sample image according to the second match result to obtain the key region of each pathological sample image; A region analysis sub-unit for: Stitching the irrelevant region and the key region of each pathological sample image, and judging whether the irrelevant region and the key region of each pathological sample image overlap according to the stitching result; If the irrelevant region and the key region of the pathological sample image overlap, then use the contour line of the key region as the cropping boundary line for the pathological sample image; If the irrelevant region and the key region of the pathological sample image do not overlap, then use the contour line of the irrelevant region as the cropping boundary line for the pathological sample image; An image processing sub-unit for determining the cropping boundary line of each pathological sample image according to the judgment result, and cropping each pathological sample image according to the cropping boundary line of each pathological sample image to obtain the key region of each pathological sample image; A synthesis sub-unit for synthesizing the key regions of each pathological sample image to obtain a standard pathological image sample set.

7. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 5, characterized in that The labeling unit includes: A color labeling sub-unit for reading the sub-standard pathological image sample set corresponding to each pathological layer, determining the probability that each pixel point in the sub-standard pathological sample image set is cancer, and converting the probability into different colors for labeling; A display sub-unit for displaying the sub-standard pathological image sample set in the form of a heat map according to the labeling result.

8. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 1, characterized in that, The recognition result output module includes: A result determination unit for obtaining the parsing result of the pathological large model for the target pathological image and determining the pancreatic cancer area existing in the target pathological image based on the parsing result; A marking unit, configured to demarcate and mark pancreatic cancer regions, and determine the regional characteristics and distribution characteristics of each pancreatic cancer region based on the demarcation and marking results; A result output unit, configured to generate a pathological image cancer region recognition report based on the regional characteristics and distribution characteristics, bind the pathological image cancer region recognition report to the target recognition image of the pancreatic cancer region in the target pathological image, and output the binding result.

9. An automatic recognition method for cancer regions in pathological images based on a large pathological model, characterized in that, It includes: Step 1: Optimize the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathological large model; Step 2: Capture multi-level pathological features from the cell morphology to the tissue structure of pancreatic cancer according to the pathological large model, and analyze the input target pathological image according to the multi-level pathological features; Step 3: Output the recognition result of the pancreatic cancer region in the target pathological image based on the analysis result according to the pathological large model.

10. The automatic recognition method for cancer regions of pathological images based on a large pathological model according to claim 9, characterized in that In Step 1, optimizing the architecture of the pathological image basic model according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning includes: Obtain business requirements based on the management terminal; Obtain a model library, where the model library includes multiple basic models; Screen out the models that meet the business requirements in the preset model library according to the business requirements. Determine the amount of computation required for the business requirements based on the business composition and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the amount of computation to obtain a set of available basic models; Extract the basic configurations of each basic model in the set of available basic models, obtain the self-performance parameters of each basic model based on the basic configurations, and select the basic model with the best self-performance as the pathological image basic model.

Images