A system and method for automatically identifying cancerous areas in pathological images based on a large pathological model

Abstract

The present invention provides a system and method for automatically identifying cancerous areas in pathology images based on a large-scale pathology model. The system comprises: a large-scale pathology model construction module, which optimizes the architecture of the basic pathology image model using a deep learning multi-head self-attention mechanism and multi-scale feature fusion to obtain the large-scale pathology model; a parsing module, which captures multi-level pathological features of pancreatic cancer, from cell morphology to tissue structure, based on the large-scale pathology model and parses the input target pathology image based on these multi-level pathological features; and a recognition result output module, which outputs the recognition result of pancreatic cancer areas in the target pathology image based on the parsed results of the large-scale pathology model. Using deep learning technology, pancreatic cancer tumor areas are automatically identified and displayed as heat maps, enabling auxiliary pathological diagnosis and quantitative analysis of pathological features, thereby improving the efficiency and accuracy of pancreatic cancer pathology diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a system and method for automatically identifying cancerous areas in pathological images based on a large pathological model. Background Art

[0002] Currently, pancreatic cancer has an extremely poor prognosis, with morbidity and mortality rates continuing to rise. Pathology is the "gold standard" for pancreatic cancer diagnosis, running through the diagnosis and treatment process and providing an important basis for pancreatic cancer treatment decisions.

[0003] However, the pathological diagnosis of pancreatic cancer faces numerous challenges, making quality assurance difficult. First, pancreatic cancer has unique pathological characteristics, with stroma comprising over 80% of the tumor, and the tumor interspersed with normal pancreas and pancreatitis, making accurate size assessment difficult with the naked eye. Second, the pathological morphology of the neoplastic epithelium is very similar to that of the reactive epithelium of chronic pancreatitis, and pancreatic cancer is often associated with pancreatitis, making differential diagnosis of the two extremely difficult, especially in biopsies with small and fragmented tissue and frozen sections with altered tissue morphology. Furthermore, the assessment of lymph node invasion is time-consuming and labor-intensive, and junior physicians are prone to miss small cancer foci. Therefore, accurate assessment of pancreatic cancer requires extremely high expertise and rigor. In recent years, artificial intelligence (AI) and whole-slide imaging (WSI) technologies have developed rapidly, and AI-assisted diagnosis systems have been widely used in various cancer types, including prostate cancer and cervical cancer. However, the diagnosis and annotation of pancreatic cancer are extremely difficult, and no clinically applicable pancreatic cancer-assisted diagnosis models have yet to be developed.

[0004] Therefore, in order to overcome the above technical problems, the present invention provides a system and method for automatically identifying cancerous areas in pathological images based on a large pathological model. Summary of the Invention

[0005] The present invention provides a system and method for automatically identifying cancerous areas in pathological images based on a large pathological model. The system is used to automatically identify pancreatic cancer tumor areas through deep learning technology and display them in the form of a heat map, thereby assisting pathological diagnosis, quantitatively analyzing pathological characteristics, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0006] The present invention provides a system for automatically identifying cancerous areas in pathological images based on a large pathological model, comprising:

[0007] The pathology model building module is used to optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathology model.

[0008] The parsing module is used to capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features;

[0009] The recognition result output module is used to output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

[0010] Optionally, the pathology model building module includes:

[0011] A demand analysis unit, used to obtain the business requirements of pancreatic cancer based on the management terminal;

[0012] A model library acquisition unit, the model library acquisition unit is used to acquire a model library, the model library includes a plurality of basic models;

[0013] The basic model selection unit is used to screen out models that meet business needs from a preset model library based on business needs.

[0014] Optionally, the pathology model building module includes:

[0015] Model optimization preparation unit for:

[0016] When identifying cancerous areas in pathological images based on a preset business system, the focus dimension of the input image is obtained, and at the same time, the input features of each focus dimension are obtained, and the input features of each focus dimension are converted into a corresponding feature vector sequence;

[0017] Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrix of each attention dimension to obtain a multi-head self-attention mechanism;

[0018] At the same time, based on the preset business system, the feature extraction scale for identifying cancerous areas in pathological images is determined, and the correlation and dependency between different feature extraction scales are determined; based on the hierarchical correlation relationship, the multi-scale feature fusion strategy between different feature extraction scales is determined;

[0019] Architecture Optimization Unit, used to:

[0020] Obtain the structural parameters of the basic model of pathological images, and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the basic model of pathological images based on the structural parameters;

[0021] Based on the insertion position, the multi-head self-attention mechanism and multi-scale feature fusion strategy are added to the basic model of pathological images to complete the architectural optimization of the basic model of pathological images and obtain a large pathological model.

[0022] Optionally, the automatic cancer area recognition system for pathological images based on a large pathological model further includes a training unit, which includes:

[0023] A result acquisition subunit is used to obtain the architecture optimization result of the obtained pathological image basic model and select the loss function of the architecture optimization result from a preset function library based on the task type of the pathological large model;

[0024] The model testing subunit is used to obtain test data, input the test data into the architecture optimization results of the pathology image basic model for analysis and recognition, and determine the loss value of the analysis and recognition results based on the loss function;

[0025] The model optimization and deployment subunit is used to re-optimize the architecture optimization results of the pathology image basic model when the loss value is greater than the preset threshold, and deploy the obtained pathology large model in the application environment after the architecture optimization is completed.

[0026] Optionally, the parsing module includes:

[0027] Image acquisition unit, used for:

[0028] The collection includes pathological levels from pancreatic cancer cell morphology to tissue structure;

[0029] Pathological image sample sets of pancreatic cancer at different stages and subtypes are collected according to the pathological level, and the pathological sample image sets are processed to obtain standard pathological image sample sets;

[0030] An image sample set division unit is used to divide the standard pathology image sample set according to the pathology level to obtain a sub-standard pathology image sample set corresponding to each pathology level;

[0031] a labeling unit, configured to label a substandard pathology image sample set corresponding to each pathology layer;

[0032] The learning unit is used to learn the annotation results based on the pathology model to obtain the pathology features corresponding to each pathology layer. At the same time, the pathology features corresponding to each pathology layer are integrated to obtain multi-level pathology features.

[0033] The analysis unit is used to input the currently acquired target pathology image into the pathology big model, and analyze the target pathology image based on the multi-level pathology features according to the pathology big model.

[0034] Optionally, the image acquisition unit includes:

[0035] An image processing subunit is used to read the collected pathological image sample set and determine the image pixel distribution state of each pathological sample image;

[0036] A feature acquisition subunit is used to acquire features of pixels in irrelevant areas and features of pixels in key areas;

[0037] A feature matching subunit, configured to perform a first matching of the image pixel distribution state of each pathological sample image with the pixel features of the irrelevant area and a second matching of the pixel features of the key area;

[0038] a positioning subunit, configured to perform a first positioning in each pathological sample image according to the first matching result to obtain an irrelevant area of ​​each pathological sample image, and perform a second positioning in each pathological sample image according to the second matching result to obtain a key area of ​​each pathological sample image;

[0039] Regional Analysis Subunit, used to:

[0040] Splicing the irrelevant area and the key area of ​​each pathological sample image, and judging whether the irrelevant area and the key area of ​​each pathological sample image overlap according to the splicing result;

[0041] If the irrelevant area of ​​the pathological sample image overlaps with the key area, the contour line of the key area is used as the cropping boundary line of the pathological sample image;

[0042] If the irrelevant area of ​​the pathological sample image does not overlap with the key area, the contour line of the irrelevant area is used as the cropping boundary line of the pathological sample image;

[0043] an image processing subunit, configured to determine a clipping boundary line for each pathological sample image according to the judgment result, and clip each pathological sample image according to the clipping boundary line of each pathological sample image to obtain a key area of ​​each pathological sample image;

[0044] The synthesis subunit is used to synthesize the key areas of each pathological sample image to obtain a standard pathological image sample set.

[0045] Optionally, the marking unit includes:

[0046] The color labeling subunit is used to read the substandard pathology image sample set corresponding to each pathology layer, determine the probability of each pixel in the substandard pathology sample image set being cancer, and convert the probability into different colors for labeling;

[0047] The display subunit is used to display the substandard pathology sample image set in the form of a heat map based on the annotation results.

[0048] Optionally, the recognition result output module includes:

[0049] a result determination unit, configured to obtain an analysis result of the target pathology image by the pathology large model, and determine a pancreatic cancer region existing in the target pathology image based on the analysis result;

[0050] a marking unit, configured to delineate and mark the pancreatic cancer areas, and determine the regional characteristics and distribution characteristics of each pancreatic cancer area based on the delineation and marking results;

[0051] The result output unit is used to generate a pathology image cancer area recognition report based on regional features and distribution features, bind the pathology image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathology image, and output the binding result.

[0052] This application also provides a method for automatically identifying cancerous areas in pathological images based on a large pathological model, comprising:

[0053] Step 1: Optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathology model;

[0054] Step 2: Capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features;

[0055] Step 3: Output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

[0056] Optionally, in step 1, the architecture of the pathology image basic model is optimized based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning, including:

[0057] Obtain business requirements based on management terminals;

[0058] Acquire a model library, wherein the model library includes a plurality of basic models;

[0059] Filter out models that meet your business needs from the preset model library based on your business needs.

[0060] Determine the computing power required for business needs based on the business structure and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the computing power to obtain a set of available basic models;

[0061] The basic configuration of each basic model in the available basic model set is extracted, and the performance parameters of each basic model are obtained based on the basic configuration. The basic model with the best performance is selected as the pathological image basic model based on its own performance parameters.

[0062] The present invention provides a method for automatically identifying cancerous areas in pathological images based on a large pathological model, comprising:

[0063] Step 1: Optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathology model;

[0064] Step 2: Capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features;

[0065] Step 3: Output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

[0066] Preferably, a method for automatically identifying cancerous areas in pathological images based on a large pathological model, in step 1, the architecture of the basic model of pathological images is optimized according to the multi-head self-attention mechanism and multi-scale feature fusion of deep learning, including:

[0067] Obtain business requirements based on the management terminal, analyze the business requirements, and obtain the business structure and business execution objectives;

[0068] Determine the computing power required for business needs based on the business structure and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the computing power to obtain a set of available basic models;

[0069] The basic configuration of each basic model in the available basic model set is extracted, and the performance parameters of each basic model are obtained based on the basic configuration. The basic model with the best performance is selected as the pathological image basic model based on its own performance parameters.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] Through deep learning technology, pancreatic cancer tumor areas are automatically identified and displayed in the form of heat maps, enabling auxiliary pathological diagnosis, quantitative analysis of pathological characteristics, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0072] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0073] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0075] Figure 1 This is a structural diagram of a system for automatically identifying cancerous areas in pathological images based on a large pathological model in an embodiment of the present invention;

[0076] Figure 2 This is a structural diagram of a pathology big model construction module in a pathology image cancer area automatic recognition system based on a pathology big model in an embodiment of the present invention;

[0077] Figure 3 The figure is a flow chart of a method for automatically identifying cancerous areas in pathological images based on a large pathological model in an embodiment of the present invention.

[0078] Figure 4 This is a heat map of a pancreatic cancer slice identified using a large pathology model in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0080] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. Figure 1 Shown, including:

[0081] The pathology model building module is used to optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathology model.

[0082] The core idea of ​​the Multi-Head Attention mechanism is to use multiple different attention heads to perform parallel computations, with each attention head independently learning different aspects of the input data, thereby capturing more contextual information. Specifically, the Multi-Head Self-Attention mechanism is implemented through the following steps:

[0083] Linear transformation of input: The input query, key, and value are transformed through multiple different linear transformations to obtain multiple "headers".

[0084] Independent self-attention calculation: Each head has its own self-attention calculation, resulting in multiple outputs.

[0085] Output concatenation and linear transformation: The outputs of multiple heads are concatenated and subjected to a linear transformation to obtain the final output representation.

[0086] The core formula is:

[0087] Multi-View Attention(Q,K,V)=Merge(subview1,subview2,…,subview h )A O

[0088] in:

[0089]

[0090] here, is the linear transformation matrix of each head, A O is the linear transformation matrix of the output. Through the multi-head self-attention mechanism, the model can learn different contextual relationships from multiple subspaces, thereby capturing more information.

[0091] The core idea of ​​multi-scale feature fusion is to capture the multi-scale information of the image by fusing feature maps of different scales, thereby improving the feature extraction ability and robustness of the model. Specifically, multi-scale feature fusion is achieved through the following steps:

[0092] 1. Feature extraction: Use feature maps of different scales to extract features at different levels.

[0093] 2. Feature fusion: Through the Dynamic Feature Fusion (DFF) mechanism, multi-scale features are adaptively learned and fused while reducing redundant information.

[0094] 3. Output: The fused feature map is used for the final classification or segmentation task.

[0095] The core formula is:

[0096]

[0097] Among them, F i are feature maps of different scales, α i These are weights learned through a dynamic mechanism and used to adjust the importance of feature maps at different scales. By fusing multi-scale features, the model can better capture the multi-scale information of the image and improve its ability to process complex data.

[0098] The parsing module is used to capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features;

[0099] The recognition result output module is used to output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

[0100] In this embodiment, the multi-head self-attention mechanism refers to a mechanism that can focus on the features of pathological images from different aspects.

[0101] In this embodiment, multi-scale feature fusion refers to a mechanism for extracting and fusing features of different scales (eg, local features and global features) from a pathological image.

[0102] In this embodiment, the pathological image basic model refers to a basic model selected from the model framework according to business needs and capable of adapting to current business needs, such as a convolutional model.

[0103] In this embodiment, the pathology large model refers to a tool that can effectively identify pancreatic cancer areas in pathology images.

[0104] In this embodiment, the multi-level pathological features refer to the features presented in different dimensions of pancreatic cancer, thereby facilitating the effective identification of pancreatic cancer areas.

[0105] In this embodiment, the target pathological image refers to the image currently to be analyzed, that is, the pathological image containing pancreatic cancer.

[0106] In this embodiment, through large model technology, combined with self-supervised pre-training, weakly supervised pre-training, and supervised fine-tuning, the labeling burden is reduced while the model performance is greatly improved, achieving accurate identification of cancer areas in various types of pathological sections, and greatly improving the diagnostic efficiency and accuracy of pathologists.

[0107] The working principle and beneficial effects of the above technical solution are: through deep learning technology, the tumor area of ​​pancreatic cancer is automatically identified and displayed in the form of a heat map, thereby assisting pathological diagnosis, quantitatively analyzing pathological characteristics, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0108] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. Figure 2 As shown, the pathology model building module includes:

[0109] The demand parsing unit is used to obtain business requirements based on the management terminal and parse the business requirements to obtain business components and business execution objectives. In this embodiment, the business execution objectives include:

[0110] Classification task: Classify pathology images into different categories, such as cancerous or non-cancerous.

[0111] Detection task: Detect specific regions or features in pathology images, such as cancerous areas.

[0112] Segmentation task: Segment different tissues or cells in pathological images.

[0113] And analyze the business needs to obtain the business structure and business execution purpose;

[0114] Basic model selection unit, used for:

[0115] Determine the computing power required for business needs based on the business structure and business execution objectives, and perform conditional traversal of each basic model in the preset model library based on the computing power to obtain a set of available basic models. For example, to process 1,000 high-resolution pathology images, it is necessary to evaluate the processing time of each high-resolution image and determine the computing power based on the total time. In the preset model library, select the model that meets the computing power requirements.

[0116] Specifically, the basic configuration of each basic model in the available basic model set is extracted, and the performance parameters of each basic model are obtained based on the basic configuration, and the basic model with the best performance is selected as the pathological image basic model based on its own performance parameters.

[0117] In this embodiment, the business requirements are set in advance and are used to represent the business composition and business execution purpose, wherein the business composition is the type of business currently being executed by the business requirements, and the business execution purpose is the result achieved corresponding to the business execution type.

[0118] In this embodiment, the basic model set may be a set of basic models corresponding to when the calculation amount is reached, wherein the preset model library stores basic models corresponding to different calculation amounts.

[0119] In this embodiment, the basic configuration includes the model parameters and model structure of each basic model.

[0120] In this embodiment, selecting the basic model with the best performance based on its own performance parameters refers to selecting the best basic model from the basic model set based on its own performance parameters.

[0121] The working principle and beneficial effects of the above technical solution are: obtaining and analyzing business needs through the management terminal, clarifying the business composition and execution purpose, determining the computing amount based on the analyzed business composition and execution purpose, and finding the available basic model set by conditional traversal of the basic models in the preset model library; then extracting the basic configuration of each basic model in the set, and based on this, obtaining its own performance parameters, and finally selecting the one with the best performance as the basic model of pathological images; being able to select the most appropriate basic model according to specific business needs, thereby improving the applicability and accuracy of the model; optimizing resource utilization and improving work efficiency by considering the computing amount and model performance; and helping to build a more accurate and efficient large pathology model.

[0122] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. The large pathological model construction module includes:

[0123] Model optimization preparation unit for:

[0124] Our pre-defined business system specifies how to extract features from input data, how to process these features, and how to use them for the ultimate business goal. Based on this pre-defined business system, when identifying cancerous areas in pathology images, we obtain the input dimensions of interest for the input image, simultaneously obtain the input features for each dimension of interest, and convert each input feature of the dimension of interest into a corresponding feature vector sequence.

[0125] Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrix of each attention dimension to obtain a multi-head self-attention mechanism;

[0126] At the same time, based on the preset business system, the feature extraction scale for identifying cancerous areas in pathological images is determined, and the hierarchical correlation between different feature extraction scales is determined. Cell-level features may be correlated with tissue-level features, and tissue-level features may be correlated with image-level features. This is a hierarchical correlation.

[0127] Determine the multi-scale feature fusion strategy between different feature extraction scales based on hierarchical correlation relationships;

[0128] Architecture Optimization Unit, used to:

[0129] Obtain the structural parameters of the basic model of pathological images (number of layers, feature map size of each layer, convolution kernel size, etc.), and determine the insertion position of the multi-head self-attention mechanism and multi-scale feature fusion strategy in the basic model of pathological images based on the structural parameters. The multi-head self-attention mechanism can be inserted in the middle layer of certain residual blocks to enhance the model's attention to local features; the multi-scale feature fusion strategy can be inserted at the jump connection of the model to fuse features of different scales and improve the model's feature extraction ability;

[0130] Based on the insertion position, the multi-head self-attention mechanism and multi-scale feature fusion strategy are added to the basic model of pathological images to complete the architectural optimization of the basic model of pathological images and obtain a large pathological model.

[0131] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. The system further includes a training unit. The training unit includes:

[0132] The result acquisition subunit is used to obtain the architecture optimization results of the pathology image basic model and select the loss function of the architecture optimization results from the preset function library based on the business requirements of the pathology large model (such as image classification, object detection, segmentation, generation, retrieval, etc. These task types determine the type of loss function required for model training, such as cross entropy for classification, Dice or IoU for segmentation, etc.);

[0133] The model testing subunit is used to obtain test data, input the test data into the architecture optimization results of the pathology image basic model for analysis and recognition, and determine the loss value of the analysis and recognition results based on the loss function;

[0134] The model optimization and deployment subunit is used to re-optimize the architecture optimization results of the pathology image basic model when the loss value is greater than the preset threshold (such as adjusting the number of network layers or structure, adding or reducing convolutional layers, modifying the connection method, improving residual connections, attention mechanisms, jump connections, etc.), and then retrain and evaluate, and finally deploy the obtained pathology model in the application environment.

[0135] In this embodiment, the preset threshold is set in advance and is used as a standard to measure whether the architecture optimization result of the pathological image basic model needs to be re-optimized.

[0136] The working principle and beneficial effects of the above technical solution are: by determining the architecture optimization results and determining the loss value based on the loss function, it is possible to effectively measure whether the architecture optimization results of the pathology image basic model need to be re-optimized, thereby ensuring that the obtained pathology large model is more accurate and improving the accuracy of the pathology large model when applied in the application environment.

[0137] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided, wherein the parsing module includes:

[0138] Image acquisition unit, used for:

[0139] The collection includes pathological levels from pancreatic cancer cell morphology to tissue structure;

[0140] Pathological image sample sets of pancreatic cancer at different stages and subtypes are collected according to the pathological level, and the pathological sample image sets are processed to obtain standard pathological image sample sets;

[0141] An image sample set division unit is used to divide the standard pathology image sample set according to the pathology level to obtain a sub-standard pathology image sample set corresponding to each pathology level;

[0142] a labeling unit, configured to label a substandard pathology image sample set corresponding to each pathology layer;

[0143] The learning unit is used to learn the annotation results based on the pathology model to obtain the pathology features corresponding to each pathology layer. At the same time, the pathology features corresponding to each pathology layer are integrated to obtain multi-level pathology features.

[0144] The analysis unit is used to input the currently acquired target pathology image into the pathology big model, and analyze the target pathology image based on the multi-level pathology features according to the pathology big model.

[0145] The working principle and beneficial effects of the above technical solution are: by capturing multi-level pathological features from cell morphology to tissue structure, comprehensively analyzing pathological images, and providing technical support for accurate diagnosis; combining multi-view data processing and mask modeling, it realizes multi-level information extraction, adapts to the needs of high-resolution and complex structure image analysis, and reduces dependence on labeled data.

[0146] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. The image acquisition unit includes:

[0147] An image processing subunit is used to read the collected pathological image sample set and determine the image pixel distribution state of each pathological sample image;

[0148] A feature acquisition subunit is used to acquire features of pixels in irrelevant areas and features of pixels in key areas;

[0149] A feature matching subunit, configured to perform a first matching of the image pixel distribution state of each pathological sample image with the pixel features of the irrelevant area and a second matching of the pixel features of the key area;

[0150] a positioning subunit, configured to perform a first positioning in each pathological sample image according to the first matching result to obtain an irrelevant area of ​​each pathological sample image, and perform a second positioning in each pathological sample image according to the second matching result to obtain a key area of ​​each pathological sample image;

[0151] Regional Analysis Subunit, used to:

[0152] Splicing the irrelevant area and the key area of ​​each pathological sample image, and judging whether the irrelevant area and the key area of ​​each pathological sample image overlap according to the splicing result;

[0153] If the irrelevant area of ​​the pathological sample image overlaps with the key area, the contour line of the key area is used as the cropping boundary line of the pathological sample image;

[0154] If the irrelevant area of ​​the pathological sample image does not overlap with the key area, the contour line of the irrelevant area is used as the cropping boundary line of the pathological sample image;

[0155] an image processing subunit, configured to determine a clipping boundary line for each pathological sample image according to the judgment result, and clip each pathological sample image according to the clipping boundary line of each pathological sample image to obtain a key area of ​​each pathological sample image;

[0156] The synthesis subunit is used to synthesize the key areas of each pathological sample image to obtain a standard pathological image sample set.

[0157] In this embodiment, the region analysis subunit determines whether the irrelevant region of each pathological sample image overlaps with the key region based on the stitching result, including:

[0158] Get the point (x p ,y p ), and at the same time, obtain the point (x q ,y q ), and according to the point (x p ,y p ) and the key area B in the coordinate system (x q ,y q ), construct the first support function corresponding to the irrelevant area A and the second support function of the key area B;

[0159] h A (θ)=max p∈A (x p cosθ+y p sinθ);

[0160] h B (θ)=max q∈B (x q cosθ+y q sinθ);

[0161] Among them, h A (θ) represents the first support function corresponding to the irrelevant region A; θ represents the rotation angle relative to the x-axis; max(·) represents the maximum value function; p represents a point in the irrelevant region A; x p represents the abscissa value of the midpoint of the irrelevant area A; cosθ represents the cosine value of the rotation angle θ relative to the x-axis; y p represents the ordinate value of the midpoint of the irrelevant area A; sinθ represents the sine value of the rotation angle θ relative to the x-axis; h B (θ) represents the second support function of the key region B; q represents a point in the key region B; x q Indicates the horizontal coordinate value of the midpoint of the key area B; y q Indicates the ordinate value of the midpoint of key area B;

[0162] Calculate the target difference between the first support function corresponding to the irrelevant area A and the second support function corresponding to the key area B;

[0163] d(θ)=h A (θ)-h B (θ);

[0164] Where d(θ) represents the target difference;

[0165] Determine the minimum value of the target difference between θ and θ in [0, 2π];

[0166] Compare the minimum value of the target difference with 0 to determine whether the irrelevant area overlaps with the key area;

[0167] If the minimum value of the target difference is less than or equal to 0, it is determined that the irrelevant area overlaps with the key area;

[0168] Otherwise, it is determined that the irrelevant area does not overlap with the key area.

[0169] The beneficial effect of the above technical solution is: by calculating the first support function and the second support function respectively, the target difference can be effectively determined, and then the minimum value of the target difference can be found in [0, 2π] by θ, which can effectively determine whether the irrelevant area and the key area overlap based on the minimum value, and can effectively determine areas of different shapes, thereby improving the effectiveness and accuracy of determining whether there is overlap.

[0170] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. The system includes a labeling unit, wherein the labeling unit includes:

[0171] The color labeling subunit is used to read the substandard pathology image sample set corresponding to each pathology layer, determine the probability of each pixel in the substandard pathology sample image set being cancer, and convert the probability into different colors for labeling;

[0172] The display subunit is used to display the substandard pathology sample image set in the form of a heat map based on the annotation results.

[0173] See also Figure 4 In this embodiment, the probability of each pixel being cancerous is converted into different colors. The higher the probability, the redder the color, and vice versa. After filtering out pixels below the threshold, the image is displayed in the form of a heat map.

[0174] The beneficial effects of the above technical solution are: helping doctors quickly locate cancer areas, improving doctors' diagnostic efficiency, and reducing the missed diagnosis rate.

[0175] In one embodiment, a system for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. The system includes a recognition result output module, wherein the recognition result output module includes:

[0176] a result determination unit, configured to obtain an analysis result of the target pathology image by the pathology large model, and determine a pancreatic cancer region existing in the target pathology image based on the analysis result;

[0177] a marking unit, configured to delineate and mark the pancreatic cancer areas, and determine the regional characteristics and distribution characteristics of each pancreatic cancer area based on the delineation and marking results;

[0178] The result output unit is used to generate a pathology image cancer area recognition report based on regional features and distribution features, bind the pathology image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathology image, and output the binding result.

[0179] In this embodiment, the demarcation mark refers to marking the shape of the pancreatic cancer area and its respective conditions in the entire area, thereby determining the regional characteristics and distribution characteristics.

[0180] The working principle and beneficial effects of the above technical solution are: by circling and marking the pancreatic cancer area, the regional characteristics and distribution characteristics of the pancreatic cancer area are effectively realized, thereby accurately generating a pathology image cancer area recognition report; by binding and outputting the pathology image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathology image, it is beneficial to ensure the convenience and interpretability of the report reading.

[0181] In one embodiment, a method for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. Figure 3 Shown, including:

[0182] Step 1: Optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathology model;

[0183] Step 2: Capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features;

[0184] Step 3: Output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

[0185] The working principle and beneficial effects of the above technical solution are: through deep learning technology, the tumor area of ​​pancreatic cancer is automatically identified and displayed in the form of a heat map, thereby assisting pathological diagnosis, quantitatively analyzing pathological characteristics, and improving the efficiency and accuracy of pancreatic cancer pathological diagnosis.

[0186] In one embodiment, a method for automatically identifying cancerous areas in pathological images based on a large pathological model is provided. In step 1, the architecture of the basic model of pathological images is optimized based on a multi-head self-attention mechanism and multi-scale feature fusion of deep learning, including:

[0187] Obtain business requirements based on the management terminal, analyze the business requirements, and obtain the business structure and business execution objectives;

[0188] Determine the computing power required for business needs based on the business structure and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the computing power to obtain a set of available basic models;

[0189] The basic configuration of each basic model in the available basic model set is extracted, and the performance parameters of each basic model are obtained based on the basic configuration. The basic model with the best performance is selected as the pathological image basic model based on its own performance parameters.

[0190] The working principle and beneficial effects of the above technical solution are: obtaining and analyzing business needs through the management terminal, clarifying the business composition and execution purpose, determining the computing amount based on the analyzed business composition and execution purpose, and finding the available basic model set by conditional traversal of the basic models in the preset model library; then extracting the basic configuration of each basic model in the set, and based on this, obtaining its own performance parameters, and finally selecting the one with the best performance as the basic model of pathological images; being able to select the most appropriate basic model according to specific business needs, thereby improving the applicability and accuracy of the model; optimizing resource utilization and improving work efficiency by considering the computing amount and model performance; and helping to build a more accurate and efficient large pathology model.

[0191] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A system for automatically identifying cancerous areas in pathological images based on a large pathological model, characterized by: include: The pathology model building module is used to optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a pathology model. The pathology model building module includes: Model optimization preparation unit for: When identifying cancerous areas in pathological images based on a preset business system, the focus dimension of the input image is obtained, and at the same time, the input features of each focus dimension are obtained, and the input features of each focus dimension are converted into a corresponding feature vector sequence; Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrix of each attention dimension to obtain a multi-head self-attention mechanism; At the same time, based on the preset business system, the feature extraction scale for identifying cancerous areas in pathological images is determined, and the correlation and dependency between different feature extraction scales are determined; based on the hierarchical correlation relationship, the multi-scale feature fusion strategy between different feature extraction scales is determined; Architecture Optimization Unit, used to: Obtain the structural parameters of the basic model of pathological images, and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the basic model of pathological images based on the structural parameters; Based on the insertion position, the multi-head self-attention mechanism and multi-scale feature fusion strategy are added to the basic model of pathological images to complete the architecture optimization of the basic model of pathological images and obtain the large pathological model; The parsing module is used to capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features; The recognition result output module is used to output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

2. The automatic cancer area recognition system in pathology images based on a large pathology model according to claim 1 is characterized in that: Pathology model building modules, including: A demand analysis unit, used to obtain the business requirements of pancreatic cancer based on the management terminal; A model library acquisition unit, the model library acquisition unit is used to acquire a model library, the model library includes a plurality of basic models; The basic model selection unit is used to screen out models that meet business needs from a preset model library based on business needs.

3. The automatic cancer area recognition system in pathology images based on a large pathology model according to claim 2 is characterized in that: The automatic cancer area recognition system for pathological images based on the pathological large model further includes a training unit, which includes: The result acquisition subunit is used to obtain the architecture optimization results of the obtained pathology image basic model and select the loss function of the architecture optimization results from the preset function library based on the business requirements of the pathology large model; The model testing subunit is used to obtain test data, input the test data into the architecture optimization results of the pathology image basic model for analysis and recognition, and determine the loss value of the analysis and recognition results based on the loss function; The model optimization and deployment subunit is used to re-optimize the architecture optimization results of the pathology image basic model when the loss value is greater than the preset threshold, and then retrain and evaluate it, and finally deploy the obtained pathology model in the application environment.

4. The automatic cancer area recognition system for pathological images based on a large pathological model according to claim 1 is characterized in that: The parsing module includes: Image acquisition unit, used for: The collection includes pathological levels from pancreatic cancer cell morphology to tissue structure; Pathological image sample sets of pancreatic cancer at different stages and subtypes are collected according to the pathological level, and the pathological sample image sets are processed to obtain standard pathological image sample sets; An image sample set division unit is used to divide the standard pathology image sample set according to the pathology level to obtain a sub-standard pathology image sample set corresponding to each pathology level; a labeling unit, configured to label a substandard pathology image sample set corresponding to each pathology layer; The learning unit is used to learn the annotation results based on the pathology model to obtain the pathology features corresponding to each pathology layer. At the same time, the pathology features corresponding to each pathology layer are integrated to obtain multi-level pathology features. The analysis unit is used to input the currently acquired target pathology image into the pathology big model, and analyze the target pathology image based on the multi-level pathology features according to the pathology big model.

5. The automatic cancer area recognition system in pathology images based on a large pathology model according to claim 4 is characterized in that: The image acquisition unit includes: An image processing subunit is used to read the collected pathological image sample set and determine the image pixel distribution state of each pathological sample image; A feature acquisition subunit is used to acquire features of pixels in irrelevant areas and features of pixels in key areas; A feature matching subunit, configured to perform a first matching of the image pixel distribution state of each pathological sample image with the pixel features of the irrelevant area and a second matching of the pixel features of the key area; a positioning subunit, configured to perform a first positioning in each pathological sample image according to the first matching result to obtain an irrelevant area of ​​each pathological sample image, and perform a second positioning in each pathological sample image according to the second matching result to obtain a key area of ​​each pathological sample image; Regional Analysis Subunit, used to: Splicing the irrelevant area and the key area of ​​each pathological sample image, and judging whether the irrelevant area and the key area of ​​each pathological sample image overlap according to the splicing result; If the irrelevant area of ​​the pathological sample image overlaps with the key area, the contour line of the key area is used as the cropping boundary line of the pathological sample image; If the irrelevant area of ​​the pathological sample image does not overlap with the key area, the contour line of the irrelevant area is used as the cropping boundary line of the pathological sample image; an image processing subunit, configured to determine a clipping boundary line for each pathological sample image according to the judgment result, and clip each pathological sample image according to the clipping boundary line of each pathological sample image to obtain a key area of ​​each pathological sample image; The synthesis subunit is used to synthesize the key areas of each pathological sample image to obtain a standard pathological image sample set.

6. The automatic cancer area recognition system in pathology images based on a large pathology model according to claim 4 is characterized in that: The marking unit includes: The color labeling subunit is used to read the substandard pathology image sample set corresponding to each pathology layer, determine the probability of each pixel in the substandard pathology sample image set being cancer, and convert the probability into different colors for labeling; The display subunit is used to display the substandard pathology sample image set in the form of a heat map based on the annotation results.

7. The automatic cancer area recognition system in pathology images based on a large pathology model according to claim 1 is characterized in that: The recognition result output module includes: a result determination unit, configured to obtain an analysis result of the target pathology image by the pathology large model, and determine a pancreatic cancer region existing in the target pathology image based on the analysis result; a marking unit, configured to delineate and mark the pancreatic cancer areas, and determine the regional characteristics and distribution characteristics of each pancreatic cancer area based on the delineation and marking results; The result output unit is used to generate a pathology image cancer area recognition report based on regional features and distribution features, bind the pathology image cancer area recognition report with the target recognition image of the pancreatic cancer area in the target pathology image, and output the binding result.

8. A method for automatically identifying cancerous areas in pathological images based on a large pathological model, characterized in that: include: Step 1: Optimize the architecture of the pathology image basic model based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathology model; The architecture of the pathology image basic model is optimized based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning to obtain a large pathology model, including: Model optimization preparation unit for: When identifying cancerous areas in pathological images based on a preset business system, the focus dimension of the input image is obtained, and at the same time, the input features of each focus dimension are obtained, and the input features of each focus dimension are converted into a corresponding feature vector sequence; Generate a self-attention matrix based on the feature vector sequence, and synthesize the self-attention matrix of each attention dimension to obtain a multi-head self-attention mechanism; At the same time, based on the preset business system, the feature extraction scale for identifying cancerous areas in pathological images is determined, and the correlation and dependency between different feature extraction scales are determined; based on the hierarchical correlation relationship, the multi-scale feature fusion strategy between different feature extraction scales is determined; Architecture Optimization Unit, used to: Obtain the structural parameters of the basic model of pathological images, and determine the insertion positions of the multi-head self-attention mechanism and the multi-scale feature fusion strategy in the basic model of pathological images based on the structural parameters; Based on the insertion position, the multi-head self-attention mechanism and multi-scale feature fusion strategy are added to the basic model of pathological images to complete the architecture optimization of the basic model of pathological images and obtain the large pathological model; Step 2: Capture multi-level pathological features from pancreatic cancer cell morphology to tissue structure based on the pathological model, and parse the input target pathological image based on the multi-level pathological features; Step 3: Output the recognition result of the pancreatic cancer area in the target pathology image based on the analysis result according to the pathology model.

9. The method for automatically identifying cancerous areas in pathological images based on a large pathological model according to claim 8, characterized in that: In step 1, the architecture of the pathology image basic model is optimized based on the multi-head self-attention mechanism and multi-scale feature fusion of deep learning, including: Obtain business requirements based on management terminals; Acquire a model library, wherein the model library includes a plurality of basic models; Filter out models that meet business needs from the preset model library based on business needs; Determine the computing power required for business needs based on the business structure and business execution purpose, and perform conditional traversal on each basic model in the preset model library based on the computing power to obtain a set of available basic models; The basic configuration of each basic model in the available basic model set is extracted, and the performance parameters of each basic model are obtained based on the basic configuration. The basic model with the best performance is selected as the pathological image basic model based on its own performance parameters.

Images