A method for constructing a target detection model based on a large pathological model

Abstract

The present invention provides a method for constructing a target detection model based on a pathology large model, including: reading the pathology large model, determining multiple feature output dimensions of the pathology large model; obtaining the pathology sample image corresponding to each feature output dimension, and learning the deep neural network based on the pathology sample image to determine the target detection model corresponding to each feature output dimension; matching and grafting the pathology large model with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology large model. The target detection model of the pathology large model is accurately and effectively constructed, which is convenient for processing different types of pathology slice images through the pathology large model, enriching the detection tasks, and also ensuring the accuracy and reliability of the pathology large model in performing different detection tasks.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for constructing a target detection model based on a large pathology model. Background Art

[0002] At present, emerging technologies such as deep neural network (DNN) have been widely developed and applied in medical pathology slice recognition with unprecedented performance, including the exploration of the application of large models in the direction of pathology;

[0003] However, due to its business specificity, the pathology model has a lot of downstream tasks, one of which is target detection. Therefore, it is necessary to graft the target detection head with the features of the large model. However, the pathology large model in the existing technology does not have a target detection head and cannot realize the target detection function. In addition, the downstream tasks of the large model are relatively simple and lack richness.

[0004] Therefore, in order to overcome the above technical problems, the present invention provides a method for constructing a target detection model based on a large pathology model. Summary of the invention

[0005] The present invention provides a method for constructing a target detection model based on a pathology big model, which is used to analyze and process the pathology sample image according to the multiple feature output dimensions by determining the multiple feature output dimensions of the pathology big model, and to accurately and effectively construct the target detection model corresponding to different feature output dimensions. Secondly, by matching the target detection model with the dimension of the pathology big model, the pathology big model is accurately and effectively connected with different target detection models, thereby realizing accurate and effective construction of the target detection model of the pathology big model, facilitating the processing of different types of pathology slice images by the pathology big model, enriching the detection tasks, and also ensuring the accuracy and reliability of the pathology big model in performing different detection tasks. .

[0006] The present invention provides a method for constructing a target detection model based on a large pathology model, comprising:

[0007] Step 1: Read the pathology model and determine multiple feature output dimensions of the pathology model;

[0008] Step 2: Obtain the pathological sample image corresponding to each feature output dimension, and learn the deep neural network based on the pathological sample image to determine the target detection model corresponding to each feature output dimension;

[0009] Step 3: Match and graft the pathology big model with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology big model.

[0010] Preferably, a method for constructing a target detection model based on a pathology large model, in step 1, reading the pathology large model and determining multiple feature output dimensions of the pathology large model, including:

[0011] Obtain the network structure of the pathology model and determine the module parameter settings in the pathology model;

[0012] Based on the module parameter settings of the pathology model, the feature output dimension corresponding to each module in the pathology model is determined.

[0013] Preferably, a method for constructing a target detection model based on a large pathology model, in step 2, obtaining a pathology sample image corresponding to each feature output dimension, and learning a deep neural network based on the pathology sample image to determine the target detection model corresponding to each feature output dimension, including:

[0014] Read the feature output dimension to determine the dimension information corresponding to the feature output dimension;

[0015] Determine the lesion key points of the pathological image to be extracted and the lesion contour points of the image to be extracted based on the dimensional information, and at the same time, determine the relative position relationship between the lesion key points of the pathological image to be extracted and the lesion contour points of the pathological image to be extracted based on the dimensional information;

[0016] Associating the lesion key points of the pathological image to be extracted with the lesion contour points of the pathological image to be extracted based on the relative position relationship, and determining the image attribute identifier of the pathological image to be extracted based on the association result;

[0017] Input the image attribute identifier into the preset sample library for matching, and obtain the target image in the preset sample library that matches the image attribute identifier, wherein the target image is the pathological sample image corresponding to the feature output dimension;

[0018] The deep neural network is learned based on the pathological sample images to obtain the target detection model corresponding to the feature output samples.

[0019] Preferably, a method for constructing a target detection model based on a large pathology model, learning a deep network model based on a pathology sample image to obtain a target detection model corresponding to a feature output sample, includes:

[0020] Preprocessing the pathological sample image to obtain a target pathological sample image;

[0021] Identify the target pathological sample image, determine the visual features of the target pathological sample image, and at the same time, perform feature mapping on the visual features of the target pathological sample image based on the visual mapping matrix, and output a first feature corresponding to the target pathological image;

[0022] Obtaining a pathological description of a target pathological sample image, and obtaining key semantic fields of the pathological description, and performing semantic feature mapping on the key semantic fields based on a semantic mapping matrix to obtain a second feature corresponding to the target pathological image;

[0023] Annotating key semantic fields of the pathology description in the target pathology sample image based on visual features of the target pathology sample image, and determining a target constraint condition between the first feature and the second feature based on the annotation result;

[0024] Associating the first feature with the second feature according to a target constraint condition between the first feature and the second feature;

[0025] The deep neural network is learned and updated according to the first feature, the second feature and the association result, and a target detection model corresponding to the feature output sample is output based on the learning update result.

[0026] Preferably, a method for constructing a target detection model based on a large pathology model preprocesses a pathology sample image to obtain a target pathology sample image, including:

[0027] The pathological sample image is read to determine the pixel distribution state of the pathological sample image, and at the same time, the pathological sample image is segmented based on the pixel distribution state of the pathological sample image to obtain a number of patch-level pathological sample images;

[0028] Obtaining position data of each pixel in each patch-level pathology sample image, and determining the position distance between each pixel in the patch-level pathology sample image according to the position data of each pixel;

[0029] Obtaining the minimum position distance and the maximum position distance among the position distances between the pixels, and calculating the average value of the maximum position distance and the minimum position distance, and adjusting the spacing between the pixels of the corresponding patch-level pathology sample image according to the average value of the maximum position distance and the minimum position distance, and outputting the target patch-level pathology sample image corresponding to each patch-level pathology sample image based on the adjustment result;

[0030] The contour information of each target patch-level pathology sample image is obtained, and the contour information of each target patch-level pathology sample image is matched, and each target patch-level pathology sample image is spliced ​​based on the matching result to output the target pathology sample image.

[0031] Preferably, a method for constructing a target detection model based on a large pathological model, learning and updating a deep neural network according to a first feature, a second feature, and a correlation result, and outputting a target detection model corresponding to a feature output sample based on the learning and updating result, comprises:

[0032] Based on the first feature, the second feature and the association result, the node state of each layer in the deep neural network is updated upward and downward to obtain a weight update result;

[0033] Based on the weight update results, the learning update of the deep neural network is completed to obtain the target detection model.

[0034] Preferably, a method for constructing a target detection model based on a large pathology model, in step 2, when learning a deep neural network based on a pathology sample image, includes:

[0035] The loss functions of deep neural networks include: classification loss function and regression loss function;

[0036] Among them, the classification loss function calculates the probability of each category based on the Sigmoid function and uses VFL Loss or BCE Loss to calculate the global category loss;

[0037] The regression loss function calculates the correlation between the predicted box and the target box based on the CIoU_Loss function.

[0038] Preferably, a method for constructing a target detection model based on a pathology large model, in step 3, matching and grafting the pathology large model with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology large model, including:

[0039] Read the characteristic output dimension of the pathology model, determine the dimension label corresponding to the characteristic output dimension of the pathology model, and obtain the dimension representation corresponding to the dimension label;

[0040] Read the target detection model, determine the detection type of the target detection model, and convert the detection type into a label according to the dimensional representation to obtain the detection label of the target detection model;

[0041] Match the dimension labels with the detection labels, and build a grafting module between the pathology model and the target detection model based on the matching results;

[0042] The pathology model is grafted with the corresponding target detection model based on the grafting module.

[0043] Preferably, a method for constructing a target detection model based on a pathology large model, when grafting the pathology large model with the corresponding target detection model based on a grafting module, includes:

[0044] The grafting module is integrated with the large pathology model and trained. During the training process, the parameters of the large pathology model are kept unchanged. At the same time, the parameters of the grafting module are continuously adjusted until the target detection model converges. Then, the grafting module completes the grafting of the large pathology model with the corresponding target detection model.

[0045] Preferably, a method for constructing a target detection model based on a large pathology model, the grafting module includes: a network layer, which is, in order from front to back: a convolutional network layer, an activation function layer.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. By determining multiple feature output dimensions of the pathology big model, the pathology sample images can be analyzed and processed according to the multiple feature output dimensions, and the target detection models corresponding to different feature output dimensions can be accurately and effectively constructed. Secondly, by matching the target detection model with the dimensions of the pathology big model, the pathology big model can be accurately and effectively connected with different target detection models, thereby realizing accurate and effective construction of the target detection model of the pathology big model, facilitating the processing of different types of pathology slice images through the pathology big model, enriching the detection tasks, and also ensuring the accuracy and reliability of the pathology big model in performing different detection tasks.

[0048] 2. By preprocessing the pathological sample images, the quality of the target pathological sample images can be effectively guaranteed, and then the accuracy of the target pathological sample images can be improved. By performing feature mapping of visual features and semantic feature mapping on the target pathological sample images, the first feature and the second feature of the target pathological image can be accurately determined. By determining the constraints of the first feature and the second feature, the first feature and the second feature can be effectively associated, and then the learning and updating of the deep neural network can be realized according to the association results and the first feature and the second feature, thereby effectively ensuring the accuracy and effectiveness of the target detection model construction.

[0049] 3. By constructing a grafting module, the target detection models of different detection types can be accurately and effectively matched and grafted to the pathology model according to the grafting module, so as to facilitate the effective analysis of different types of pathology slice images through the pathology model, and ensure the accuracy and reliability of performing different detection tasks.

[0050] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0051] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0053] Figure 1 A flowchart of a method for constructing a target detection model based on a large pathology model in an embodiment of the present invention;

[0054] Figure 2 This is a flowchart of step 1 in a method for building a target detection model based on a large pathology model in an embodiment of the present invention;

[0055] Figure 3 This is a flowchart of step 2 in a method for building a target detection model based on a large pathology model in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0057] This embodiment provides a method for constructing a target detection model based on a large pathology model. Figure 1 As shown, including:

[0058] Step 1: Read the pathology model and determine multiple feature output dimensions of the pathology model;

[0059] Step 2: Obtain the pathological sample image corresponding to each feature output dimension, and learn the deep neural network based on the pathological sample image to determine the target detection model corresponding to each feature output dimension;

[0060] Step 3: Match and graft the pathology big model with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology big model.

[0061] In this embodiment, the feature output dimension may be sample information of lesion samples in case sample images, including lesion type, location distribution information of lesion areas, and the like.

[0062] In this embodiment, the pathology large model refers to the Vision Transformer obtained through self-supervised training, specifically ViT-L-16: Model Overview: Advanced image segmentation methods are used to divide the image into blocks of fixed size. Each block is linearly embedded, and after adding position embedding, a vector sequence is formed, which is then fed into the standard Transformer encoder. In order to ensure that the model structure is consistent with the Transformer structure widely used in natural language processing, we use standard methods to introduce learnable "classification tags" into the sequence. Specifically, the standard encoder architecture consists of stacked layer normalization, multi-head self-attention, and a multi-layer perceptron (MLP), and residual connections are used to prevent the gradient vanishing problem.

[0063] In this embodiment, matching and grafting the pathology big model with the target detection model based on the feature output dimension is to associate the pathology big model with the target detection model, so that different types of pathology slice images can be processed by the pathology big model, ensuring the accuracy and reliability of the pathology big model in performing different detection tasks.

[0064] The working principle and beneficial effects of the above technical solution are: by determining multiple feature output dimensions of the pathology big model, the pathology sample image can be analyzed and processed according to the multiple feature output dimensions, and the target detection model corresponding to different feature output dimensions can be accurately and effectively constructed; secondly, by matching the target detection model with the dimension of the pathology big model, the pathology big model can be accurately and effectively connected with different target detection models, thereby realizing accurate and effective construction of the target detection model of the pathology big model, facilitating the processing of different types of pathology slice images through the pathology big model, enriching the detection tasks, and also ensuring the accuracy and reliability of the pathology big model in performing different detection tasks.

[0065] In one embodiment, Figure 2 As shown, in step 1, the pathology model is read to determine multiple feature output dimensions of the pathology model, including:

[0066] Step 101: Obtain the network structure of the pathology model and determine the module parameter settings in the pathology model;

[0067] Step 102: Based on the module parameter settings of the pathology model, determine the feature output dimension corresponding to each module in the pathology model.

[0068] The working principle of the above technical solution is: by determining the network structure of the pathology large model (that is, determining the modules included in the model), the setting of the module parameters in the pathology large model (for example, including but not limited to convolution size, number of convolution kernels, and activation function type, etc.) can be effectively determined. By setting the module parameters of the pathology large model, the feature output dimension corresponding to each module in the pathology large model can be effectively determined (that is, the output dimension reflects the feature graph of different states).

[0069] The beneficial effect of the above technical solution is that by determining the network structure of the pathology large model, the setting of module parameters in the pathology large model can be effectively realized, which is conducive to accurately determining the feature output dimension and providing an effective reference for the construction of the subsequent target detection model.

[0070] In one embodiment, in step 2, a pathological sample image corresponding to each feature output dimension is obtained, and a deep neural network is learned based on the pathological sample image to determine a target detection model corresponding to each feature output dimension, including:

[0071] Step 201: Read the feature output dimension to determine the dimension information corresponding to the feature output dimension;

[0072] Step 202: determining the lesion key points of the pathological image to be extracted and the lesion contour points of the image to be extracted based on the dimensional information, and at the same time, determining the relative position relationship between the lesion key points of the pathological image to be extracted and the lesion contour points of the pathological image to be extracted based on the dimensional information;

[0073] Step 203: associating the lesion key points of the pathological image to be extracted with the lesion contour points of the pathological image to be extracted based on the relative position relationship, and determining the image attribute identifier of the pathological image to be extracted based on the association result;

[0074] Step 204: input the image attribute identifier into a preset sample library for matching, and obtain a target image in the preset sample library that matches the image attribute identifier, wherein the target image is a pathological sample image corresponding to the feature output dimension;

[0075] Step 205: The deep neural network is learned based on the pathological sample image to obtain a target detection model corresponding to the feature output sample.

[0076] The working principle of the above technical solution is: by reading the feature output dimension, the dimensional information of the feature output dimension (that is, the sample information of the lesion sample, including the lesion type, the location distribution information of the lesion area, etc.) can be determined. The dimensional information can effectively determine the lesion key points of the pathological image to be extracted (that is, the core location point where the lesion occurs) and the lesion contour points (that is, the boundary location point of the entire lesion area when the lesion occurs). The dimensional information can effectively determine the relative position relationship between the lesion key points of the pathological image to be extracted and the lesion contour points of the pathological image to be extracted. Through the relative position relationship, the association between the lesion key points and the lesion contour points of the pathological image can be effectively achieved, so that the image attribute identification of the pathological image to be extracted can be determined through the association result (in order to distinguish different image attributes), and the image attribute identification is input into a preset sample library (a sample library set in advance, which stores sample images corresponding to different image attribute identifications). The pathological sample image corresponding to the feature output dimension can be effectively determined, so that the deep neural network is learned based on the pathological sample image, and the acquisition of the target detection model corresponding to the feature output sample is effectively achieved.

[0077] The beneficial effect of the above technical solution is: by acquiring the dimensional information of the feature output dimension and effectively acquiring the attribute identification of the pathological image to be extracted based on the dimensional information analysis, it lays the foundation for acquiring the pathological sample image, so that the target detection model can be effectively acquired based on the learning of the deep neural network based on the pathological sample image, thereby ensuring the accuracy of the target detection model acquisition and providing effective guarantee for the recognition of large pathological models.

[0078] In one embodiment, in step 205, the deep network model is learned based on the pathological sample image to obtain a target detection model corresponding to the feature output sample, including:

[0079] Preprocessing the pathological sample image to obtain a target pathological sample image;

[0080] Identify the target pathological sample image, determine the visual features of the target pathological sample image, and at the same time, perform feature mapping on the visual features of the target pathological sample image based on the visual mapping matrix, and output a first feature corresponding to the target pathological image;

[0081] Obtaining a pathological description of a target pathological sample image, and obtaining key semantic fields of the pathological description, and performing semantic feature mapping on the key semantic fields based on a semantic mapping matrix to obtain a second feature corresponding to the target pathological image;

[0082] Annotating key semantic fields of the pathology description in the target pathology sample image based on visual features of the target pathology sample image, and determining a target constraint condition between the first feature and the second feature based on the annotation result;

[0083] Associating the first feature with the second feature according to a target constraint condition between the first feature and the second feature;

[0084] The deep neural network is learned and updated according to the first feature, the second feature and the association result, and a target detection model corresponding to the feature output sample is output based on the learning update result.

[0085] The working principle of the above technical solution is: by preprocessing the pathological sample image (that is, standardizing the scale of the pathological sample image), it is effectively guaranteed that the target pathological sample image obtained is more standardized and accurate; by identifying the target pathological sample image, the visual features of the target pathological sample image are effectively determined (that is, the abstract features of the image content of the target pathological sample image, including but not limited to: edge features, texture features, angle features, etc.); and by mapping the visual features of the target pathological sample image through the visual mapping matrix, the first feature corresponding to the target pathological image can be effectively determined (that is, the image feature that can be recognized by the deep neural network); by obtaining the pathological description of the target pathological sample image (that is, the pathological condition of the lesion), the key semantic fields of the pathological description (that is, the core fields of the pathological description) are extracted; and by The mapping matrix performs semantic feature mapping on the key semantic fields, thereby acquiring the second feature corresponding to the target pathological image (that is, the semantic feature that can be recognized by the deep neural network), and annotating the corresponding key semantic field in the target pathological sample image through the visual features of the target pathological sample image, so as to obtain the constraint conditions between the first feature and the second feature (for example: a feature of a lesion image corresponds to a language description, when the image feature of the lesion image is determined, the pathological description of the corresponding lesion image can be known according to the constraint conditions), and the target constraint conditions between the first feature and the second feature can effectively associate the first feature with the second feature, and then the deep neural network is learned and updated according to the first feature, the second feature and the association result, and the target detection model corresponding to the feature output sample is output based on the learning update result.

[0086] In the above, the deep neural network is learned and updated according to the first feature, the second feature and the association result, and the target detection model corresponding to the feature output sample is output based on the learning update result, including:

[0087] Based on the first feature, the second feature and the association result, the node state of each layer in the deep neural network is updated upward and downward to obtain a weight update result;

[0088] Based on the weight update results, the learning update of the deep neural network is completed to obtain the target detection model.

[0089] Among them, the upward weight update is to learn the features of pathological sample images based on the deep neural network, and the downward weight update is to recognize pathological images based on the deep neural network.

[0090] The beneficial effects of the above technical solution are: by preprocessing the pathological sample image, the quality of the target pathological sample image can be effectively guaranteed, and then the accuracy of the target pathological sample image can be improved; by performing feature mapping of visual features and semantic feature mapping on the target pathological sample image, the first feature and the second feature of the target pathological image can be accurately determined; by determining the constraints of the first feature and the second feature, the first feature and the second feature can be effectively associated, and then the learning and updating of the deep neural network can be realized according to the association result and the first feature and the second feature, thereby effectively ensuring the accuracy and effectiveness of the target detection model construction.

[0091] In one embodiment, preprocessing the pathological sample image to obtain a target pathological sample image includes:

[0092] The pathological sample image is read to determine the pixel distribution state of the pathological sample image, and at the same time, the pathological sample image is segmented based on the pixel distribution state of the pathological sample image to obtain a number of patch-level pathological sample images;

[0093] Obtaining position data of each pixel in each patch-level pathology sample image, and determining the position distance between each pixel in the patch-level pathology sample image according to the position data of each pixel;

[0094] Obtaining the minimum position distance and the maximum position distance among the position distances between the pixels, and calculating the average value of the maximum position distance and the minimum position distance, and adjusting the spacing between the pixels of the corresponding patch-level pathology sample image according to the average value of the maximum position distance and the minimum position distance, and outputting the target patch-level pathology sample image corresponding to each patch-level pathology sample image based on the adjustment result;

[0095] The contour information of each target patch-level pathology sample image is obtained, and the contour information of each target patch-level pathology sample image is matched, and each target patch-level pathology sample image is spliced ​​based on the matching result to output the target pathology sample image.

[0096] The working principle of the above technical solution is: by analyzing the pathological sample image, the pixel distribution state of different pixels in the pathological sample image (the distribution position of different pixels in the pathological sample image) is determined, and the pathological sample image is divided according to the pixel distribution state to obtain the patch-level pathological sample image (because the whole slice is too large, the training model will overflow the video memory, so it is cut into data blocks, that is, the patch-level pathological sample image is used for training), and then the position data of each pixel in each patch-level pathological sample image (the specific position of each pixel in the patch-level pathological sample image) is extracted, and the position distance between different pixels is calculated according to the position data. Determine and determine the minimum position distance and the maximum position distance among the position distances between each pixel point, and calculate the average value of the minimum position distance and the maximum position distance, so as to adjust the spacing between the pixels of the patch-level pathology sample image according to the minimum position distance, the maximum position distance and the average value, thereby completing the preprocessing of the pathology sample image, and finally, by extracting the contour information (specific morphological information presented by the target patch-level pathology sample image) of each target patch-level pathology sample image (pathology sample image after adjusting the pixel spacing), each target patch-level pathology sample image is spliced ​​according to the contour information, and the target pathology sample image is output.

[0097] The beneficial effects of the above technical solution are: by preprocessing the pathological sample images, it is convenient to learn the deep neural network, which provides a reliable guarantee for building target detection models corresponding to different feature output dimensions. At the same time, it provides convenience and guarantee for improving the accuracy and reliability of the large pathological model in performing detection tasks.

[0098] In one embodiment, in step 2, learning a deep neural network based on the pathological sample image includes:

[0099] The loss functions of deep neural networks include: classification loss function and regression loss function;

[0100] Among them, the classification loss function calculates the probability of each category based on the Sigmoid function and uses VFL Loss or BCE Loss to calculate the global category loss;

[0101] The regression loss function calculates the correlation between the predicted box and the target box based on the CIoU_Loss function.

[0102] In this embodiment, category loss refers to the loss function when the model predicts a category in a classification task, which is used to measure the degree of difference between the model's prediction results for each category and the true label. That is, when the deep neural network is learned through pathological sample images, it is used to measure the degree of difference in the learning of the deep neural network for different types of pathological sample images.

[0103] In this embodiment, the prediction box is set in advance.

[0104] In one embodiment, in step 3, the pathology big model is matched and grafted with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology big model, including:

[0105] Read the characteristic output dimension of the pathology model, determine the dimension label corresponding to the characteristic output dimension of the pathology model, and obtain the dimension representation corresponding to the dimension label;

[0106] Read the target detection model, determine the detection type of the target detection model, and convert the detection type into a label according to the dimensional representation to obtain the detection label of the target detection model;

[0107] Match the dimension labels with the detection labels, and build a grafting module between the pathology model and the target detection model based on the matching results;

[0108] The pathology model is grafted with the corresponding target detection model based on the grafting module.

[0109] The working principle of the above technical solution is: by reading the feature output dimensions of the pathology big model, determining the dimension label of each feature output dimension (a marking symbol that can characterize the identity of different feature output dimensions) and the corresponding dimension representation (the business type and the purpose that each feature output dimension can achieve), at the same time, reading the target detection model, locking the detection type of different target detection models, and secondly, converting the labels of the detection types of different target detection models through the dimension representation, so as to obtain the detection labels of different target detection models (a marking symbol that can distinguish the detection types of different target detection models), and finally, matching the dimension label of the pathology big model with the detection label of the target detection model, and constructing a grafting model connecting the pathology big model and the target detection model according to the matching results, and grafting the pathology big model with the corresponding target detection model through the constructed grafting module.

[0110] The beneficial effect of the above technical solution is: by constructing a grafting module, the target detection models of different detection types can be accurately and effectively matched and grafted to the pathology large model according to the grafting module, thereby facilitating the effective analysis of different types of pathology slice images through the pathology large model, thereby ensuring the accuracy and reliability of performing different detection tasks.

[0111] In one embodiment, when the pathology large model is grafted with the corresponding target detection model based on the grafting module, it includes:

[0112] The grafting module is integrated with the large pathology model and trained. During the training process, the parameters of the large pathology model are kept unchanged. At the same time, the parameters of the grafting module are continuously adjusted until the target detection model converges. Then, the grafting module completes the grafting of the large pathology model with the corresponding target detection model.

[0113] In this embodiment, the grafting module can ensure that cross-domain data is not leaked, thereby improving data security.

[0114] The grafting module includes: network layers, which are, in order from front to back: convolutional network layer, activation function layer.

[0115] The working principle of the above technical solution is: by fusing the grafting module with the pathology large model for training, and setting the parameters of the pathology large model as quantitative parameters during the training process, and continuously adjusting the working parameters of the grafting module, the grafting of the pathology large model and the target detection model is completed through the grafting module.

[0116] The beneficial effect of the above technical solution is: by grafting the pathology large model with the corresponding target detection model, it is convenient to effectively integrate the two, ensuring the diversity of detection tasks of the final target detection model, and at the same time, improving the accuracy and reliability of the pathology large model in performing different detection tasks.

[0117] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for constructing a target detection model based on a large pathological model, characterized in that: include: Step 1: Read the pathology model and determine multiple feature output dimensions of the pathology model; Step 2: Obtain the pathological sample image corresponding to each feature output dimension, and learn the deep neural network based on the pathological sample image to determine the target detection model corresponding to each feature output dimension; Step 3: Match and graft the pathology big model with the target detection model based on the feature output dimension to complete the construction of the target detection model based on the pathology big model; Among them, in step 2, the pathological sample image corresponding to each feature output dimension is obtained, and the deep neural network is learned based on the pathological sample image to determine the target detection model corresponding to each feature output dimension, including: Read the feature output dimension to determine the dimension information corresponding to the feature output dimension; Determine the lesion key points of the pathological image to be extracted and the lesion contour points of the image to be extracted based on the dimensional information, and at the same time, determine the relative position relationship between the lesion key points of the pathological image to be extracted and the lesion contour points of the pathological image to be extracted based on the dimensional information; Associating the lesion key points of the pathological image to be extracted with the lesion contour points of the pathological image to be extracted based on the relative position relationship, and determining the image attribute identifier of the pathological image to be extracted based on the association result; Input the image attribute identifier into the preset sample library for matching, and obtain the target image in the preset sample library that matches the image attribute identifier, wherein the target image is the pathological sample image corresponding to the feature output dimension; Based on the pathological sample images, the deep neural network is learned to obtain the target detection model corresponding to the feature output samples; Among them, in step 3, the pathology big model and the target detection model are matched and grafted based on the feature output dimension to complete the construction of the target detection model based on the pathology big model, including: Read the characteristic output dimension of the pathology model, determine the dimension label corresponding to the characteristic output dimension of the pathology model, and obtain the dimension representation corresponding to the dimension label; Read the target detection model, determine the detection type of the target detection model, and convert the detection type into a label according to the dimensional representation to obtain the detection label of the target detection model; Match the dimension labels with the detection labels, and build a grafting module between the pathology model and the target detection model based on the matching results; The pathology model is grafted with the corresponding target detection model based on the grafting module.

2. The method for constructing a target detection model based on a pathological large model according to claim 1, characterized in that: In step 1, the pathology model is read to determine multiple feature output dimensions of the pathology model, including: Obtain the network structure of the pathology model and determine the module parameter settings in the pathology model; Based on the module parameter settings of the pathology model, the feature output dimension corresponding to each module in the pathology model is determined.

3. The method for constructing a target detection model based on a pathology large model according to claim 1, characterized in that: Based on the pathological sample images, the deep network model is learned to obtain the target detection model corresponding to the feature output sample, including: Preprocessing the pathological sample image to obtain a target pathological sample image; Identify the target pathological sample image, determine the visual features of the target pathological sample image, and at the same time, perform feature mapping on the visual features of the target pathological sample image based on the visual mapping matrix, and output a first feature corresponding to the target pathological image; Obtaining a pathological description of a target pathological sample image, and obtaining key semantic fields of the pathological description, and performing semantic feature mapping on the key semantic fields based on a semantic mapping matrix to obtain a second feature corresponding to the target pathological image; Annotating key semantic fields of the pathology description in the target pathology sample image based on visual features of the target pathology sample image, and determining a target constraint condition between the first feature and the second feature based on the annotation result; Associating the first feature with the second feature according to a target constraint condition between the first feature and the second feature; The deep neural network is learned and updated according to the first feature, the second feature and the association result, and a target detection model corresponding to the feature output sample is output based on the learning update result.

4. The method for constructing a target detection model based on a pathology large model according to claim 3, characterized in that: Preprocess the pathological sample image to obtain the target pathological sample image, including: The pathological sample image is read to determine the pixel distribution state of the pathological sample image, and at the same time, the pathological sample image is segmented based on the pixel distribution state of the pathological sample image to obtain a number of patch-level pathological sample images; Obtaining position data of each pixel in each patch-level pathology sample image, and determining the position distance between each pixel in the patch-level pathology sample image according to the position data of each pixel; Obtaining the minimum position distance and the maximum position distance among the position distances between the pixels, and calculating the average value of the maximum position distance and the minimum position distance, and adjusting the spacing between the pixels of the corresponding patch-level pathology sample image according to the average value of the maximum position distance and the minimum position distance, and outputting the target patch-level pathology sample image corresponding to each patch-level pathology sample image based on the adjustment result; The contour information of each target patch-level pathology sample image is obtained, and the contour information of each target patch-level pathology sample image is matched, and each target patch-level pathology sample image is spliced ​​based on the matching result to output the target pathology sample image.

5. The method for constructing a target detection model based on a pathology large model according to claim 3, characterized in that: The deep neural network is learned and updated according to the first feature, the second feature and the association result, and a target detection model corresponding to the feature output sample is output based on the learning update result, including: Based on the first feature, the second feature and the association result, the node state of each layer in the deep neural network is updated upward and downward to obtain a weight update result; Based on the weight update results, the learning update of the deep neural network is completed to obtain the target detection model.

6. The method for constructing a target detection model based on a pathology large model according to claim 1, characterized in that: In step 2, when learning the deep neural network based on the pathological sample image, it includes: The loss functions of deep neural networks include: classification loss function and regression loss function; Among them, the classification loss function calculates the probability of each category based on the Sigmoid function and uses VFL Loss or BCELoss to calculate the global category loss; The regression loss function calculates the correlation between the predicted box and the target box based on the CIoU_Loss function.

7. The method for constructing a target detection model based on a pathology large model according to claim 1, characterized in that: When the pathology large model is grafted with the corresponding target detection model based on the grafting module, it includes: The grafting module is integrated with the large pathology model and trained. During the training process, the parameters of the large pathology model are kept unchanged. At the same time, the parameters of the grafting module are continuously adjusted until the target detection model converges. Then, the grafting module completes the grafting of the large pathology model with the corresponding target detection model.

8. The method for constructing a target detection model based on a pathology large model according to claim 1, characterized in that: The grafting module includes: network layers, which are, in order from front to back: convolutional network layer, activation function layer.

Images