Method and system for constructing a clinical application-level pathological large model based on weakly supervised learning

Abstract

The application provides a clinical application level pathological large model construction method and system based on weak supervision learning, comprising: respectively performing weak supervision training on each historical clinical pathology image, adjusting upstream learning parameters of a large model according to a training result and constructing an upstream task of the large model, respectively performing segmentation decoding on each weak supervision training result, determining downstream learning parameters of the large model according to a segmentation result and constructing a downstream task of the large model, acquiring upstream fine features and downstream fine features, constructing an upstream and downstream collaborative task of the large model, combining the upstream task of the large model and the downstream task of the large model to generate a pathological large model, inputting a current clinical pathology image into the pathological large model to perform pathological information recognition, obtaining a plurality of pathological recognition labels corresponding to a patient, and generating a pathological report of the patient, so that the computational complexity in processing a high-resolution image is reduced, the collaborative mechanism between different tasks in the large model is optimized, and the pathological recognition technology can be widely applied.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model generation, in particular to a clinical application level pathological large model construction method and system based on weakly supervised learning. BACKGROUND

[0002] With the rapid development of medical technology, especially under the promotion of artificial intelligence, big data, cloud computing and other technologies, the medical field has gradually entered the era of data-driven intelligence. Pathology, as a core field of disease diagnosis, plays a crucial role in the diagnosis of major diseases such as cancer and tumors. The accuracy of pathology directly affects the treatment plan and prognosis of the disease. However, the traditional pathological diagnosis method highly depends on the experience and knowledge accumulation of pathologists. In many areas, the problem of uneven distribution of medical resources and shortage of pathologist talents is more prominent, which limits the pathological diagnosis ability of primary hospitals and makes it difficult to guarantee the diagnosis quality and efficiency.

[0003] At the same time, with the change of disease spectrum and the diversification of medical needs, traditional pathological diagnosis methods face many challenges, especially in the diagnosis of rare diseases, unknown diseases and complex lesions. The method of artificial pathological diagnosis has been difficult to meet the increasing demand for precision medicine. Moreover, existing WSI diagnosis technology is often plagued by problems such as data annotation difficulty, training data scale and model generalization ability in actual application, limiting its performance in wide application.

[0004] In order to cope with these challenges, intelligent pathology technology has been proposed and gradually developed. By using deep learning, image recognition, natural language processing and other artificial intelligence technologies, intelligent pathology not only accelerates the processing of WSI, but also excavates more potential information in pathological data to assist pathologists in making more accurate diagnoses. This development direction has received widespread attention in medical research institutions and enterprises, and has achieved certain results. However, existing technologies still have problems such as being able to recognize only specific lesion types, lacking wide applicability across disease types and organs, and weak model generalization ability.

[0005] Therefore, the present application provides a clinical application level pathological large model construction method and system based on weakly supervised learning. SUMMARY

[0006] The clinical application level pathological large model construction method and system based on weakly supervised learning of the present application can realize cross-disease and cross-organ lesion recognition, assist doctors in making accurate diagnoses in a short time, reduce the dependence on a large amount of labeled data in traditional technologies, and enhance the accuracy of diagnosis results.

[0007] The present application provides a clinical application level pathological large model construction method based on weakly supervised learning, comprising:

[0008] Step 1: weakly supervised training is performed on each historical clinical pathology image respectively, the upstream learning parameters of the large model are adjusted according to the training results, and the upstream task of the large model is constructed;

[0009] Step 2: each weakly supervised training result is segmented and decoded respectively, the downstream learning parameters of the large model are determined according to the segmentation result, and the downstream task of the large model is constructed;

[0010] Step 3: the upstream fine features and the downstream fine features are obtained, the upstream and downstream collaborative tasks of the large model are constructed, and the pathological large model is generated by combining the upstream task of the large model and the downstream task of the large model;

[0011] Step 4: inputting the current clinical pathology image into the pathological large model to identify the pathological information, obtaining a plurality of pathological identification labels corresponding to the patient, and generating a pathological report of the patient.

[0012] In an implementable manner,

[0013] The step 1 comprises:

[0014] Step 11: each historical clinical pathology image is digitally processed to obtain WSI features of each historical clinical pathology image, to determine a plurality of basic pathological information corresponding to each historical clinical pathology image, to obtain historical diagnosis information corresponding to each historical clinical pathology image respectively, and to determine standard pathological labels corresponding to each historical clinical pathology image;

[0015] Step 12: weakly supervised training is performed on each historical clinical pathology image to obtain inaccurate supervision results, partial supervision results and indirect supervision results corresponding to each historical clinical pathology image, and label matching is performed on the basic pathological information by using the supervision results to obtain a plurality of supervised pathological labels corresponding to the historical clinical pathology image;

[0016] Step 13: comparing the supervised pathological labels corresponding to each historical clinical pathology image with the corresponding standard pathological labels, and determining the identification sensitivity and identification error value corresponding to the weakly supervised training in different label fields in combination with the label field corresponding to each standard pathological label;

[0017] Step 14: adjusting the error label field with an identification sensitivity lower than a standard sensitivity, generating the upstream learning parameters of the large model in combination with the effective identification sensitivity corresponding to the effective error label field with an identification sensitivity not lower than a standard sensitivity, adjusting the training parameters of the weakly supervised training according to the upstream learning parameters, and generating the upstream task of the large model.

[0018] In an implementable mode,

[0019] Further comprising:

[0020] Divide the standard pathology label contained in each of the historical clinical pathology images into cancerous pathology label and non-cancerous pathology label, respectively;

[0021] Screen the key label field containing the cancerous pathology label, and correct the key recognition sensitivity according to the corresponding key recognition error value to obtain the weighted sensitivity of the key label field;

[0022] When the weighted sensitivity is lower than the standard sensitivity, the key label field is regarded as an error label field.

[0023] In an implementable mode,

[0024] The step 2 comprises:

[0025] Step 21: using a pan-cancer recognition decoder to respectively perform preliminary segmentation decoding on the weak supervision training result corresponding to each of the historical clinical pathology images, determining the specific organ corresponding to each of the historical clinical pathology images, and assigning the corresponding tumor typing decoder and lesion analysis decoder to each of the historical clinical pathology images according to the specific organ;

[0026] Step 22: using the tumor typing decoder and lesion analysis decoder to perform organ lesion decoding on the corresponding preliminary segmentation decoding result, constructing a semantic segmentation data set corresponding to each of the historical clinical pathology images, and generating lesion recognition features of corresponding attributes in the semantic segmentation data set according to the task attributes of the upstream task of the large model;

[0027] Step 23: respectively performing enhancement processing on each of the lesion recognition features, determining the region division image frame of each of the historical clinical pathology images, determining the frame pathology information corresponding to each of the region division image frames, and combining the corresponding historical medical records to respectively evaluate the consistency of each of the frame pathology information;

[0028] Step 24: adjusting the downstream learning parameters of the large model according to the consistency evaluation value, adjusting the decoding accuracy of each decoder using the downstream learning parameters, and determining the decoding task generation large model downstream task corresponding to each of the decoders.

[0029] In an implementable mode,

[0030] The step 3 comprises:

[0031] Step 31: adjusting the convolution kernel size of the large kernel convolution according to the large model upstream task and the large model downstream task, establishing the backbone network of the large model, and capturing upstream fine features of the upstream pathological image and downstream fine features of the downstream pathological image by using the backbone network;

[0032] Step 32: docking the upstream fine features and the downstream fine features, determining a plurality of dockable points, fixing the dockable points in the upstream fine features and the downstream fine features, and matching each non-fixed upstream point / non-fixed downstream point with a corresponding upstream cooperative task / downstream cooperative task according to the point relationship between the non-fixed upstream point / non-fixed downstream point and the dockable point;

[0033] Step 33: cooperatively simulating the upstream cooperative task and the downstream cooperative task according to the fixed position of the dockable point, generating the upstream and downstream cooperative tasks of the large model, docking the large model upstream task and the large model downstream task by using the upstream and downstream cooperative tasks, and generating a pathological large model.

[0034] In an implementable manner,

[0035] Capturing the upstream fine features of the upstream pathological image and the downstream fine features of the downstream pathological image by using the backbone network comprises:

[0036] Respectively performing upstream weak supervision simulation on each of the historical clinical pathological images by using the large model upstream task to obtain an upstream pathological image corresponding to each of the historical clinical pathological images;

[0037] Respectively performing downstream segmentation simulation on each of the upstream pathological images by using the large model downstream task to obtain a downstream pathological image corresponding to each of the historical clinical pathological images;

[0038] Setting the convolution kernel of the backbone network to 4*4 and the stride to 4, respectively performing first downsampling on the upstream pathological image and the downstream pathological image to obtain corresponding upstream semantic features and downstream semantic features, and constructing corresponding upstream semantic pathological images and downstream semantic pathological images;

[0039] Setting the convolution kernel of the backbone network to 2*2 and the stride to 2, respectively performing iterative and cyclic downsampling on the upstream semantic pathological image and the downstream semantic pathological image until upstream high-dimensional pathological images and downstream high-dimensional pathological images are obtained;

[0040] According to the upstream high-dimensional feature corresponding to the upstream high-dimensional pathological image and the corresponding upstream semantic feature, an upstream fine feature of the upstream pathological image is constructed, and according to the downstream high-dimensional feature corresponding to the downstream high-dimensional pathological image and the corresponding downstream semantic feature, a downstream fine feature of the downstream pathological image is constructed.

[0041] In an implementable manner,

[0042] The step 4 comprises:

[0043] Step 41: inputting the current clinical image into the pathological large model to perform the large model upstream task and the large model downstream task, so as to obtain a plurality of pathological identification labels of the current clinical pathological image;

[0044] Step 42: respectively searching for the corresponding pathological identification annotation of the current clinical pathological image of each pathological identification label corresponding to the label meaning;

[0045] Step 43: updating the pathological identification annotation according to the additional annotation issued by the doctor, generating and displaying the pathological report of the patient.

[0046] In an implementable manner,

[0047] Further comprising:

[0048] According to the additional annotation, the model identification defect of the pathological large model is derived;

[0049] In the pathological report, the model unidentified content corresponding to the model identification defect is searched, and the numerical information corresponding to the model unidentified content is converted into a model update parameter;

[0050] The model update parameter is used to adjust the model parameter of the pathological large model.

[0051] The present application provides a clinical application level pathological large model construction system based on weak supervision learning, comprising:

[0052] An upstream training module is used for weakly supervised training of each historical clinical pathological image, adjusting the upstream learning parameter of the large model according to the training result, and constructing the upstream task of the large model;

[0053] A downstream training module is used for segmenting and decoding each weakly supervised training result, determining the segmentation result, adjusting the downstream learning parameter of the large model, and constructing the downstream task of the large model;

[0054] A comprehensive training module is used for obtaining the upstream fine feature and the downstream fine feature, constructing the upstream and downstream collaborative task of the large model, combining the upstream task of the large model and the downstream task of the large model to generate a pathological large model;

[0055] An identification execution module is configured to input a current clinical pathology image into the pathology large model to identify pathology information, obtain a plurality of pathology identification labels corresponding to a patient, and generate a pathology report of the patient.

[0056] In an implementable manner,

[0057] The upstream training module comprises:

[0058] A preliminary processing unit is configured to perform digital processing on each of the historical clinical pathology images to obtain a WSI feature of each of the historical clinical pathology images, determine a plurality of basic pathology information corresponding to each of the historical clinical pathology images, acquire historical diagnosis information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images.

[0059] A supervised training unit is configured to perform weak supervision training on each of the historical clinical pathology images to obtain an inaccurate supervision result, a partial supervision result, and an indirect supervision result corresponding to each of the historical clinical pathology images, perform label matching on the basic pathology information by using the supervision results, and obtain a plurality of supervised pathology labels corresponding to each of the historical clinical pathology images.

[0060] An identification analysis unit is configured to compare the supervised pathology labels corresponding to each of the historical clinical pathology images with the corresponding standard pathology labels, and determine an identification sensitivity and an identification error value of the weak supervision training in different label fields in combination with a label field corresponding to each of the standard pathology labels.

[0061] A task generation unit is configured to perform error adjustment on an error label field with an identification sensitivity lower than a standard sensitivity, combine an effective identification sensitivity corresponding to an effective error label field with an identification sensitivity not lower than a standard sensitivity, generate an upstream learning parameter of the large model, adjust a training parameter of the weak supervision training according to the upstream learning parameter, and generate a large model upstream task.

[0062] The implementable beneficial effects of the above technical solutions are: in order to reduce the misleading influence of the wrong label on the diagnosis, and realize efficient collaborative pathological analysis, the model technology can realize the above requirements, first, the upstream part and the downstream part of the large model are trained respectively, which can ensure better fit of the large model with the pathology field, then the collaborative task between the two parts is formulated, which effectively improves the efficiency of the model operation and the accuracy of the output result, significantly improves the learning ability and prediction accuracy in the scene where the annotation data quality is not high, finally, when the pathological large model is actually used, the corresponding report is output according to the demand for reference of patients and doctors, in this way, the original wrong label can be eliminated in the upstream part of the large model, the patient's lesion can be located in the downstream part of the large model, the calculation complexity when processing high-resolution images is reduced, the collaborative mechanism between different tasks in the large model is optimized, and the pathological recognition technology can be widely applied.

[0063] Other features and advantages of the present application will be set forth in the following specification, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims.

[0064] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0065] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and serve to explain the application, and do not constitute a limitation of the application. In the drawings:

[0066] Figure 1 A workflow schematic diagram of the method for constructing a clinical application level pathological large model based on weakly supervised learning in the embodiment of the present application is shown in the figure.

[0067] Figure 2 A composition schematic diagram of the system for constructing a clinical application level pathological large model based on weakly supervised learning in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present application will be described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and do not serve as a limitation of the present application.

[0069] Embodiment 1

[0070] The embodiment provides a method for constructing a clinical application level pathological large model based on weakly supervised learning, as shown in the figure, which comprises: Figure 1

[0071] ​Step 1: weakly supervised training is performed on each historical clinical pathology image respectively, the upstream learning parameters of the large model are adjusted according to the training results, and the upstream task of the large model is constructed;

[0072] Step 2: each weakly supervised training result is respectively segmented and decoded to determine the segmentation result, the downstream learning parameters of the large model are adjusted, and the downstream task of the large model is constructed;

[0073] Step 3: obtain the upstream fine features and the downstream fine features, construct the upstream and downstream collaborative task of the large model, combine the upstream task of the large model and the downstream task of the large model to generate a pathology large model;

[0074] Step 4: input the current clinical pathology image into the pathology large model to identify the pathology information, obtain a plurality of pathology identification labels corresponding to the patient, and generate a pathology report of the patient.

[0075] In this example, the pathology large model is divided into an upstream part and a downstream part, the upstream part improves the feature extraction capability, and the downstream part improves the fine classification of the disease;

[0076] In this example, the upstream task of the large model represents the task that needs to be performed when the upstream part of the large model processes the current clinical image, and the downstream task of the large model represents the task that needs to be performed when the downstream part of the large model processes the current clinical image;

[0077] In this example, the upstream and downstream collaborative task represents the task that needs to be performed when the image processed by the upstream part is transferred to the downstream part for processing;

[0078] In this example, the pathology identification label represents a label used to distinguish different organ diseases;

[0079] In this example, the historical clinical pathology image and the current clinical pathology image can be images that have undergone WSI diagnosis, or can be original images.

[0080] The working principle and beneficial effects of the above technical solutions are as follows: in order to reduce the misleading influence of incorrect labels on diagnosis and realize efficient collaborative pathological analysis, the above requirements can be realized by using model technology. First, the upstream part and the downstream part of the large model are trained respectively, which can ensure better fit between the large model and the pathology field. Then, the collaborative task between the two parts is formulated, which effectively improves the efficiency of model operation and the accuracy of output results, significantly improves the learning ability and prediction accuracy in the scene where the quality of labeled data is not high. Finally, when using the pathological large model in practice, the corresponding report is output according to the demand for reference of patients and doctors. In this way, the original error label can be eliminated in the upstream part of the large model, and the patient's lesion can be located in the downstream part of the large model, reducing the computational complexity when processing high-resolution images, optimizing the collaborative mechanism between different tasks in the large model, and can be widely applied to pathological recognition technology.

[0081] Embodiment 2

[0082] Based on the embodiment 1, the step 1 comprises:

[0083] Step 11: respectively digitizing each of the historical clinical pathological images to obtain the WSI features of each of the historical clinical pathological images, determining a plurality of basic pathological information corresponding to each of the historical clinical pathological images, respectively acquiring the historical diagnosis information corresponding to each of the historical clinical pathological images, and determining the standard pathological label corresponding to each of the historical clinical pathological images;

[0084] Step 12: respectively weakly supervising each of the historical clinical pathological images to obtain the inaccurate supervision result, the partial supervision result and the indirect supervision result corresponding to each of the historical clinical pathological images, and using the supervision result to label match the basic pathological information to obtain a plurality of supervised pathological labels corresponding to the historical clinical pathological images;

[0085] Step 13: comparing the supervised pathological labels corresponding to each of the historical clinical pathological images with the corresponding standard pathological labels, and combining each of the standard pathological labels to determine the recognition sensitivity and recognition error value corresponding to the weak supervision training in different label fields;

[0086] Step 14: adjusting the error label field with a recognition sensitivity lower than the standard sensitivity, combining the effective recognition sensitivity corresponding to the effective error label field with a recognition sensitivity not lower than the standard sensitivity, generating the upstream learning parameters of the large model, adjusting the training parameters of the weak supervision training according to the upstream learning parameters, and generating the upstream task of the large model.

[0087] In this example, digitizing process refers to the process of digitizing the scanned historical clinical pathology images;

[0088] In this example, WSI features refer to the information extracted from the historical clinical pathology images;

[0089] In this example, historical diagnosis information refers to the information retained after the diagnosis of the patient;

[0090] In this example, standard pathology labels refer to labels that can accurately express the patient's disease;

[0091] In this example, inaccurate supervision results refer to the results of supervising only the data-labeled content or any data-labeled content in the historical clinical pathology images, partial supervision results refer to the results of supervising the content with unclear data in the historical clinical pathology images, and indirect supervision results refer to the results of supervising the results derived from the existing data content;

[0092] In this example, supervised pathology labels refer to the pathology labels identified in the historical clinical pathology images;

[0093] In this example, label domain refers to the pathology domain to which a standard pathology label belongs;

[0094] In this example, recognition sensitivity refers to the sensitivity and accuracy of weak supervision training in recognizing a label domain, and recognition error value refers to the maximum error of weak supervision training in recognizing a label domain;

[0095] In this example, the standard sensitivity is 98.9%.

[0096] The working principle and beneficial effects of the above technical solution are: in order to ensure the weak supervision learning ability of the upstream part of the large model, construct an efficient physiological large model, and ensure the recognition accuracy of the current clinical image, first, the historical clinical pathological image is digitally processed to obtain the WSI feature of the historical clinical pathological image, and the basic pathological information corresponding to the device is determined, thereby combining the historical diagnosis information to determine the standard pathological label of each historical clinical pathological image, then the weak supervision learning is used to generate the supervised pathological label of the historical clinical pathological image, the recognition sensitivity and recognition error of the weak supervision training are determined by comparing the standard pathological label and the historical pathological label, and the error adjustment is performed on the error label field with high recognition error value, the upstream learning parameter of the large model is determined, the training parameter of the weak supervision learning is adjusted to construct the upstream task of the large model, and the accuracy of the upstream task can be gradually improved through training. The accuracy is continuously optimized and adjusted by using accurate numerical values, the influence of the error label is gradually reduced by optimizing the discriminative learning method of the model and improving the accurate capture ability of the features, so that the model can pay more attention to the key feature area in the positive case, thereby improving the robustness and generalization ability of feature extraction, and ensuring the stability of the model in clinical application.

[0097] Embodiment 3

[0098] Based on the embodiment 2, the method for constructing a clinical application level pathological large model based on weak supervision learning further comprises:

[0099] The standard pathological label contained in each of the historical clinical pathological images is divided into a cancerous pathological label and a non-cancerous pathological label.

[0100] The key label field containing the cancerous pathological label is screened, the key recognition sensitivity is corrected according to the corresponding key recognition error value, and the weighted sensitivity of the key label field is obtained.

[0101] When the weighted sensitivity is lower than the standard sensitivity, the key label field is regarded as an error label field.

[0102] The working principle and beneficial effects of the above technical solution are: in order to further improve the recognition accuracy of cancer, the key correction is performed on the key label field where the cancerous pathological label is located, and whether it is an error label field is judged, and the corresponding processing method is executed subsequently, thereby improving the capture sensitivity of cancer.

[0103] Embodiment 4

[0104] Based on the embodiment 1, the method for constructing a clinical application level pathological large model based on weak supervision learning, the step 2 comprises:

[0105] Step 21: using a pan-cancer recognition decoder to decode the weakly supervised training result corresponding to each of the historical clinical pathology images respectively to determine the specific organ corresponding to each of the historical clinical pathology images, and assigning a corresponding tumor typing decoder and lesion analysis decoder to each of the historical clinical pathology images according to the specific organ;

[0106] Step 22: using the tumor typing decoder and lesion analysis decoder to decode the preliminary segmentation decoding result corresponding to each of the historical clinical pathology images to construct a semantic segmentation data set corresponding to each of the historical clinical pathology images, and generating lesion recognition features of corresponding attributes in the semantic segmentation data set according to the task attributes of the upstream tasks of the large model;

[0107] Step 23: respectively enhancing each of the lesion recognition features to determine the region division image frame of each of the historical clinical pathology images, determining the frame pathological information corresponding to each of the region division image frames, and combining the corresponding historical medical records to evaluate the degree of coincidence of each of the frame pathological information;

[0108] Step 24: adjusting the downstream learning parameters of the large model according to the degree of coincidence evaluation value, adjusting the decoding accuracy of each decoder using the downstream learning parameters, and determining the decoding task generation large model downstream task corresponding to each of the decoders.

[0109] In this example, the specific organ represents the organ contained in the historical clinical pathology image;

[0110] In this example, organ lesion decoding represents the process of locating the patient's lesion organ and determining the organ disease of the lesion organ;

[0111] In this example, the lesion recognition feature table is used to distinguish the features of different diseases;

[0112] In this example, the enhancement processing label is the process of highlighting the salient points in the lesion recognition feature.

[0113] The working principle and beneficial effects of the above technical solution are as follows: in order to improve the efficiency and quality of pathological recognition, the pathological recognition process is constructed when training the downstream part, first, the pan-cancer recognition decoder is used to determine the organ to be recognized, then the corresponding tumor typing decoder and lesion analysis decoder are assigned, the semantic segmentation dataset of the historical clinical pathological image is obtained by further analyzing the preliminary segmentation decoding result of the pan-cancer recognition decoder, thereby recognizing the lesion recognition features obtained in the upstream task of the large model, further enhancing the features to obtain the region division frame result of the clinical pathological image, and then judging the coincidence evaluation value of the frame pathological information of each image frame according to the historical pathology, using the value to adjust the downstream learning parameters of the large model and the decoding accuracy of the decoder, generating the downstream task of the large model. In this way, not only can the lesion area position and range be intuitively labeled during classification, a high-resolution semantic segmentation heat map is generated to provide clear lesion distribution information for doctors and provide an important basis for further diagnosis and treatment, but also the sensitive decoder can be used to deal with complex and variable pathological tasks, and the adaptability of the diagnosis model in multiple diseases and organs is improved.

[0114] Embodiment 5

[0115] Based on the clinical application level pathological large model construction method based on weakly supervised learning in embodiment 1, the step 3 comprises:

[0116] Step 31: adjusting the convolution kernel size of the large nuclear convolution according to the upstream task of the large model and the downstream task of the large model, establishing the backbone network of the large model, and using the backbone network to capture the upstream fine features of the upstream pathological image and the downstream fine features of the downstream pathological image;

[0117] Step 32: connecting the upstream fine features and the downstream fine features, determining a plurality of connectable points, fixing the connectable points in the upstream fine features and the downstream fine features, and matching each non-fixed upstream point / non-fixed downstream point with a corresponding upstream cooperative task / downstream cooperative task according to the point relationship between the non-fixed upstream point / non-fixed downstream point and the connectable point;

[0118] Step 33: simulating the upstream cooperative task and the downstream cooperative task according to the fixed position of the connectable point, generating the upstream and downstream cooperative tasks of the large model, connecting the upstream task of the large model and the downstream task of the large model using the upstream and downstream cooperative tasks, and generating a pathological large model.

[0119] In this example, the upstream fine feature represents the features presented by the numerical invariable fine values contained in the upstream task of the large model, and the downstream fine feature represents the features presented by the numerical invariable fine values contained in the downstream task of the large model.

[0120] In this example, the butt joint point represents the point at which the upstream fine feature and the downstream fine feature can be directly butt jointed.

[0121] In this example, the non-fixed upstream point represents the remaining point in the upstream fine feature except the butt joint point, and the non-fixed downstream point represents the remaining point in the downstream fine feature except the butt joint point.

[0122] In this example, the collaborative task represents the task of connecting and cooperating the upstream task of the large model and the downstream task of the large model.

[0123] The working principle and beneficial effects of the above technical solution are as follows: the backbone network is set for the large model to ensure that the large model can normally run, the convolution size of the large kernel convolution is first adjusted according to the upstream task of the large model and the downstream task of the large model to construct the backbone network of the large model, then the fine features of the upstream pathological image and the downstream pathological image are captured by using the backbone network, the fine features are further butt jointed, then the corresponding collaborative task is matched for the point which is not butt jointed, the upstream and downstream collaborative tasks of the large model are generated by simulation, finally the upstream task of the large model, the downstream task of the large model and the upstream and downstream collaborative tasks are butt jointed to generate the pathological large model, the high-dimensional features are extracted by the large kernel convolution encoder, and the loss function optimization strategy is combined to solve the misleading problem of the error label to the feature extraction, optimize the discriminative learning method of the model and improve the accurate capture ability of the features, further gradually reduce the influence of the error label, and improve the effectiveness of the large model.

[0124] Embodiment 6

[0125] Based on the embodiment 5, the clinical application level pathological large model construction method based on weakly supervised learning uses the backbone network to capture the upstream fine features of the upstream pathological image and the downstream fine features of the downstream pathological image, which includes:

[0126] The upstream weakly supervised simulation is performed on each of the historical clinical pathological images by using the upstream task of the large model to obtain the upstream pathological image corresponding to each of the historical clinical pathological images.

[0127] The downstream segmentation simulation is performed on each of the upstream pathological images by using the downstream task of the large model to obtain the downstream pathological image corresponding to each of the historical clinical pathological images.

[0128] The convolution kernel of the backbone network is set to 4*4, and the first downsampling is performed on the upstream pathological image and the downstream pathological image respectively, to obtain the corresponding upstream semantic features and downstream semantic features, and to construct the corresponding upstream semantic pathological image and downstream semantic pathological image;

[0129] The convolution kernel of the backbone network is set to 2*2, and the iterative loop downsampling is performed on the upstream semantic pathological image and the downstream semantic pathological image respectively, until the upstream high-dimensional pathological image and the downstream high-dimensional pathological image are obtained;

[0130] The upstream fine features of the upstream pathological image are constructed according to the corresponding upstream high-dimensional features and the corresponding upstream semantic features of the upstream high-dimensional pathological image, and the downstream fine features of the downstream pathological image are constructed according to the corresponding downstream high-dimensional features and the corresponding downstream semantic features of the downstream high-dimensional pathological image.

[0131] In this example, the number of iterative loop downsampling is not more than 3, that is, the total number of downsampling is not more than 4 including the first downsampling.

[0132] The working principle and beneficial effects of the above technical solution are as follows: the upstream pathological image and the downstream pathological image of the historical clinical pathological image are derived by simulating the upstream task and the downstream task of the large model, and then the fine features of the upstream pathological image and the downstream pathological image are determined by the loop sampling method, which facilitates the subsequent task docking work.

[0133] Embodiment 7

[0134] Based on the embodiment 1, the clinical application level pathological large model construction method based on weakly supervised learning, the step 4 comprises:

[0135] Step 41: inputting the current clinical image into the pathological large model to execute the upstream task and the downstream task of the large model, to obtain a plurality of pathological recognition labels of the current clinical pathological image;

[0136] Step 42: respectively searching for the corresponding pathological recognition annotation of the current clinical pathological image of each pathological recognition label corresponding to the label meaning;

[0137] Step 43: updating the pathological recognition annotation according to the additional annotation issued by the doctor, generating and displaying the pathological report of the patient.

[0138] The working principle and beneficial effects of the above technical solution are as follows: the doctor can often judge the patient's condition according to his own experience and the patient's current condition, and the information issued by the doctor is combined when generating the pathological report, which can assist the doctor in diagnosis.

[0139] Embodiment 8

[0140] Based on the embodiment 7, the method for constructing a clinical application level pathological large model based on weakly supervised learning further comprises:

[0141] According to the additional annotation, the model recognition defects of the pathological large model are derived;

[0142] In the pathological report, the model un-recognized content corresponding to the model recognition defects is found, and the numerical information corresponding to the model un-recognized content is converted into model updating parameters;

[0143] The model parameters of the pathological large model are adjusted by using the model updating parameters.

[0144] The working principle and beneficial effects of the above technical solution are as follows: according to the additional annotation of the doctor, the failure defects of the pathological large model are derived, and then repaired and updated, so as to continuously improve the operation recognition accuracy of the pathological large model.

[0145] Embodiment 9

[0146] The embodiment provides a system for constructing a clinical application level pathological large model based on weakly supervised learning, as shown in Figure 2 The system comprises:

[0147] An upstream training module is configured to perform weakly supervised training on each historical clinical pathology image respectively, adjust the upstream learning parameters of the large model according to the training result, and construct an upstream task of the large model;

[0148] A downstream training module is configured to perform segmentation decoding on each weakly supervised training result respectively, determine a segmentation result, adjust the downstream learning parameters of the large model, and construct a downstream task of the large model;

[0149] A comprehensive training module is configured to obtain upstream fine features and downstream fine features, construct a collaborative task of the upstream and downstream of the large model, and generate a pathological large model by combining the upstream task of the large model and the downstream task of the large model;

[0150] An identification execution module is configured to input a current clinical pathology image into the pathological large model to perform pathological information identification, obtain a plurality of pathological identification labels corresponding to a patient, and generate a pathological report of the patient.

[0151] In this example, the pathological large model is divided into an upstream part and a downstream part, the upstream part improves the feature extraction capability, and the downstream part improves the fine classification of the disease;

[0152] In this example, the large model upstream task represents a task that needs to be performed by the upstream part of the large model when processing the current clinical image, and the large model downstream task represents a task that needs to be performed by the downstream part of the large model when processing the current clinical image.

[0153] In this example, the upstream and downstream collaborative task represents a task that needs to be performed when the image processed by the upstream part is transferred to the downstream part for processing.

[0154] In this example, the pathology recognition label represents a label used to distinguish different diseases of different organs.

[0155] In this example, the historical clinical pathology image and the current clinical pathology image can be images that have undergone WSI diagnosis, or can be original images.

[0156] The working principle and beneficial effects of the above technical solutions are as follows: in order to reduce the misleading influence of incorrect labels on diagnosis and achieve efficient collaborative pathology analysis, the above requirements can be achieved by using model technology. First, the upstream part and the downstream part of the large model are trained separately, which can ensure better integration of the large model with the pathology field. Then, the collaborative task between the two parts is formulated to effectively improve the efficiency of model operation and the accuracy of output results, significantly improve the learning ability and prediction accuracy in the scene where the annotation data quality is not high. Finally, when using the pathology large model in practice, the corresponding report is output according to the demand for reference by patients and doctors. In this way, the original incorrect labels can be eliminated in the upstream part of the large model, and the patient's lesion can be located in the downstream part of the large model, reducing the computational complexity when processing high-resolution images, optimizing the collaborative mechanism between different tasks in the large model, and can be widely applied in pathology recognition technology.

[0157] Embodiment 10

[0158] Based on the embodiment 9, the clinical application level pathology large model construction system based on weakly supervised learning, the upstream training module comprises:

[0159] The preliminary processing unit is configured to perform digital processing on each of the historical clinical pathology images to obtain WSI features of each of the historical clinical pathology images, determine a plurality of basic pathology information corresponding to each of the historical clinical pathology images, respectively acquire historical diagnosis information corresponding to each of the historical clinical pathology images, and determine standard pathology labels corresponding to each of the historical clinical pathology images.

[0160] a supervision training unit, configured to perform weak supervision training on each of the historical clinicopathological images respectively to obtain inaccurate supervision results, partial supervision results and indirect supervision results corresponding to each of the historical clinicopathological images, and to perform label matching on the basic pathological information by using the supervision results to obtain a plurality of supervised pathological labels corresponding to the historical clinicopathological images;

[0161] a recognition analysis unit, configured to compare the supervised pathological labels corresponding to each of the historical clinicopathological images with the corresponding standard pathological labels, and to determine recognition sensitivity and recognition error values of the weak supervision training in different label fields in combination with label fields corresponding to each of the standard pathological labels;

[0162] a task generation unit, configured to perform error adjustment on error label fields with a recognition sensitivity lower than a standard sensitivity, to generate upstream learning parameters of the large model in combination with effective recognition sensitivity of effective error label fields with a recognition sensitivity not lower than the standard sensitivity, and to generate a large model upstream task by adjusting training parameters of the weak supervision training according to the upstream learning parameters.

[0163] In this example, the digitization processing represents a process of scanning historical clinicopathological images and digitizing them;

[0164] In this example, the WSI feature represents information extracted from the historical clinicopathological images;

[0165] In this example, the historical diagnosis information represents retained information after diagnosis of the patient;

[0166] In this example, the standard pathological label represents a label that can accurately express the disease of the patient;

[0167] In this example, the inaccurate supervision result represents a result of supervision on only data-labeled content or any content without data in the historical clinicopathological images, the partial supervision result represents a result of supervision on unclear data content in the historical clinicopathological images, and the indirect supervision result represents a supervision result derived from existing data content;

[0168] In this example, the supervised pathological label represents a pathological label recognized in the historical clinicopathological images;

[0169] In this example, the label field represents a pathological field to which a standard pathological label belongs;

[0170] In this example, the recognition sensitivity represents sensitivity and accuracy of the weak supervision training in recognizing a label field, and the recognition error value represents a maximum error of the weak supervision training in recognizing a label field;

[0171] In this example, the standard sensitivity is 98.9%.

[0172] The working principle and beneficial effects of the above technical solution are as follows: in order to ensure the weak supervision learning capability of the upstream part of the large model, construct an efficient physiological large model, and ensure the recognition accuracy of the current clinical image, first, the historical clinical pathology image is digitally processed to obtain the WSI feature of the historical clinical pathology image, and the basic pathological information corresponding to the organ is determined, thereby combining the historical diagnosis information to determine the standard pathological label of each historical clinical pathology image, then the supervised pathological label of the historical clinical pathology image is generated through weak supervision learning, the recognition sensitivity and recognition error of weak supervision training are determined by comparing the standard pathological label and the historical pathological label, and the error label field with a higher recognition error value is adjusted for error, the upstream learning parameter of the large model is determined, the training parameter of the weak supervision learning is adjusted, the upstream task of the large model is constructed, the accuracy of the upstream task can be gradually improved through training, the accurate numerical value is used for continuous optimization and adjustment, the discriminant learning method of the optimization model is optimized, and the accurate capture ability of the feature is improved, the influence of the error label is gradually reduced, the model can pay more attention to the key feature area in the positive case, thereby the robustness and generalization ability of feature extraction are improved, and the stability of the model in clinical application is ensured.

[0173] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for constructing a clinical application-level pathological large model based on weakly supervised learning, characterized in that, The method comprises the following steps: Step 1: respectively weakly supervised training is carried out on each historical clinical pathology image, the upstream learning parameters of the large model are adjusted according to the training result, and the upstream task of the large model is constructed; Step 2: respectively segmenting and decoding each weakly supervised training result, determining the downstream learning parameters of the large model, and constructing the downstream task of the large model; Step 3: obtain the upstream fine features and downstream fine features, construct the upstream and downstream collaborative task of the large model, combine the upstream task of the large model and the downstream task of the large model to generate a pathology large model; Step 4: input the current clinical pathology image into the pathology large model to identify the pathology information, obtain a plurality of pathology identification tags corresponding to the patient, and generate a pathology report of the patient; The step 1 comprises: Step 11: respectively digitizing each historical clinical pathology image to obtain the WSI features of each historical clinical pathology image, determining a plurality of basic pathology information corresponding to each historical clinical pathology image, respectively obtaining the historical diagnosis information corresponding to each historical clinical pathology image, and determining the standard pathology label corresponding to each historical clinical pathology image; Step 12: respectively weakly supervised training is carried out on each historical clinical pathology image, and the inaccurate supervision result, the partial supervision result and the indirect supervision result corresponding to each historical clinical pathology image are obtained, the basic pathology information is matched with the supervision result to obtain a plurality of supervision pathology labels corresponding to the historical clinical pathology image; Step 13: comparing the supervision pathology label corresponding to each historical clinical pathology image with the corresponding standard pathology label, combining each standard pathology label corresponding to the label field to determine the identification sensitivity and identification error value corresponding to the weakly supervised training in different label fields; Step 14: adjusting the error label field with an identification sensitivity lower than a standard sensitivity, combining the effective identification sensitivity corresponding to the effective error label field with an identification sensitivity not lower than a standard sensitivity to generate the upstream learning parameters of the large model, adjusting the training parameters of the weakly supervised training according to the upstream learning parameters, and generating the upstream task of the large model; The step 2 comprises: Step 21: respectively preliminarily segmenting and decoding each weakly supervised training result corresponding to the historical clinical pathology image by using a pan-cancer identification decoder, determining a specific organ corresponding to each historical clinical pathology image, and assigning a corresponding tumor typing decoder and lesion analysis decoder to each historical clinical pathology image according to the specific organ; Step 22: organ lesion decoding is carried out on the corresponding preliminary segmentation decoding result by using the tumor typing decoder and the lesion analysis decoder, a semantic segmentation data set corresponding to each historical clinical pathology image is constructed, and lesion identification features of corresponding attributes are generated in the semantic segmentation data set according to the task attributes of the upstream task of the large model. Step 23: respectively identify the features of each of the lesions for enhancement processing, determine the region division image frame of each of the historical clinical pathology images, determine the frame pathological information corresponding to each of the region division image frames, and respectively evaluate the degree of fit of each of the frame pathological information in combination with the corresponding historical medical records; Step 24: adjusting the downstream learning parameters of the large model according to the degree of fit evaluation value, adjusting the decoding accuracy of each decoder using the downstream learning parameters, and determining the decoding task generation downstream task of the large model corresponding to each of the decoders.

2. The method of claim 1, wherein the method is performed by a computer system. Also includes: respectively dividing the standard pathological labels contained in each of the historical clinical pathology images into cancerous pathological labels and non-cancerous pathological labels; screening the key label field containing the cancerous pathological label, reducing and correcting the key recognition sensitivity according to the corresponding key recognition error value, and obtaining the weighted sensitivity of the key label field; When the weighted sensitivity is lower than the standard sensitivity, the key label field is regarded as an error label field.

3. The method of claim 1, wherein the method is performed by a computer system. The step 3 includes: Step 31: adjusting the convolution kernel size of the large nuclear convolution according to the large model upstream task and the large model downstream task, establishing the backbone network of the large model, and capturing the upstream fine features of the upstream pathology image and the downstream fine features of the downstream pathology image using the backbone network; Step 32: butt joint the upstream fine features and the downstream fine features to determine a plurality of butt joint points, fix the butt joint points in the upstream fine features and the downstream fine features, and match each non-fixed upstream point / non-fixed downstream point with the corresponding upstream cooperative task / downstream cooperative task according to the point relationship between each non-fixed upstream point / non-fixed downstream point and the butt joint points; Step 33: simulating the upstream cooperative task and the downstream cooperative task according to the fixed position of the butt joint points to generate the upstream and downstream cooperative tasks of the large model, butt joint the large model upstream task and the large model downstream task using the upstream and downstream cooperative tasks, and generate the pathology large model.

4. The method of claim 3, wherein the weakly supervised learning-based clinical application-level pathology large model construction method is characterized by, Capturing the upstream fine features of the upstream pathology image and the downstream fine features of the downstream pathology image using the backbone network includes: respectively performing upstream weak supervision simulation on each of the historical clinical pathology images using the large model upstream task to obtain the upstream pathology image corresponding to each of the historical clinical pathology images; respectively performing downstream segmentation simulation on each of the upstream pathology images using the large model downstream task to obtain the downstream pathology image corresponding to each of the historical clinical pathology images; setting the convolution kernel of the backbone network to 4*4 and the stride to 4, respectively performing first downsampling on the upstream pathology image and the downstream pathology image to obtain the corresponding upstream semantic features and downstream semantic features, and constructing the corresponding upstream semantic pathology image and downstream semantic pathology image; The convolution kernel of the backbone network is set to 2*2, and the stride is set to 2, and the upstream semantic pathological image and the downstream semantic pathological image are iteratively and circularly down-sampled until the upstream high-dimensional pathological image and the downstream high-dimensional pathological image are obtained; The upstream fine features of the upstream pathological image are constructed according to the corresponding upstream high-dimensional features and the corresponding upstream semantic features of the upstream high-dimensional pathological image, and the downstream fine features of the downstream pathological image are constructed according to the corresponding downstream high-dimensional features and the corresponding downstream semantic features of the downstream high-dimensional pathological image.

5. The method of claim 1, wherein the weakly supervised learning-based clinical application-level pathology large model construction method is characterized by, The step 4 comprises: Step 41: inputting the current clinical image into the pathological large model to perform the large model upstream task and the large model downstream task, and obtaining a plurality of pathological identification labels of the current clinical pathological image; Step 42: respectively searching for the corresponding pathological identification annotation of the label meaning of each pathological identification label in the current clinical pathological image; Step 43: updating the pathological identification annotation according to the additional annotation issued by the doctor, generating and displaying the pathological report of the patient.

6. The method of claim 5, wherein the weakly supervised learning-based clinical application-level pathology large model construction method is characterized by, Further comprising: deducing the model identification defect of the pathological large model according to the additional annotation; finding the model unidentified content corresponding to the model identification defect in the pathological report, and converting the numerical information corresponding to the model unidentified content into a model update parameter; adjusting the model parameters of the pathological large model by using the model update parameter.

7. A clinical application-level pathology large model construction system based on weakly supervised learning for implementing the method of claim 1, characterized in that, Comprise: An upstream training module for respectively performing weak supervision training on each historical clinical pathological image, adjusting the upstream learning parameters of the large model according to the training result, and constructing the large model upstream task; A downstream training module for respectively performing segmentation decoding on each weak supervision training result, determining the segmentation result, adjusting the downstream learning parameters of the large model, and constructing the large model downstream task; An integrated training module for obtaining upstream fine features and downstream fine features, constructing the upstream and downstream collaborative tasks of the large model, and generating a pathological large model in combination with the large model upstream task and the large model downstream task; An identification execution module for inputting a current clinical pathological image into the pathological large model to perform pathological information identification, obtaining a plurality of pathological identification labels corresponding to the patient, and generating a pathological report of the patient. 8.The weakly supervised learning based clinical application level pathological large model construction system of claim 7, wherein, The upstream training module comprises: A preliminary processing unit for respectively performing digital processing on each historical clinical pathological image to obtain WSI features of each historical clinical pathological image, determining a plurality of basic pathological information corresponding to each historical clinical pathological image, respectively obtaining historical diagnosis information corresponding to each historical clinical pathological image, and determining standard pathological labels corresponding to each historical clinical pathological image; A supervision training unit for respectively performing weak supervision training on each historical clinical pathological image to obtain inaccurate supervision results, partial supervision results and indirect supervision results corresponding to each historical clinical pathological image, performing label matching on the basic pathological information by using the supervision results, and obtaining a plurality of supervised pathological labels corresponding to the historical clinical pathological image; The recognition analysis unit is configured to compare the supervision pathology label corresponding to each historical clinicopathological image with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value of the weak supervision training corresponding to different label fields in combination with the label field corresponding to each standard pathology label; The task generation unit is configured to adjust the error label field with a recognition sensitivity lower than a standard sensitivity, generate an upstream learning parameter of the large model in combination with the effective recognition sensitivity corresponding to the effective error label field with a recognition sensitivity not lower than the standard sensitivity, and adjust the training parameter of the weak supervision training according to the upstream learning parameter to generate a large model upstream task.

Images