Clinical application level pathological large model construction method and system based on weak supervised learning

Abstract

The invention provides a clinical application level pathology large model construction method and system based on weak supervised learning. The method comprises the steps of performing weak supervised training on each historical clinical pathology image, adjusting an upstream learning parameter of a large model according to a training result, constructing an upstream task of the large model, and performing segmentation decoding on each weak supervised training result, determining a segmentation result, adjusting downstream learning parameters of the large model, constructing a large model downstream task, obtaining upstream fine features and downstream fine features, constructing an upstream and downstream cooperative task of the large model, and generating a pathological large model in combination with the large model upstream task and the large model downstream task. And inputting the current clinical pathological image into the large pathological model for pathological information identification, obtaining a plurality of pathological identification tags corresponding to the patient, and generating a pathological report of the patient, thereby reducing the calculation complexity when processing a high-resolution image, optimizing a collaboration mechanism among different tasks in the large model, and improving the efficiency of the large model. The method can be widely applied to pathological recognition technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of large model generation, and in particular to a method and system for constructing a clinical application-level pathology large model based on weakly supervised learning. Background Art

[0002] With the rapid development of medical technology, especially driven by technologies such as artificial intelligence, big data, and cloud computing, the medical field has gradually entered a data-driven, intelligent era. Pathology, as a core area of ​​disease diagnosis, plays a vital role, especially in the diagnosis of major diseases such as cancer and tumors. The accuracy of pathology directly affects the treatment plan and prognosis of the disease. However, traditional pathological diagnostic methods are highly dependent on the experience and knowledge accumulated by pathologists. In many regions, the uneven distribution of medical resources and the shortage of pathologists are more prominent, resulting in limited pathological diagnostic capabilities in grassroots hospitals and difficulty in ensuring diagnostic quality and efficiency.

[0003] At the same time, with the changing spectrum of diseases and the diversification of medical needs, traditional pathology diagnostic methods face numerous challenges. This is particularly true for the diagnosis of rare diseases, unknown diseases, and complex lesions. Manual pathology diagnostic methods are no longer able to meet the growing demand for precision medicine. Furthermore, existing WSI diagnostic technologies, in practical applications, are often plagued by issues such as data annotation difficulty, training data size, and model generalization capabilities, limiting their widespread application.

[0004] To address these challenges, intelligent pathology technology has been proposed and gradually developed. By leveraging artificial intelligence technologies such as deep learning, image recognition, and natural language processing, intelligent pathology not only accelerates WSI processing but also unlocks more potential information within pathology data, assisting pathologists in making more accurate diagnoses. This development direction has garnered widespread attention from research institutions and businesses within the medical community and has achieved some success. However, existing technologies still suffer from issues such as limited ability to identify specific lesion types, a lack of broad applicability across diseases and organs, and weak model generalization capabilities.

[0005] Therefore, the present invention provides a method and system for constructing a large clinical application-level pathology model based on weakly supervised learning. Summary of the Invention

[0006] The present invention's clinical application-level pathology large-scale model construction method and system based on weakly supervised learning can realize cross-disease and cross-organ lesion identification, assisting doctors in making accurate diagnoses in a short period of time, reducing the reliance on large amounts of labeled data in traditional technologies and enhancing the accuracy of diagnostic results.

[0007] The present invention provides a method for constructing a large clinical application-level pathology model based on weakly supervised learning, comprising:

[0008] Step 1: Perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model;

[0009] Step 2: Segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model;

[0010] Step 3: Obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a large pathology model by combining the upstream tasks of the large model and the downstream tasks of the large model;

[0011] Step 4: Input the current clinical pathology image into the pathology model to identify the pathology information, obtain a number of pathology identification labels corresponding to the patient, and generate a pathology report for the patient.

[0012] In one practicable manner,

[0013] The step 1 comprises:

[0014] Step 11: digitally process each of the historical clinical pathology images to obtain the WSI features of each of the historical clinical pathology images, determine a number of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images;

[0015] Step 12: performing weak supervision training on each of the historical clinical pathology images to obtain inaccurate supervision results, partial supervision results, and indirect supervision results corresponding to each of the historical clinical pathology images, and using the supervision results to perform label matching on the basic pathology information to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images;

[0016] Step 13: Compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value corresponding to the weakly supervised training in different label fields in combination with the label field corresponding to each of the standard pathology labels;

[0017] Step 14: Perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, and combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity to generate the upstream learning parameters of the large model, adjust the training parameters of the weakly supervised training according to the upstream learning parameters, and generate the upstream task of the large model.

[0018] In one practicable manner,

[0019] Also includes:

[0020] respectively classifying the standard pathology labels contained in each of the historical clinical pathology images into cancerous pathology labels and non-cancer pathology labels;

[0021] Screening key label areas containing cancer pathology labels, reducing and correcting the key recognition sensitivity according to the corresponding key recognition error value, and obtaining the weighted sensitivity of the key label areas;

[0022] When the weighted sensitivity is lower than the standard sensitivity, the key label area is regarded as an error label area.

[0023] In one practicable manner,

[0024] The step 2 comprises:

[0025] Step 21: Using a pan-cancer recognition decoder, perform preliminary segmentation decoding on the weakly supervised training results corresponding to each of the historical clinical pathology images, determine the specific organ corresponding to each of the historical clinical pathology images, and assign a corresponding tumor classification decoder and lesion analysis decoder to each of the historical clinical pathology images based on the specific organ;

[0026] Step 22: Using the tumor classification decoder and lesion analysis decoder to perform organ lesion decoding on the corresponding preliminary segmentation decoding results, construct a semantic segmentation dataset corresponding to each of the historical clinical pathology images, and generate lesion recognition features with corresponding attributes in the semantic segmentation dataset according to the task attributes of the upstream task of the large model;

[0027] Step 23: performing enhancement processing on each of the lesion identification features, determining a region division image frame of each of the historical clinical pathology images, determining the frame pathology information corresponding to each of the region division image frames, and performing consistency evaluation on each of the frame pathology information in combination with the corresponding historical medical records;

[0028] Step 24: Adjust the downstream learning parameters of the large model according to the goodness of fit evaluation value, use the downstream learning parameters to adjust the decoding accuracy corresponding to each decoder, and determine the decoding task corresponding to each decoder to generate the downstream task of the large model.

[0029] In one practicable manner,

[0030] The step 3 comprises:

[0031] Step 31: adjusting the convolution kernel size of the large kernel convolution according to the upstream task of the large model and the downstream task of the large model, establishing the backbone network of the large model, and using the backbone network to capture upstream fine features of the upstream pathology image and downstream fine features of the downstream pathology image;

[0032] Step 32: docking the upstream fine feature with the downstream fine feature to determine a number of dockable points, fixing the dockable points in the upstream fine feature and the downstream fine feature, and matching a corresponding upstream collaborative task / downstream collaborative task to each non-fixed upstream point / non-fixed downstream point based on the point relationship between each non-fixed upstream point and the dockable point;

[0033] Step 33: Collaboratively simulate the upstream collaborative task and the downstream collaborative task according to the fixed position of the dockable point to generate the upstream and downstream collaborative tasks of the large model, and use the upstream and downstream collaborative tasks to dock the upstream task of the large model with the downstream task of the large model to generate a large pathological model.

[0034] In one practicable manner,

[0035] Capturing upstream fine features of the upstream pathology image and capturing downstream fine features of the downstream pathology image using the backbone network includes:

[0036] Using the upstream task of the large model to perform upstream weakly supervised simulation on each of the historical clinical pathology images, to obtain an upstream pathology image corresponding to each of the historical clinical pathology images;

[0037] Performing downstream segmentation simulation on each upstream pathology image using the downstream task of the large model to obtain a downstream pathology image corresponding to each historical clinical pathology image;

[0038] The convolution kernel of the backbone network is set to 4*4, the stride is set to 4, the upstream pathology image and the downstream pathology image are respectively downsampled for the first time to obtain corresponding upstream semantic features and downstream semantic features, and the corresponding upstream semantic pathology image and downstream semantic pathology image are constructed;

[0039] The convolution kernel of the backbone network is set to 2*2, the stride is set to 2, and the upstream semantic pathology image and the downstream semantic pathology image are iteratively downsampled respectively until the upstream high-dimensional pathology image and the downstream high-dimensional pathology image are obtained;

[0040] The upstream fine features of the upstream pathology image are constructed according to the upstream high-dimensional features and the corresponding upstream semantic features corresponding to the upstream high-dimensional pathology image, and the downstream fine features of the downstream pathology image are constructed according to the downstream high-dimensional features and the corresponding downstream semantic features corresponding to the downstream high-dimensional pathology image.

[0041] In one practicable manner,

[0042] The step 4 comprises:

[0043] Step 41: Input the current clinical image into the pathology large model to execute the large model upstream tasks and large model downstream tasks to obtain a number of pathology identification labels of the current clinical pathology image;

[0044] Step 42: respectively searching for the pathology identification label corresponding to each pathology identification label and the corresponding pathology identification label in the current clinical pathology image;

[0045] Step 43: Update the pathology identification annotation according to the additional annotation issued by the doctor, generate the patient's pathology report and display it.

[0046] In one practicable manner,

[0047] Also includes:

[0048] Derivation of model identification defects of the pathology large model based on the additional annotations;

[0049] Searching the pathology report for model-unrecognized content corresponding to the model-recognized defect, and converting numerical information corresponding to the model-unrecognized content into model update parameters;

[0050] The model update parameters are used to adjust the model parameters of the large pathology model.

[0051] The present invention provides a system for building a large clinical application-level pathology model based on weakly supervised learning, including:

[0052] The upstream training module is used to perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model;

[0053] A downstream training module is used to segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model;

[0054] A comprehensive training module is used to obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a pathology large model by combining the upstream tasks of the large model and the downstream tasks of the large model;

[0055] The recognition execution module is used to input the current clinical pathology image into the pathology large model to perform pathology information recognition, obtain a number of pathology identification tags corresponding to the patient, and generate a pathology report for the patient.

[0056] In one practicable manner,

[0057] The upstream training module includes:

[0058] a preliminary processing unit, configured to digitally process each of the historical clinical pathology images, obtain the WSI features of each of the historical clinical pathology images, determine a plurality of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images;

[0059] a supervised training unit, configured to perform weakly supervised training on each of the historical clinical pathology images, obtain an inaccurate supervised result, a partial supervised result, and an indirect supervised result corresponding to each of the historical clinical pathology images, and perform label matching on the basic pathology information using the supervised results to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images;

[0060] a recognition and analysis unit, configured to compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value of the weakly supervised training corresponding to different label fields in combination with the label field corresponding to each of the standard pathology labels;

[0061] The task generation unit is used to perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity, generate the upstream learning parameters of the large model, adjust the training parameters of the weak supervision training according to the upstream learning parameters, and generate the upstream task of the large model.

[0062] The achievable beneficial effects of the above technical solution are: in order to reduce the misleading impact of erroneous labels on diagnosis and to achieve efficient and collaborative pathological analysis, the above requirements can be achieved by using model technology. First, the upstream and downstream parts of the large model are trained separately to ensure that the large model is better aligned with the pathology field. Then, the collaborative tasks between the two parts are formulated, which effectively improves the efficiency of the model operation and the accuracy of the output results, and significantly improves the learning ability and prediction accuracy in scenarios with low quality of labeled data. Finally, when the large pathology model is actually used, the corresponding report is output according to demand for reference by patients and doctors. In this way, the original erroneous labels can be eliminated in the upstream part of the large model, and the patient's lesions can be located in the downstream part of the large model, reducing the computational complexity of processing high-resolution images, and optimizing the collaborative mechanism between different tasks in the large model. It can be widely used in pathology recognition technology.

[0063] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 Schematic diagram of the workflow of a method for constructing a large clinical application-level pathology model based on weakly supervised learning in an embodiment of the present invention;

[0067] Figure 2 Schematic diagram of the composition of a system for building a large clinical application-level pathology model based on weakly supervised learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0069] Example 1

[0070] This embodiment provides a method for constructing a large clinical application-level pathology model based on weakly supervised learning, such as Figure 1 Shown, including:

[0071] Step 1: Perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model;

[0072] Step 2: Segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model;

[0073] Step 3: Obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a large pathology model by combining the upstream tasks of the large model and the downstream tasks of the large model;

[0074] Step 4: Input the current clinical pathology image into the pathology model to identify the pathology information, obtain a number of pathology identification labels corresponding to the patient, and generate a pathology report for the patient.

[0075] In this example, the pathology model is divided into upstream and downstream parts. The upstream part improves feature extraction capabilities, while the downstream part improves the refined classification of diseases.

[0076] In this example, the large model upstream task refers to the task that the upstream part of the large model needs to perform when processing the current clinical image, and the large model downstream task refers to the task that the downstream part of the large model needs to perform when processing the current clinical image;

[0077] In this example, the upstream and downstream collaborative tasks represent the tasks that need to be performed when transferring the image processed by the upstream part to the downstream part for processing;

[0078] In this example, the pathology identification label represents a mark used to distinguish diseases of different organs;

[0079] In this example, the historical clinical pathology image and the current clinical pathology image may be images that have undergone WSI diagnosis or may be original images.

[0080] The working principle and beneficial effects of the above technical solution: In order to reduce the misleading impact of erroneous labels on diagnosis and realize efficient and collaborative pathological analysis, the above requirements can be achieved by using model technology. First, the upstream and downstream parts of the large model are trained separately to ensure that the large model is better aligned with the pathology field. Then, the collaborative tasks between the two parts are formulated, which effectively improves the efficiency of the model operation and the accuracy of the output results, and significantly improves the learning ability and prediction accuracy in scenarios with low quality of labeled data. Finally, when the pathology large model is actually used, the corresponding report is output according to demand for reference by patients and doctors. In this way, the original erroneous labels can be eliminated in the upstream part of the large model, and the patient's lesions can be located in the downstream part of the large model, which reduces the computational complexity of processing high-resolution images and optimizes the collaborative mechanism between different tasks in the large model. It can be widely used in pathology recognition technology.

[0081] Example 2

[0082] On the basis of Example 1, the method for constructing a large clinical application-level pathology model based on weakly supervised learning, step 1, comprises:

[0083] Step 11: digitally process each of the historical clinical pathology images to obtain the WSI features of each of the historical clinical pathology images, determine a number of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images;

[0084] Step 12: performing weak supervision training on each of the historical clinical pathology images to obtain inaccurate supervision results, partial supervision results, and indirect supervision results corresponding to each of the historical clinical pathology images, and using the supervision results to perform label matching on the basic pathology information to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images;

[0085] Step 13: Compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value corresponding to the weakly supervised training in different label fields in combination with the label field corresponding to each of the standard pathology labels;

[0086] Step 14: Perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, and combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity to generate the upstream learning parameters of the large model, adjust the training parameters of the weakly supervised training according to the upstream learning parameters, and generate the upstream task of the large model.

[0087] In this example, digital processing refers to the process of scanning historical clinical pathology images and digitizing them;

[0088] In this instance, the WSI features represent the information extracted from historical clinical pathology images;

[0089] In this example, historical diagnosis information refers to the retained information after the patient is diagnosed;

[0090] In this example, the standard pathology label refers to a label that can accurately express the patient's disease;

[0091] In this example, inaccurate supervision results refer to the results of supervision on content in historical clinical pathology images that only have data annotations or no data at all; partial supervision results refer to the results of supervision on content in historical clinical pathology images that have unclear data; and indirect supervision results refer to the results of supervision derived from existing data content.

[0092] In this instance, supervised pathology labels represent pathology labels identified in historical clinical pathology images;

[0093] In this instance, the label field represents the pathology field to which a standard pathology label belongs;

[0094] In this example, the recognition sensitivity indicates the sensitivity and accuracy of weakly supervised training in identifying a label domain, and the recognition error value indicates the maximum error that exists in weakly supervised training in identifying a label domain.

[0095] In this example, the standard sensitivity is 98.9%.

[0096] The working principle and beneficial effects of the above technical solution are as follows: In order to ensure the ability of the upstream part of the large model to perform weakly supervised learning, construct a highly efficient physiological large model, and ensure the recognition accuracy of current clinical images, the historical clinical pathology images are first digitized to obtain the WSI features of the historical clinical pathology images and determine the basic pathological information corresponding to the device. The standard pathology label of each historical clinical pathology image is then determined in combination with the historical diagnostic information. Then, the supervised pathology label of the historical clinical pathology image is generated through weakly supervised learning. The recognition sensitivity and recognition error of the weakly supervised training are determined by comparing the standard pathology label with the historical pathology label, so that the error adjustment is performed for the error label area with high recognition error values. The upstream learning parameters of the large model are determined, and the training parameters of the weakly supervised learning are adjusted to construct the upstream task of the large model. The accuracy of the upstream task can be gradually improved through training. Continuous optimization and adjustment are performed using precise numerical values. By optimizing the discriminant learning method of the model and improving the ability to accurately capture features, the impact of incorrect labels is gradually reduced, so that the model can focus more on the key feature areas in positive cases, thereby improving the robustness and generalization ability of feature extraction and ensuring the stability of the model in clinical applications.

[0097] Example 3

[0098] Based on Example 2, the method for constructing a large clinical application-level pathology model based on weakly supervised learning further includes:

[0099] respectively classifying the standard pathology labels contained in each of the historical clinical pathology images into cancerous pathology labels and non-cancer pathology labels;

[0100] Screening key label areas containing cancer pathology labels, reducing and correcting the key recognition sensitivity according to the corresponding key recognition error value, and obtaining the weighted sensitivity of the key label areas;

[0101] When the weighted sensitivity is lower than the standard sensitivity, the key label area is regarded as an error label area.

[0102] The working principle and beneficial effects of the above technical solution are as follows: In order to further improve the accuracy of cancer recognition, key label areas with cancer pathology labels are corrected, and the corresponding processing methods are subsequently performed to determine whether they are error label areas, thereby improving the sensitivity of cancer capture.

[0103] Example 4

[0104] On the basis of Example 1, the method for constructing a large clinical application-level pathology model based on weakly supervised learning, step 2 includes:

[0105] Step 21: Using a pan-cancer recognition decoder, perform preliminary segmentation decoding on the weakly supervised training results corresponding to each of the historical clinical pathology images, determine the specific organ corresponding to each of the historical clinical pathology images, and assign a corresponding tumor classification decoder and lesion analysis decoder to each of the historical clinical pathology images based on the specific organ;

[0106] Step 22: Using the tumor classification decoder and lesion analysis decoder to perform organ lesion decoding on the corresponding preliminary segmentation decoding results, construct a semantic segmentation dataset corresponding to each of the historical clinical pathology images, and generate lesion recognition features with corresponding attributes in the semantic segmentation dataset according to the task attributes of the upstream task of the large model;

[0107] Step 23: performing enhancement processing on each of the lesion identification features, determining a region division image frame of each of the historical clinical pathology images, determining the frame pathology information corresponding to each of the region division image frames, and performing consistency evaluation on each of the frame pathology information in combination with the corresponding historical medical records;

[0108] Step 24: Adjust the downstream learning parameters of the large model according to the goodness of fit evaluation value, use the downstream learning parameters to adjust the decoding accuracy corresponding to each decoder, and determine the decoding task corresponding to each decoder to generate the downstream task of the large model.

[0109] In this instance, the specific organ represents an organ included in a historical clinical pathology image;

[0110] In this example, organ lesion decoding refers to the process of locating the patient's lesion organ and determining the organ disease type of the lesion organ;

[0111] In this example, the lesion identification feature table is used to distinguish the characteristics of different diseases;

[0112] In this example, the label enhancement process is to highlight the significant points in the lesion identification features.

[0113] The working principle and beneficial effects of the above technical solution are as follows: To improve the efficiency and quality of pathology discrimination, a pathology discrimination process is constructed simultaneously during downstream training. First, a pan-cancer recognition decoder is used to determine the characteristic organs to be identified. Then, corresponding tumor classification decoders and lesion analysis decoders are assigned to them. By further analyzing the initial segmentation and decoding results of the pan-cancer recognition decoder, a semantic segmentation dataset of historical clinical pathology images is obtained, thereby identifying the lesion recognition features obtained in the upstream task of the large model. Further enhancement processing is performed on these features to obtain the regional segmentation frame results of the clinical pathology images. Then, based on the historical pathology, the consistency evaluation value of the frame pathology information of each image frame is determined. This value is used to adjust the downstream learning parameters of the large model and the decoding accuracy of the decoder, generating the downstream tasks of the large model. This approach not only allows the location and range of the lesion area to be intuitively annotated during classification, but also generates high-resolution semantic segmentation heat maps, providing doctors with clear lesion distribution information and providing an important basis for further diagnosis and treatment. In addition, the sensitive decoder can be used to cope with complex and variable pathology tasks, improving the adaptability of the diagnostic model to various diseases and organs.

[0114] Example 5

[0115] On the basis of Example 1, the method for constructing a large clinical application-level pathology model based on weakly supervised learning, step 3, includes:

[0116] Step 31: adjusting the convolution kernel size of the large kernel convolution according to the upstream task of the large model and the downstream task of the large model, establishing the backbone network of the large model, and using the backbone network to capture upstream fine features of the upstream pathology image and downstream fine features of the downstream pathology image;

[0117] Step 32: docking the upstream fine feature with the downstream fine feature to determine a number of dockable points, fixing the dockable points in the upstream fine feature and the downstream fine feature, and matching a corresponding upstream collaborative task / downstream collaborative task to each non-fixed upstream point / non-fixed downstream point based on the point relationship between each non-fixed upstream point and the dockable point;

[0118] Step 33: Collaboratively simulate the upstream collaborative task and the downstream collaborative task according to the fixed position of the dockable point to generate the upstream and downstream collaborative tasks of the large model, and use the upstream and downstream collaborative tasks to dock the upstream task of the large model with the downstream task of the large model to generate a large pathological model.

[0119] In this example, the upstream fine features represent the features presented by the fine values ​​whose values ​​are immutable and included in the upstream tasks of the large model, and the downstream fine features represent the features presented by the fine values ​​whose values ​​are immutable and included in the downstream tasks of the large model;

[0120] In this example, the dockable points represent the points where the upstream fine features and the downstream fine features can be directly docked;

[0121] In this example, the non-fixed upstream points represent the remaining points in the upstream fine feature except the dockable points, and the non-fixed lower points represent the remaining points in the downstream fine feature except the dockable points.

[0122] In this example, the collaborative task refers to a task that connects and collaborates the large model upstream task and the large model downstream task.

[0123] The working principle and beneficial effects of the above technical solution are as follows: by setting a backbone network for the large model to ensure that the large model can run normally, first, the convolution size of the large kernel convolution is adjusted according to the upstream tasks and downstream tasks of the large model, and the backbone network of the large model is constructed. Then, the backbone network is used to capture the fine features of the upstream pathology image and the downstream pathology image, and the fine features are further docked. Then, the corresponding collaborative tasks are matched for the undocking points, and the upstream and downstream collaborative tasks of the large model are generated through simulation. Finally, the upstream tasks, downstream tasks and upstream and downstream collaborative tasks of the large model are docked to generate a large pathology model. High-dimensional features are extracted through the large kernel convolution encoder, and combined with the loss function optimization strategy, the problem of misleading feature extraction caused by incorrect labels is solved, the discriminant learning method of the model is optimized and the ability to accurately capture features is improved, the impact of incorrect labels is further gradually reduced, and the effectiveness of the large model is improved.

[0124] Example 6

[0125] Based on Example 5, the method for constructing a large clinical application-level pathology model based on weakly supervised learning, using the backbone network to capture upstream fine features of the upstream pathology image and downstream fine features of the downstream pathology image, includes:

[0126] Using the upstream task of the large model to perform upstream weakly supervised simulation on each of the historical clinical pathology images, to obtain an upstream pathology image corresponding to each of the historical clinical pathology images;

[0127] Performing downstream segmentation simulation on each upstream pathology image using the downstream task of the large model to obtain a downstream pathology image corresponding to each historical clinical pathology image;

[0128] The convolution kernel of the backbone network is set to 4*4, the stride is set to 4, the upstream pathology image and the downstream pathology image are respectively downsampled for the first time to obtain corresponding upstream semantic features and downstream semantic features, and the corresponding upstream semantic pathology image and downstream semantic pathology image are constructed;

[0129] The convolution kernel of the backbone network is set to 2*2, the stride is set to 2, and the upstream semantic pathology image and the downstream semantic pathology image are iteratively downsampled respectively until the upstream high-dimensional pathology image and the downstream high-dimensional pathology image are obtained;

[0130] The upstream fine features of the upstream pathology image are constructed according to the upstream high-dimensional features and the corresponding upstream semantic features corresponding to the upstream high-dimensional pathology image, and the downstream fine features of the downstream pathology image are constructed according to the downstream high-dimensional features and the corresponding downstream semantic features corresponding to the downstream high-dimensional pathology image.

[0131] In this example, the number of iterative downsampling cycles does not exceed 3 times, that is, including the first downsampling, the total number of downsampling cycles does not exceed 4 times.

[0132] The working principle and beneficial effects of the above technical solution are as follows: by simulating the upstream tasks and downstream tasks of the large model, the upstream pathology images and downstream pathology images of the historical clinical pathology images are derived, and then the fine features of the upstream pathology images and downstream pathology images are determined by circular sampling, which facilitates subsequent task docking work.

[0133] Example 7

[0134] On the basis of Example 1, the method for constructing a large clinical application-level pathology model based on weakly supervised learning, step 4, includes:

[0135] Step 41: Input the current clinical image into the pathology large model to execute the large model upstream tasks and large model downstream tasks to obtain a number of pathology identification labels of the current clinical pathology image;

[0136] Step 42: respectively searching for the pathology identification label corresponding to each pathology identification label and the corresponding pathology identification label in the current clinical pathology image;

[0137] Step 43: Update the pathology identification annotation according to the additional annotation issued by the doctor, generate the patient's pathology report and display it.

[0138] The working principle and beneficial effects of the above technical solution: Doctors can often judge the patient's condition based on their own experience and the patient's current condition. When generating a pathology report, the information issued by the doctor should be combined to assist the doctor in diagnosis.

[0139] Example 8

[0140] Based on Example 7, the method for constructing a large clinical application-level pathology model based on weakly supervised learning further includes:

[0141] Derivation of model identification defects of the pathology large model based on the additional annotations;

[0142] Searching the pathology report for model-unrecognized content corresponding to the model-recognized defect, and converting numerical information corresponding to the model-unrecognized content into model update parameters;

[0143] The model update parameters are used to adjust the model parameters of the large pathology model.

[0144] The working principle and beneficial effects of the above technical solution are as follows: based on the additional annotations by doctors, the failure defects of the pathology model are deduced, and then repaired and updated to continuously improve the operational recognition accuracy of the pathology model.

[0145] Example 9

[0146] This embodiment provides a clinical application-level pathology model construction system based on weakly supervised learning, such as Figure 2 Shown, including:

[0147] The upstream training module is used to perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model;

[0148] A downstream training module is used to segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model;

[0149] A comprehensive training module is used to obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a pathology large model by combining the upstream tasks of the large model and the downstream tasks of the large model;

[0150] The recognition execution module is used to input the current clinical pathology image into the pathology large model to perform pathology information recognition, obtain a number of pathology identification tags corresponding to the patient, and generate a pathology report for the patient.

[0151] In this example, the pathology model is divided into upstream and downstream parts. The upstream part improves feature extraction capabilities, while the downstream part improves the refined classification of diseases.

[0152] In this example, the large model upstream task refers to the task that the upstream part of the large model needs to perform when processing the current clinical image, and the large model downstream task refers to the task that the downstream part of the large model needs to perform when processing the current clinical image;

[0153] In this example, the upstream and downstream collaborative tasks represent the tasks that need to be performed when transferring the image processed by the upstream part to the downstream part for processing;

[0154] In this example, the pathology identification label represents a mark used to distinguish diseases of different organs;

[0155] In this example, the historical clinical pathology image and the current clinical pathology image may be images that have undergone WSI diagnosis or may be original images.

[0156] The working principle and beneficial effects of the above technical solution: In order to reduce the misleading impact of erroneous labels on diagnosis and realize efficient and collaborative pathological analysis, the above requirements can be achieved by using model technology. First, the upstream and downstream parts of the large model are trained separately to ensure that the large model is better aligned with the pathology field. Then, the collaborative tasks between the two parts are formulated, which effectively improves the efficiency of the model operation and the accuracy of the output results, and significantly improves the learning ability and prediction accuracy in scenarios with low quality of labeled data. Finally, when the pathology large model is actually used, the corresponding report is output according to demand for reference by patients and doctors. In this way, the original erroneous labels can be eliminated in the upstream part of the large model, and the patient's lesions can be located in the downstream part of the large model, which reduces the computational complexity of processing high-resolution images and optimizes the collaborative mechanism between different tasks in the large model. It can be widely used in pathology recognition technology.

[0157] Example 10

[0158] Based on Example 9, the system for constructing a large clinical application-level pathology model based on weakly supervised learning, the upstream training module includes:

[0159] a preliminary processing unit, configured to digitally process each of the historical clinical pathology images, obtain the WSI features of each of the historical clinical pathology images, determine a plurality of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images;

[0160] a supervised training unit, configured to perform weakly supervised training on each of the historical clinical pathology images, obtain an inaccurate supervised result, a partial supervised result, and an indirect supervised result corresponding to each of the historical clinical pathology images, and perform label matching on the basic pathology information using the supervised results to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images;

[0161] a recognition and analysis unit, configured to compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value of the weakly supervised training corresponding to different label fields in combination with the label field corresponding to each of the standard pathology labels;

[0162] The task generation unit is used to perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity, generate the upstream learning parameters of the large model, adjust the training parameters of the weak supervision training according to the upstream learning parameters, and generate the upstream task of the large model.

[0163] In this example, digital processing refers to the process of scanning historical clinical pathology images and digitizing them;

[0164] In this instance, the WSI features represent the information extracted from historical clinical pathology images;

[0165] In this example, historical diagnosis information refers to the retained information after the patient is diagnosed;

[0166] In this example, the standard pathology label refers to a label that can accurately express the patient's disease;

[0167] In this example, inaccurate supervision results refer to the results of supervision on content in historical clinical pathology images that only have data annotations or no data at all; partial supervision results refer to the results of supervision on content in historical clinical pathology images that have unclear data; and indirect supervision results refer to the results of supervision derived from existing data content.

[0168] In this instance, supervised pathology labels represent pathology labels identified in historical clinical pathology images;

[0169] In this instance, the label field represents the pathology field to which a standard pathology label belongs;

[0170] In this example, the recognition sensitivity indicates the sensitivity and accuracy of weakly supervised training in identifying a label domain, and the recognition error value indicates the maximum error that exists in weakly supervised training in identifying a label domain.

[0171] In this example, the standard sensitivity is 98.9%.

[0172] The working principle and beneficial effects of the above technical solution are as follows: In order to ensure the ability of the upstream part of the large model to perform weakly supervised learning, construct a highly efficient physiological large model, and ensure the recognition accuracy of current clinical images, the historical clinical pathology images are first digitized to obtain the WSI features of the historical clinical pathology images and determine the basic pathological information corresponding to the device. The standard pathology label of each historical clinical pathology image is then determined in combination with the historical diagnostic information. Then, the supervised pathology label of the historical clinical pathology image is generated through weakly supervised learning. The recognition sensitivity and recognition error of the weakly supervised training are determined by comparing the standard pathology label with the historical pathology label, so that the error adjustment is performed for the error label area with high recognition error values. The upstream learning parameters of the large model are determined, and the training parameters of the weakly supervised learning are adjusted to construct the upstream task of the large model. The accuracy of the upstream task can be gradually improved through training. Continuous optimization and adjustment are performed using precise numerical values. By optimizing the discriminant learning method of the model and improving the ability to accurately capture features, the impact of incorrect labels is gradually reduced, so that the model can focus more on the key feature areas in positive cases, thereby improving the robustness and generalization ability of feature extraction and ensuring the stability of the model in clinical applications.

[0173] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for constructing a large clinical application-level pathology model based on weakly supervised learning, characterized by: include: Step 1: Perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model; Step 2: Segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model; Step 3: Obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a large pathology model by combining the upstream tasks of the large model and the downstream tasks of the large model; Step 4: Input the current clinical pathology image into the pathology model to identify the pathology information, obtain a number of pathology identification labels corresponding to the patient, and generate a pathology report for the patient.

2. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 1, wherein: The step 1 comprises: Step 11: digitally process each of the historical clinical pathology images to obtain the WSI features of each of the historical clinical pathology images, determine a number of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images; Step 12: performing weak supervision training on each of the historical clinical pathology images to obtain inaccurate supervision results, partial supervision results, and indirect supervision results corresponding to each of the historical clinical pathology images, and using the supervision results to perform label matching on the basic pathology information to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images; Step 13: Compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value corresponding to the weakly supervised training in different label fields in combination with the label field corresponding to each of the standard pathology labels; Step 14: Perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, and combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity to generate the upstream learning parameters of the large model, adjust the training parameters of the weakly supervised training according to the upstream learning parameters, and generate the upstream task of the large model.

3. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 2, wherein: Also includes: respectively classifying the standard pathology labels contained in each of the historical clinical pathology images into cancerous pathology labels and non-cancer pathology labels; Screening key label areas containing cancer pathology labels, reducing and correcting the key recognition sensitivity according to the corresponding key recognition error value, and obtaining the weighted sensitivity of the key label areas; When the weighted sensitivity is lower than the standard sensitivity, the key label area is regarded as an error label area.

4. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 1, wherein: The step 2 comprises: Step 21: Using a pan-cancer recognition decoder, perform preliminary segmentation decoding on the weakly supervised training results corresponding to each of the historical clinical pathology images, determine the specific organ corresponding to each of the historical clinical pathology images, and assign a corresponding tumor classification decoder and lesion analysis decoder to each of the historical clinical pathology images based on the specific organ; Step 22: Using the tumor classification decoder and lesion analysis decoder to perform organ lesion decoding on the corresponding preliminary segmentation decoding results, construct a semantic segmentation dataset corresponding to each of the historical clinical pathology images, and generate lesion recognition features with corresponding attributes in the semantic segmentation dataset according to the task attributes of the upstream task of the large model; Step 23: performing enhancement processing on each of the lesion identification features, determining a region division image frame of each of the historical clinical pathology images, determining the frame pathology information corresponding to each of the region division image frames, and performing consistency evaluation on each of the frame pathology information in combination with the corresponding historical medical records; Step 24: Adjust the downstream learning parameters of the large model according to the goodness of fit evaluation value, use the downstream learning parameters to adjust the decoding accuracy corresponding to each decoder, and determine the decoding task corresponding to each decoder to generate the downstream task of the large model.

5. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 1, wherein: The step 3 comprises: Step 31: adjusting the convolution kernel size of the large kernel convolution according to the upstream task of the large model and the downstream task of the large model, establishing the backbone network of the large model, and using the backbone network to capture upstream fine features of the upstream pathology image and downstream fine features of the downstream pathology image; Step 32: docking the upstream fine feature with the downstream fine feature to determine a number of dockable points, fixing the dockable points in the upstream fine feature and the downstream fine feature, and matching a corresponding upstream collaborative task / downstream collaborative task to each non-fixed upstream point / non-fixed downstream point based on the point relationship between each non-fixed upstream point and the dockable point; Step 33: Collaboratively simulate the upstream collaborative task and the downstream collaborative task according to the fixed position of the dockable point to generate the upstream and downstream collaborative tasks of the large model, and use the upstream and downstream collaborative tasks to dock the upstream task of the large model with the downstream task of the large model to generate a large pathological model.

6. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 5, characterized in that: Capturing upstream fine features of the upstream pathology image and capturing downstream fine features of the downstream pathology image using the backbone network includes: Using the upstream task of the large model to perform upstream weakly supervised simulation on each of the historical clinical pathology images, to obtain an upstream pathology image corresponding to each of the historical clinical pathology images; Performing downstream segmentation simulation on each upstream pathology image using the downstream task of the large model to obtain a downstream pathology image corresponding to each historical clinical pathology image; The convolution kernel of the backbone network is set to 4*4, the stride is set to 4, the upstream pathology image and the downstream pathology image are respectively downsampled for the first time to obtain corresponding upstream semantic features and downstream semantic features, and the corresponding upstream semantic pathology image and downstream semantic pathology image are constructed; The convolution kernel of the backbone network is set to 2*2, the stride is set to 2, and the upstream semantic pathology image and the downstream semantic pathology image are iteratively downsampled respectively until the upstream high-dimensional pathology image and the downstream high-dimensional pathology image are obtained; The upstream fine features of the upstream pathology image are constructed according to the upstream high-dimensional features and the corresponding upstream semantic features corresponding to the upstream high-dimensional pathology image, and the downstream fine features of the downstream pathology image are constructed according to the downstream high-dimensional features and the corresponding downstream semantic features corresponding to the downstream high-dimensional pathology image.

7. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 1, wherein: The step 4 comprises: Step 41: Input the current clinical image into the pathology large model to execute the large model upstream tasks and large model downstream tasks to obtain a number of pathology identification labels of the current clinical pathology image; Step 42: respectively searching for the pathology identification label corresponding to each pathology identification label and the corresponding pathology identification label in the current clinical pathology image; Step 43: Update the pathology identification annotation according to the additional annotation issued by the doctor, generate the patient's pathology report and display it.

8. The method for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 7, wherein: Also includes: Derivation of model identification defects of the pathology large model based on the additional annotations; Searching the pathology report for model-unrecognized content corresponding to the model-recognized defect, and converting numerical information corresponding to the model-unrecognized content into model update parameters; The model update parameters are used to adjust the model parameters of the large pathology model.

9. A clinical application-level pathology model construction system based on weakly supervised learning, characterized by: include: The upstream training module is used to perform weakly supervised training on each historical clinical pathology image, adjust the upstream learning parameters of the large model based on the training results, and construct the upstream tasks of the large model; A downstream training module is used to segment and decode each weakly supervised training result, determine the segmentation results, adjust the downstream learning parameters of the large model, and construct the downstream tasks of the large model; A comprehensive training module is used to obtain upstream fine features and downstream fine features, build upstream and downstream collaborative tasks of the large model, and generate a pathology large model by combining the upstream tasks of the large model and the downstream tasks of the large model; The recognition execution module is used to input the current clinical pathology image into the pathology large model to perform pathology information recognition, obtain a number of pathology identification tags corresponding to the patient, and generate a pathology report for the patient.

10. The system for constructing a large clinical application-level pathology model based on weakly supervised learning according to claim 9, characterized in that: The upstream training module includes: a preliminary processing unit, configured to digitally process each of the historical clinical pathology images, obtain the WSI features of each of the historical clinical pathology images, determine a plurality of basic pathological information corresponding to each of the historical clinical pathology images, obtain historical diagnostic information corresponding to each of the historical clinical pathology images, and determine a standard pathology label corresponding to each of the historical clinical pathology images; a supervised training unit, configured to perform weakly supervised training on each of the historical clinical pathology images, obtain an inaccurate supervised result, a partial supervised result, and an indirect supervised result corresponding to each of the historical clinical pathology images, and perform label matching on the basic pathology information using the supervised results to obtain a plurality of supervised pathology labels corresponding to the historical clinical pathology images; a recognition and analysis unit, configured to compare the supervised pathology label corresponding to each of the historical clinical pathology images with the corresponding standard pathology label, and determine the recognition sensitivity and recognition error value of the weakly supervised training corresponding to different label fields in combination with the label field corresponding to each of the standard pathology labels; The task generation unit is used to perform error adjustment on the error label field whose recognition sensitivity is lower than the standard sensitivity, combine the effective recognition sensitivity corresponding to the effective error label field whose recognition sensitivity is not lower than the standard sensitivity, generate the upstream learning parameters of the large model, adjust the training parameters of the weak supervision training according to the upstream learning parameters, and generate the upstream task of the large model.

Images