A general pathology diagnosis method based on weakly supervised learning large kernel convolution ConvNeXt V2 model

Abstract

The present invention discloses a weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method, device, equipment and medium. According to the technical solution provided by the present invention, digital pathology slice image data for a target patient is obtained through a medical information system as target image data; a universal pathology diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathology diagnosis model; the target image data is input into the trained universal pathology diagnosis model, and the model is used to perform pathological analysis to obtain the corresponding analysis results; based on the pathological analysis results, a pathology analysis report for the target patient is generated. Through the combination of the ConvNeXt V2 full convolutional network architecture and the FCMAE full convolutional masked autoencoder, the present invention significantly reduces the labor cost and dependence of data annotation, and is applicable to various types of diseases of more than ten organs, with strong generalization and high robustness, which greatly breaks through the barriers of existing pathology AI technology diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of website information screening, and in particular to a general pathology diagnosis method, device, computing equipment and computer storage medium of a weakly supervised learning large kernel convolution ConvNeXt V2 model. Background Art

[0002] With the rapid development of artificial intelligence and deep learning technologies, the application scope of these technologies in various industries is also expanding. Artificial intelligence and deep learning technologies have indeed empowered these industries and fields, and have had a profound impact in areas such as image recognition, classification, and segmentation. In the medical field, CNN-based deep learning technology has made great progress in medical impact analysis, especially in pathological image analysis. It avoids the traditional pathological diagnosis method that relies on pathologists to observe and analyze slices under a microscope, which is easily affected by human subjective factors, resulting in the problem that the accuracy and consistency of the diagnosis results cannot be guaranteed. It also reduces the workload of doctors and improves the overall efficiency and accuracy of the diagnosis and treatment process.

[0003] However, current deep learning-based pathology image analysis is mostly focused on the analysis of a single organ or a specific disease, and a comprehensive pathology analysis system that can be widely applied to multiple organs and multiple diseases has not yet been formed. In addition, the existing intelligent pathology analysis system mainly focuses on disease diagnosis and lesion segmentation, and lacks relevant research on the automatic generation technology of pathology reports, so it cannot help pathologists perform comprehensive auxiliary diagnosis.

[0004] Moreover, for the ResNet deep convolutional neural network, as the network depth continues to increase, the amount of computation and memory consumption will also increase significantly, and the hardware resources requirements are relatively high. Although skip connections help alleviate the gradient vanishing problem, in some cases, especially when the data set is small, the deep network may still overfit; the EfficientNet convolutional neural network has a relatively complex structure and is difficult to design and implement. Larger model versions take a long time to train, which may affect rapid iteration and optimization. In the case of a small data set, the larger model version may be prone to overfitting; the encoder-decoder structure and skip connections of the U-Net convolutional neural network further increase the computational complexity and have high requirements for hardware resources. The complex structure and multi-scale feature extraction lead to a long training time, which affects the rapid iteration and optimization of the model. In the case of a small data set, especially when there are not enough data enhancement methods, U-Net may also be prone to overfitting. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method and a corresponding weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis device, computing equipment and computer storage medium.

[0006] According to one aspect of the present invention, a general pathological diagnosis method of a weakly supervised learning large kernel convolution ConvNeXt V2 model is provided, the method comprising:

[0007] Acquire digital pathological slice image data of a target patient as target image data through a medical information system;

[0008] A universal pathology diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathology diagnosis model;

[0009] Input the target image data into the trained universal pathology diagnosis model, use the model to perform pathology analysis, and obtain the corresponding analysis results;

[0010] Based on the pathological analysis results, a pathological analysis report for the target patient is generated.

[0011] In the above scheme, the step of obtaining digital pathological slice image data of a target patient as target image data through a medical information system further includes:

[0012] Obtaining an information acquisition request, and extracting identity information of a target patient from the information acquisition request;

[0013] Based on the identity information of the target patient, matching is performed from the medical information system to determine the patient data set corresponding to the target patient;

[0014] Based on the information acquisition request, the corresponding digital pathology slice image data is extracted from the patient data set as the target image data.

[0015] In the above scheme, the universal pathology diagnosis model is constructed based on the ConvNeXt V2 network architecture and trained to obtain the trained universal pathology diagnosis model, further comprising:

[0016] Based on the ConvNeXt V2 fully convolutional network architecture, a universal pathology diagnosis model is constructed; wherein, the FCMAE fully convolutional masked autoencoder framework is set in the ConvNeXt V2 fully convolutional network architecture;

[0017] Obtain historical image datasets as model training sets through medical information systems;

[0018] The DINO self-supervised learning paradigm is used to train the universal pathology diagnosis model based on the model training set to obtain the trained universal pathology diagnosis model.

[0019] In the above scheme, the construction of a universal pathology diagnosis model based on the ConvNeXt V2 fully convolutional network architecture further includes:

[0020] In the ConvNeXt V2 fully convolutional network architecture, a GRN layer is set to perform global feature aggregation, feature normalization, and feature calibration;

[0021] Use the L2 norm to aggregate the feature maps on each channel in the model to obtain the aggregated feature vector;

[0022] Normalize the aggregated feature vector using a standard division normalization function to obtain a normalized feature vector;

[0023] The normalized feature vector is used to calibrate the original feature map.

[0024] In the above scheme, the universal pathology diagnosis model is constructed based on the ConvNeXt V2 network architecture and trained to obtain the trained universal pathology diagnosis model, and further includes:

[0025] For the training image data in the model training set, a random mask is input according to the random mask strategy to generate a training signal;

[0026] Performing data processing on the training image data according to the random mask, including at least deleting, enhancing and / or cropping;

[0027] The processed training image data is pre-trained using FCMAE and fine-tuned based on imageNet-22K or imageNet-1K to complete the pre-training.

[0028] In the above scheme, the method further comprises:

[0029] In response to the user's information acquisition request, the medical information system identifies the user identity information in the information acquisition request;

[0030] Based on the user's identity information, verification is performed with the encrypted identity information pre-stored in the medical information system;

[0031] When the verification is passed, the patient data set corresponding to the corresponding target patient is determined according to the user identity information, and image data with the same feature type as the patient data set is extracted from the medical information system as a historical image data set for training the universal pathology diagnosis model;

[0032] When the verification fails, the current information acquisition process is stopped and a prompt message is issued to prompt the user to re-authenticate.

[0033] In the above scheme, the verification based on the user identity information and the encrypted identity information pre-stored in the medical information system further includes:

[0034] Obtain the first verification password in the user identity information in advance, and obtain the initial key osk through the KGC of the medical information system;

[0035] Using the Argon2 algorithm to extract characteristic parameters corresponding to the first verification password;

[0036] Based on the initial key osk and the characteristic parameters of the first verification password, a first key fsk is generated by a hash function H based on a CRC32 encryption algorithm;

[0037] Adjust the key based on the current time interval corresponding to the current key

[0038]

[0039] R i+1 =f(K i+1 ), i ≥ 0

[0040] Among them, K i is the key for the ith time; K i+1 is the key for the i+1th time; R i+1 is the time interval corresponding to the key of the i+1th time; R0 is the time interval corresponding to the initial key, which is a random number between 1 and 240;

[0041] The second verification password is extracted from the currently obtained information acquisition request, and verification is performed based on the second verification password and the hash function H with the current key.

[0042] According to another aspect of the present invention, a weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis device is provided, comprising: an image acquisition module, a model construction module, a pathology analysis module and a report generation module; wherein,

[0043] The image acquisition module is used to acquire digital pathological slice image data of a target patient through a medical information system as target image data;

[0044] The model building module is used to build a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and perform training to obtain a trained universal pathology diagnosis model;

[0045] The pathological analysis module is used to input the target image data into the trained universal pathological diagnosis model, perform pathological analysis using the model, and obtain corresponding analysis results;

[0046] The report generating module is used to generate a pathology analysis report for a target patient based on the pathology analysis result.

[0047] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0048] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned weakly supervised learning large kernel convolution ConvNeXt V2 model general pathology diagnosis method.

[0049] According to another aspect of the present invention, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the general pathology diagnosis method of the weakly supervised learning large kernel convolution ConvNeXt V2 model as described above.

[0050] According to the technical solution provided by the present invention, digital pathological slice image data for a target patient is obtained through a medical information system as target image data; a universal pathological diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathological diagnosis model; the target image data is input into the trained universal pathological diagnosis model, and the model is used to perform pathological analysis to obtain corresponding analysis results; a pathological analysis report for the target patient is generated based on the pathological analysis results. By determining the identity information of the target patient from the information acquisition request, matching the corresponding data set from the medical information system, and then extracting and acquiring the target image, the pathological image corresponding to the target patient is accurately screened out; based on the ConvNeXt V2 network architecture, a universal pathological diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathological diagnosis model; the target image data is input into the trained universal pathological diagnosis model, and the model is used to perform pathological analysis to obtain corresponding analysis results; based on the pathological analysis results, a pathological analysis report for the target patient is generated. V2 fully convolutional network architecture, build a universal pathology diagnosis model, and set the FCMAE fully convolutional masked autoencoder framework and GRN (global response normalization) layer in it. The GRN layer performs global feature aggregation, feature normalization and feature calibration on the feature map on the channel, completes the normalization processing without the need for additional parameters, and performs a large number of calculations on batches of any size, enhancing the feature competition between channels; through the random mask strategy of the FCMAE autoencoder, the missing part of the data is predicted based on the remaining context to generate high-quality feature representation, which enhances the robustness and generalization ability of the model and improves the capture of different pathological features. The capture capability enables the general pathology diagnosis model to not only intelligently analyze digital pathology slices of more than a dozen organs, but also be applied to medical tasks such as image segmentation and classification, expanding the application scenarios of the model, thereby assisting doctors in completing treatment, greatly reducing the workload of doctors, and improving the overall efficiency and accuracy of the diagnosis and treatment process; by using the DINO self-supervised learning paradigm to pre-train the processed training image data based on FCMAE, the data set is more accurately classified and detected, further improving the generalization ability and preventing overfitting, realizing the automation of the entire process from data preprocessing to feature extraction and model training, and improving the learning efficiency and accuracy of the model. In addition, when obtaining the target image data and the corresponding historical image data set, based on the verification password in the user identity information in the information acquisition request, and the key pre-stored in the system and updated and adjusted according to the time interval, it is ensured that only medical personnel or patients and their families with relevant permissions can obtain relevant pathology information, greatly improving the security of the information acquisition process and protecting the privacy information of patients.

[0051] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 A schematic flow chart of a general pathology diagnosis method using a weakly supervised learning large kernel convolution ConvNeXt V2 model according to an embodiment of the present invention is shown;

[0055] Figure 2 A schematic diagram showing a process of constructing and training a universal pathology diagnosis model according to an embodiment of the present invention;

[0056] Figure 3 A schematic flow chart of a method for verifying and acquiring pathological data based on encrypted identity information adjusted over time according to an embodiment of the present invention is shown;

[0057] Figure 4 A structural block diagram of a general pathology diagnosis device of a weakly supervised learning large kernel convolution ConvNeXt V2 model according to an embodiment of the present invention is shown;

[0058] Figure 5 A schematic structural diagram of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0060] Figure 1 A schematic flow chart of a weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method according to an embodiment of the present invention is shown, and the method comprises the following steps:

[0061] Step S101, obtaining digital pathological slice image data of a target patient through a medical information system as target image data.

[0062] Specifically, the step of obtaining digital pathological slice image data of a target patient as target image data through a medical information system further includes:

[0063] Obtaining an information acquisition request, and extracting identity information of a target patient from the information acquisition request;

[0064] Based on the identity information of the target patient, matching is performed from the medical information system to determine the patient data set corresponding to the target patient;

[0065] Based on the information acquisition request, the corresponding digital pathology slice image data is extracted from the patient data set as the target image data.

[0066] Step S102, constructing a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and training it to obtain a trained universal pathology diagnosis model.

[0067] Step S103, inputting the target image data into the trained universal pathology diagnosis model, performing pathology analysis using the model, and obtaining corresponding analysis results.

[0068] Step S104: Generate a pathology analysis report for the target patient based on the pathology analysis result.

[0069] According to the weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method provided in this embodiment, digital pathology slice image data for the target patient is obtained as target image data through the medical information system; a universal pathology diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathology diagnosis model; the target image data is input into the trained universal pathology diagnosis model, and the model is used to perform pathology analysis to obtain corresponding analysis results; based on the pathology analysis results, a pathology analysis report for the target patient is generated. The general pathology diagnosis method of the weakly supervised learning large kernel convolution ConvNeXt V2 model provided in this embodiment determines the identity information of the target patient from the information acquisition request, matches the corresponding data set from the medical information system, and then extracts the target image, and accurately screens out the pathology image corresponding to the target patient; based on the ConvNeXt V2 full convolutional network architecture, a general pathology diagnosis model is constructed to enhance the robustness and generalization ability of the model, and improve the ability to capture different pathological features, so that the general pathology diagnosis model can not only intelligently analyze the digital pathology sections of more than a dozen organs, but also expand the application scenarios of the model, thereby assisting doctors to complete treatment, greatly reducing the workload of doctors, and improving the overall efficiency and accuracy of the diagnosis and treatment process.

[0070] Figure 2 A schematic diagram showing a process of constructing and training a universal pathology diagnosis model according to an embodiment of the present invention;

[0071] like Figure 2 As shown, the method comprises the following steps:

[0072] Step S201, based on the ConvNeXt V2 fully convolutional network architecture, a universal pathology diagnosis model is constructed; wherein, a FCMAE fully convolutional masked autoencoder framework is set in the ConvNeXt V2 fully convolutional network architecture.

[0073] Specifically, a GRN (Global Response Normalization) layer is set in the ConvNeXt V2 fully convolutional network architecture to perform global feature aggregation, feature normalization, and feature calibration;

[0074] Use the L2 norm to aggregate the feature maps on each channel in the model to obtain the aggregated feature vector;

[0075] Normalize the aggregated feature vector using a standard division normalization function to obtain a normalized feature vector;

[0076] The normalized feature vector is used to calibrate the original feature map.

[0077] Preferably, the GRN layer is arranged after the GELU (Gaussian Error Linear Units) layer.

[0078] Step S202: Obtain a historical image data set as a model training set through a medical information system.

[0079] Step S203, using the DINO self-supervised learning paradigm, the universal pathology diagnosis model is trained based on the model training set to obtain a trained universal pathology diagnosis model.

[0080] For the training image data in the model training set, a random mask is input according to the random mask strategy to generate a training signal;

[0081] Performing data processing on the training image data according to the random mask, including at least deleting, enhancing and / or cropping;

[0082] The processed training image data is pre-trained using FCMAE and fine-tuned based on imageNet-22K or imageNet-1K to complete the pre-training.

[0083] Furthermore, the mask rate in the random mask strategy can be 0.6. For the training image data of the initial input autoencoder, 60% of the patches are randomly removed, and the minimum data augmentation is used to crop by random resizing. Based on the hierarchical design of the convolutional model, the features are downsampled at different stages, and the mask is generated in the final stage, and recursively upsampled until the optimal resolution is reached.

[0084] The DINO (Distributed Instance-aware NOde-based Object detection) self-supervised learning paradigm increases the diversity of representation by allowing the student network to focus on different parts of the training image data at different scales, thereby preventing overfitting through knowledge distillation and enabling the model to achieve better generalization.

[0085] According to the above method, a universal pathology diagnosis model can be constructed based on the ConvNeXt V2 fully convolutional network architecture, and an FCMAE fully convolutional masked autoencoder framework and a GRN (global response normalization) layer are set in it. The GRN layer performs global feature aggregation, feature normalization and feature calibration on the feature map on the channel, completes the normalization processing without the need for additional parameters, and performs a large amount of calculations on batches of any size, thereby enhancing the feature competition between channels; through the random masking strategy of the FCMAE autoencoder, the missing part of the data is predicted based on the remaining context to generate high-quality feature representations, thereby enhancing the robustness and generalization ability of the model, and improving the ability to capture different pathological features, so that the universal pathology diagnosis model can not only intelligently analyze ten Digital pathological sections of several organs can also be applied to medical tasks such as image segmentation and classification, expanding the application scenarios of the model to assist doctors in completing treatment, greatly reducing the workload of doctors and improving the overall efficiency and accuracy of the diagnosis and treatment process; by using the DINO self-supervised learning paradigm to pre-train the processed training image data based on FCMAE, the model can use a large amount of unlabeled data for learning, reducing the dependence on a large amount of high-quality labeled data, reducing the cost of data labeling, and more accurately classifying and detecting the data set, further improving the generalization ability and preventing overfitting, realizing the automation of the entire process from data preprocessing to feature extraction and model training, and improving the learning efficiency and accuracy of the model.

[0086] Figure 3 A schematic flow chart of a method for verifying and acquiring pathological data based on encrypted identity information adjusted over time according to an embodiment of the present invention is shown;

[0087] like Figure 3 As shown, the method comprises the following steps:

[0088] Step S301, in response to a user's information acquisition request, the medical information system identifies the user identity information in the information acquisition request.

[0089] Step S302, based on the user identity information, verification is performed with the encrypted identity information pre-stored in the medical information system.

[0090] Specifically, when the verification passes, step S303 is executed; when the verification fails, step S304 is executed.

[0091] The encrypted identity information pre-stored in the medical information system is the key, which is generated as follows:

[0092] Obtain the first verification password in the user identity information in advance, and obtain the initial key osk through the KGC of the medical information system;

[0093] Using the Argon2 algorithm to extract characteristic parameters corresponding to the first verification password;

[0094] Preferably, the characteristic parameter may be password entropy;

[0095] Based on the initial key osk and the characteristic parameters of the first verification password, a first key fsk is generated by a hash function H based on a CRC32 encryption algorithm;

[0096] Preferably, the characteristic parameter of the first verification password is the same as the number of bytes of the initial key osk;

[0097] Adjust the key based on the current time interval corresponding to the current key

[0098]

[0099] R i+1 =f(K i+1 ), i ≥ 0

[0100] Among them, K i is the key for the ith time; K i+1 is the key for the i+1th time; R i+1 is the time interval corresponding to the key of the i+1th time; R0 is the time interval corresponding to the initial key, which is a random number between 1 and 240;

[0101] The second verification password is extracted from the currently obtained information acquisition request, and verification is performed based on the second verification password and the hash function H with the current key.

[0102] Step S303, determining the patient data set corresponding to the corresponding target patient according to the user identity information, and extracting image data with the same feature type as the patient data set from the medical information system as a historical image data set for training the universal pathology diagnosis model.

[0103] Step S304, stop the current information acquisition process, and send a prompt message to prompt the user to re-authenticate.

[0104] According to the above method, when acquiring the target image data and the corresponding historical image data set, verification can be performed based on the verification password in the user identity information in the information acquisition request and the key pre-stored in the system and updated and adjusted according to time intervals, thereby ensuring that only medical personnel or patients and their families with relevant permissions can obtain relevant pathological information, greatly improving the security of the information acquisition process and protecting the patient's privacy information.

[0105] Figure 4 FIG. 4 shows a structural block diagram of a general pathology diagnosis device based on a weakly supervised learning large kernel convolution ConvNeXt V2 model according to an embodiment of the present invention. Figure 4 As shown, the system includes: an image acquisition module 401, a model building module 402, a pathology analysis module 403 and a report generation module 404; wherein,

[0106] The image acquisition module 401 is used to acquire digital pathological slice image data of a target patient through a medical information system as target image data.

[0107] Specifically, the image acquisition module 401 is further used to:

[0108] Obtaining an information acquisition request, and extracting identity information of a target patient from the information acquisition request;

[0109] Based on the identity information of the target patient, matching is performed from the medical information system to determine the patient data set corresponding to the target patient;

[0110] Based on the information acquisition request, the corresponding digital pathology slice image data is extracted from the patient data set as the target image data.

[0111] Specifically, the image acquisition module 401 is further used to:

[0112] In response to the user's information acquisition request, the medical information system identifies the user identity information in the information acquisition request;

[0113] Based on the user's identity information, verification is performed with the encrypted identity information pre-stored in the medical information system;

[0114] When the verification is passed, the patient data set corresponding to the corresponding target patient is determined according to the user identity information, and image data with the same feature type as the patient data set is extracted from the medical information system as a historical image data set for training the universal pathology diagnosis model;

[0115] When the verification fails, the current information acquisition process is stopped and a prompt message is issued to prompt the user to re-authenticate.

[0116] Wherein, the first verification password in the user identity information is obtained in advance, and the initial key osk is obtained through the KGC of the medical information system;

[0117] Using the Argon2 algorithm to extract characteristic parameters corresponding to the first verification password;

[0118] Based on the initial key osk and the characteristic parameters of the first verification password, a first key fsk is generated by a hash function H based on a CRC32 encryption algorithm;

[0119] Adjust the key based on the current time interval corresponding to the current key

[0120]

[0121] R i+1 =f(K i+1 ), i ≥ 0

[0122] Among them, K i is the key for the ith time; K i+1 is the key for the i+1th time; R i+1 is the time interval corresponding to the key of the i+1th time; R0 is the time interval corresponding to the initial key, which is a random number between 1 and 240;

[0123] The second verification password is extracted from the currently obtained information acquisition request, and verification is performed based on the second verification password and the hash function H with the current key.

[0124] The model building module 402 is used to build a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and perform training to obtain a trained universal pathology diagnosis model.

[0125] Specifically, the model building module 402 is further used to:

[0126] Based on the ConvNeXt V2 fully convolutional network architecture, a universal pathology diagnosis model is constructed; wherein, the FCMAE fully convolutional masked autoencoder framework is set in the ConvNeXt V2 fully convolutional network architecture;

[0127] Obtain historical image datasets as model training sets through medical information systems;

[0128] The DINO self-supervised learning paradigm is used to train the universal pathology diagnosis model based on the model training set to obtain the trained universal pathology diagnosis model.

[0129] The model building module 402 is further used to:

[0130] In the ConvNeXt V2 fully convolutional network architecture, a GRN layer is set to perform global feature aggregation, feature normalization, and feature calibration;

[0131] Use the L2 norm to aggregate the feature maps on each channel in the model to obtain the aggregated feature vector;

[0132] Normalize the aggregated feature vector using a standard division normalization function to obtain a normalized feature vector;

[0133] The normalized feature vector is used to calibrate the original feature map.

[0134] Preferably, the model building module 402 is further used to:

[0135] For the training image data in the model training set, a random mask is input according to the random mask strategy to generate a training signal;

[0136] Performing data processing on the training image data according to the random mask, including at least deleting, enhancing and / or cropping;

[0137] The processed training image data is pre-trained using FCMAE and fine-tuned based on imageNet-22K or imageNet-1K to complete the pre-training.

[0138] The pathological analysis module 403 is used to input the target image data into the trained universal pathological diagnosis model, perform pathological analysis using the model, and obtain corresponding analysis results.

[0139] The report generating module 404 is used to generate a pathology analysis report for a target patient based on the pathology analysis result.

[0140] The general pathology diagnosis device of the weakly supervised learning large kernel convolution ConvNeXt V2 model provided in this embodiment includes: an image acquisition module, a model construction module, a pathology analysis module and a report generation module; wherein the image acquisition module is used to acquire digital pathology slice image data for a target patient through a medical information system as target image data; the model construction module is used to build a general pathology diagnosis model based on the ConvNeXt V2 network architecture and train it to obtain a trained general pathology diagnosis model; the pathology analysis module is used to input the target image data into the trained general pathology diagnosis model, use the model to perform pathology analysis, and obtain the corresponding analysis results; the report generation module is used to generate a pathology analysis report for the target patient based on the pathology analysis results. The general pathology diagnosis device of the weakly supervised learning large kernel convolution ConvNeXt V2 model provided in this embodiment determines the identity information of the target patient from the information acquisition request, matches the corresponding data set from the medical information system, and then extracts and acquires the target image, accurately screening out the pathology image corresponding to the target patient; based on ConvNeXt V2 fully convolutional network architecture, build a universal pathology diagnosis model, and set the FCMAE fully convolutional masked autoencoder framework and GRN (global response normalization) layer in it. The GRN layer performs global feature aggregation, feature normalization and feature calibration on the feature map on the channel, completes the normalization processing without the need for additional parameters, and performs a large number of calculations on batches of any size, enhancing the feature competition between channels; through the random mask strategy of the FCMAE autoencoder, the missing part of the data is predicted based on the remaining context to generate high-quality feature representation, which enhances the robustness and generalization ability of the model and improves the capture of different pathological features. The capture capability enables the general pathology diagnosis model to not only intelligently analyze digital pathology slices of more than a dozen organs, but also be applied to medical tasks such as image segmentation and classification, expanding the application scenarios of the model, thereby assisting doctors in completing treatment, greatly reducing the workload of doctors, and improving the overall efficiency and accuracy of the diagnosis and treatment process; by using the DINO self-supervised learning paradigm to pre-train the processed training image data based on FCMAE, the data set is more accurately classified and detected, further improving the generalization ability and preventing overfitting, realizing the automation of the entire process from data preprocessing to feature extraction and model training, and improving the learning efficiency and accuracy of the model. In addition, when obtaining the target image data and the corresponding historical image data set, based on the verification password in the user identity information in the information acquisition request, and the key pre-stored in the system and updated and adjusted according to the time interval, it is ensured that only medical personnel or patients and their families with relevant permissions can obtain relevant pathology information, greatly improving the security of the information acquisition process and protecting the privacy information of patients.

[0141] The present invention also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute the weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method in any of the above method embodiments.

[0142] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0143] like Figure 5 As shown, the computing device may include: a processor (processor) 502 , a communications interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0144] in:

[0145] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0146] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0147] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned embodiment of the weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method.

[0148] Specifically, the program 510 may include program codes, and the program codes include computer operation instructions.

[0149] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0150] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0151] Program 510 can be specifically used to enable processor 502 to execute a weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method in any of the above-mentioned method embodiments. The specific implementation of each step in program 510 can refer to the corresponding descriptions in the corresponding steps and units in the above-mentioned weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described equipment and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0152] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0153] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0154] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0155] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0156] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0157] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis method, including: Acquire digital pathological slice image data of a target patient as target image data through a medical information system; A universal pathology diagnosis model is constructed and trained based on the ConvNeXt V2 network architecture to obtain a trained universal pathology diagnosis model; Input the target image data into the trained universal pathology diagnosis model, use the model to perform pathology analysis, and obtain the corresponding analysis results; Based on the pathological analysis results, a pathological analysis report for the target patient is generated.

2. The method according to claim 1, characterized in that The step of obtaining digital pathological slice image data of a target patient as target image data through a medical information system further includes: Obtaining an information acquisition request, and extracting identity information of a target patient from the information acquisition request; Based on the identity information of the target patient, matching is performed from the medical information system to determine the patient data set corresponding to the target patient; Based on the information acquisition request, the corresponding digital pathology slice image data is extracted from the patient data set as the target image data.

3. The method according to claim 1, characterized in that The method further comprises: constructing a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and training the model to obtain the trained universal pathology diagnosis model; Based on the ConvNeXt V2 fully convolutional network architecture, a universal pathology diagnosis model is constructed; wherein, the FCMAE fully convolutional masked autoencoder framework is set in the ConvNeXt V2 fully convolutional network architecture; Obtain historical image datasets as model training sets through medical information systems; The DINO self-supervised learning paradigm is used to train the universal pathology diagnosis model based on the model training set to obtain the trained universal pathology diagnosis model.

4. The method according to claim 3, characterized in that The construction of a universal pathology diagnosis model based on the ConvNeXt V2 fully convolutional network architecture further includes: In the ConvNeXt V2 fully convolutional network architecture, a GRN layer is set to perform global feature aggregation, feature normalization, and feature calibration; Use the L2 norm to aggregate the feature maps on each channel in the model to obtain the aggregated feature vector; Normalize the aggregated feature vector using a standard division normalization function to obtain a normalized feature vector; The normalized feature vector is used to calibrate the original feature map.

5. The method according to claim 1, characterized in that The method of constructing a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and training the model to obtain the trained universal pathology diagnosis model also includes: For the training image data in the model training set, a random mask is input according to the random mask strategy to generate a training signal; Performing data processing on the training image data according to the random mask, including at least deleting, enhancing and / or cropping; The processed training image data is pre-trained using FCMAE and fine-tuned on imageNet-22K and / or imageNet-1K to complete the pre-training.

6. The method according to claim 1, characterized in that The method further comprises: In response to the user's information acquisition request, the medical information system identifies the user identity information in the information acquisition request; Based on the user's identity information, verification is performed with the encrypted identity information pre-stored in the medical information system; When the verification is passed, the patient data set corresponding to the corresponding target patient is determined according to the user identity information, and image data with the same feature type as the patient data set is extracted from the medical information system as a historical image data set for training the universal pathology diagnosis model; When the verification fails, the current information acquisition process is stopped and a prompt message is issued to prompt the user to re-authenticate.

7. The method according to claim 6, characterized in that The verification based on the user identity information and the encrypted identity information pre-stored in the medical information system further includes: Obtain the first verification password in the user identity information in advance, and obtain the initial key osk through the KGC of the medical information system; Using the Argon2 algorithm to extract characteristic parameters corresponding to the first verification password; Based on the initial key osk and the characteristic parameters of the first verification password, a first key fsk is generated by a hash function H based on a CRC32 encryption algorithm; Adjust the key based on the current time interval corresponding to the current key: R i+1 =f(K i+1 ),i≧0 Among them, K i is the key for the ith time; K i+1 is the key for the i+1th time; R i+1 is the time interval corresponding to the key of the i+1th time; R0 is the time interval corresponding to the initial key, which is a random number between 1 and 240; The second verification password is extracted from the currently obtained information acquisition request, and verification is performed based on the second verification password and the hash function H with the current key.

8. A weakly supervised learning large kernel convolution ConvNeXt V2 model universal pathology diagnosis device, comprising: Image acquisition module, model building module, pathology analysis module and report generation module; among them, The image acquisition module is used to acquire digital pathological slice image data of a target patient through a medical information system as target image data; The model building module is used to build a universal pathology diagnosis model based on the ConvNeXt V2 network architecture and perform training to obtain a trained universal pathology diagnosis model; The pathological analysis module is used to input the target image data into the trained universal pathological diagnosis model, perform pathological analysis using the model, and obtain corresponding analysis results; The report generating module is used to generate a pathology analysis report for a target patient based on the pathology analysis result.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to a general pathology diagnosis method of a weakly supervised learning large kernel convolution ConvNeXt V2 model as described in any one of claims 1-7.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, wherein the executable instruction enables a processor to perform operations corresponding to a general pathology diagnosis method of a weakly supervised learning large kernel convolution ConvNeXt V2 model as described in any one of claims 1 to 7.

Images