Renal biopsy electron microscope image recognition method, device, equipment, medium and product
By using a pre-trained deep residual network model to extract and identify renal biopsy electron microscope images, the problems of low efficiency and poor accuracy caused by relying on artificial judgment of electron dense deposition position in the prior art are solved, and efficient and accurate electron dense deposition position recognition is achieved.
Patent Information
- Application Number
- CN202510204885.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-24
- Publication Date
- 2025-05-13
AI Technical Summary
In the prior art, the judgment of the deposition location of electron dense substances depends on the doctor's visual examination, which is inefficient and poorly accurate, and is greatly affected by the doctor's professional background, experience level and diagnostic standards.
The pre-trained deep residual network model is used to extract and identify the renal biopsy electron microscope images to identify the presence of electron dense material deposition in the renal structural area.
By reducing artificial dependence, the efficiency and accuracy of electron dense deposition position recognition is improved, and the electron dense deposition situation in the renal structural area can be quickly and accurately identified.
Smart Images
Figure CN119992547A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image recognition technology, and in particular to a method, device, equipment, medium and product for renal biopsy electron microscope image recognition. Background Art
[0002] Chronic kidney disease (CKD) is a major global health problem, and glomerular disease is the main cause of end-stage renal disease. To diagnose chronic kidney disease, kidney biopsy is performed based on electron microscopy (EM) analysis to evaluate kidney conditions. In EM evaluation, it is necessary to determine whether electron-dense deposits are present in each renal structural area.
[0003] Currently, the evaluation of electron-dense objects under electron microscopy mainly relies on visual inspection by doctors. This process is not only time-consuming, but also has poor accuracy in diagnostic results due to differences in professional backgrounds, experience levels and diagnostic criteria among different doctors. Summary of the invention
[0004] The embodiments of the present invention provide a method, device, equipment, medium and product for renal biopsy electron microscopy image recognition, which can identify the presence of electron-dense material deposits in renal structural areas through a pre-trained electron-dense material deposit position recognition model, reduce manual dependence on electron-dense material position recognition, and improve recognition efficiency and accuracy.
[0005] In a first aspect, an embodiment of the present invention provides a method for renal biopsy electron microscope image recognition, the method comprising:
[0006] Acquire electron microscope images of renal biopsy to be identified;
[0007] Inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified;
[0008] Among them, the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is the recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
[0009] In a second aspect, an embodiment of the present invention provides a renal biopsy electron microscope image recognition device, the device comprising:
[0010] An image acquisition module, used for acquiring an electron microscope image of a renal biopsy to be identified;
[0011] A deposition position recognition module is used to input the electron microscope image of the renal biopsy to be recognized into a pre-trained electron-dense object deposition position recognition model to obtain an electron-dense object deposition position recognition result of the renal biopsy electron microscope image to be recognized;
[0012] Among them, the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is the recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
[0013] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:
[0014] one or more processors;
[0015] A memory for storing one or more programs;
[0016] When the one or more programs are executed by one or more processors, the one or more processors implement the renal biopsy electron microscope image recognition method provided by any embodiment of the present invention.
[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for renal biopsy electron microscopy image recognition as provided in any embodiment of the present invention.
[0018] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the renal biopsy electron microscopy image recognition method provided in any embodiment of the present invention.
[0019] The embodiments of the above invention have the following advantages or beneficial effects:
[0020] The embodiment of the present invention obtains an electron microscopic image of a renal biopsy to be identified; inputs the electron microscopic image of the renal biopsy to be identified into a pre-trained electron-dense material deposit position recognition model to obtain an electron-dense material deposit position recognition result of the renal biopsy electron microscopic image to be identified; wherein the electron-dense material deposit position recognition model is a deep residual network model, and the electron-dense material deposit position recognition result is an identification result of the presence of electron-dense material deposits in each renal structural region of the renal biopsy electron microscopic image to be identified. The technical solution of the embodiment of the present invention solves the problem of low efficiency in the current reliance on manual judgment for electron-dense material deposit position judgment, and can identify the presence of electron-dense material deposits in renal structural regions through a pre-trained electron-dense material deposit position recognition model, thereby reducing manual reliance on electron-dense material position recognition and improving recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1It is a flow chart of a method for renal biopsy electron microscope image recognition provided by an embodiment of the present invention;
[0022] Figure 2 It is a flow chart of a method for renal biopsy electron microscope image recognition provided by an embodiment of the present invention;
[0023] Figure 3 This is a schematic diagram of a renal biopsy electron microscope image recognition page provided by an embodiment of the present invention;
[0024] Figure 4 It is a flow chart of a method for renal biopsy electron microscope image recognition provided by an embodiment of the present invention;
[0025] Figure 5 is a schematic diagram of a renal biopsy electron microscope sample image provided by an embodiment of the present invention;
[0026] Figure 6 It is a structural schematic diagram of a renal biopsy electron microscope image recognition device provided by an embodiment of the present invention;
[0027] Figure 7 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0029] Figure 1 The present invention provides a flowchart of a method for renal biopsy electron microscope image recognition, which can be applied to the scenario of renal biopsy electron microscope image recognition. The method can be performed by a renal biopsy electron microscope image recognition device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.
[0030] like Figure 1 As shown, the renal biopsy electron microscope image recognition method of this embodiment includes the following steps:
[0031] S110, obtaining an electron microscope image of a renal biopsy to be identified.
[0032] The renal biopsy electron microscope image can be an image taken by a transmission electron microscope (TEM) that can be used to observe the location of electron-dense material deposition. An online platform for the location of electron-dense material can be established, and a front-end page based on Html+Javascript can be provided to obtain the renal biopsy electron microscope image to be identified uploaded by the user on the front-end page.
[0033] S120, inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified.
[0034] Before identifying the renal biopsy electron microscopy image to be identified by using the pre-trained electron-dense material deposition location identification model, the renal biopsy electron microscopy image to be identified can be preprocessed, such as lowering the resolution, so as to reduce interference information and improve the stability of the model prediction process.
[0035] The electron-dense material deposition location recognition model is a deep residual network model, such as deep residual network 18ResNet18, residual network Resent and Inception-resnetv2 network models. The electron-dense material deposition location recognition model can also include VGG (Visual Geometry Group), swintransformer and visiontransformer models. This embodiment preferably uses the ResNet18 model. The ResNet18 model has good performance and generalization ability, can effectively handle image recognition tasks, has a small amount of calculation and parameters, and is easy to deploy and apply. In this embodiment, the pre-trained electron-dense material deposition location recognition model can be packaged through a model packaging tool such as the torch-model-archiver tool. The packaged model can provide an API call through a model service tool such as the torchserve tool to identify the acquired renal biopsy electron microscope image to be identified.
[0036] The electron-dense material deposition position identification result is the electron-dense material deposition presence identification result of each renal structural region of the renal biopsy electron microscope image to be identified, and the identification result of whether the electron-dense material deposition exists in each renal structural region. The renal structural regions include the mesangial region, subepithelial region, basement membrane region, and subendothelial region, so that even the electron microscope images to be identified of patients with non-immune complex-mediated diseases can be identified.
[0037] The pre-trained electron-dense deposit location recognition model has powerful feature extraction capabilities. It can extract features from the renal biopsy electron microscopy images to be identified through convolutional layers, and avoid feature loss through residual connections, ensuring that the feature information of each part of the image is retained.
[0038] The pre-trained electron-dense material deposition location recognition model can grasp various characteristics of the electron-dense material deposition location through sufficient supervised learning. When identifying the kidney electron microscopy image to be identified, it can associate the characteristics with whether there is electron-dense material deposition in each kidney structure area, thereby obtaining the electron-dense material deposition location recognition result of the kidney biopsy electron microscopy image to be identified.
[0039] The technical solution of this embodiment is to obtain an electron microscopic image of a renal biopsy to be identified; input the electron microscopic image of the renal biopsy to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscopic image to be identified; wherein the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is an identification result of the presence of electron-dense material deposition in each renal structural region of the renal biopsy electron microscopic image to be identified. The technical solution of the embodiment of the present invention solves the problem of low efficiency in the current reliance on manual judgment for the determination of the location of electron-dense material deposition, and can identify the presence of electron-dense material deposition in the renal structural region through a pre-trained electron-dense material deposition position recognition model, thereby reducing the manual reliance on the identification of the location of electron-dense material and improving the efficiency and accuracy of the identification.
[0040] Figure 2 A flowchart of a method for renal biopsy electron microscopy image recognition provided in an embodiment of the present invention, this embodiment and the method for renal biopsy electron microscopy image recognition in the above embodiment belong to the same inventive concept, and further describes the process of determining the electron dense material deposition location recognition result. The method can be executed by a renal biopsy electron microscopy image recognition device, which can be implemented by software and / or hardware, and integrated into a computer device with application development function.
[0041] like Figure 2 As shown, the renal biopsy electron microscope image recognition method of this embodiment includes the following steps:
[0042] S210, obtaining an electron microscope image of a renal biopsy to be identified.
[0043] S220, inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model, performing feature extraction on the renal biopsy electron microscope image to be identified through a feature extraction module of the pre-trained electron-dense material deposition position recognition model, and obtaining a feature extraction module processing result.
[0044] In this embodiment, the pre-trained electron-dense material deposition location recognition model is a ResNet18 model. The feature extraction module can be composed of a convolution layer with a larger convolution kernel, which can perform preliminary feature extraction on the input renal biopsy electron microscope image to be identified, such as detecting edges, textures and other features in the image, and obtaining multiple feature maps reflecting the feature information of different positions of the renal biopsy electron microscope image to be identified. These feature maps are the processing results of the feature extraction module.
[0045] S230, performing feature extraction and feature fusion on the processing result of the feature extraction module through the residual module of the pre-trained electron-dense material deposition position recognition model to obtain the processing result of the residual module.
[0046] The residual module contains multiple residual blocks, each of which can be composed of two 3x3 convolutional layers. These convolutional layers will further extract features from the processing results of the feature extraction module, and fuse the features extracted by the residual blocks through the residual connection of the residual module to obtain the processing results of the residual module.
[0047] In an optional embodiment, feature extraction and feature fusion are performed on the processing result of the feature extraction module through the residual module of a pre-trained electron-dense material deposition position recognition model to obtain the residual module processing result. Feature extraction can be performed on the processing result of the feature extraction module through the convolution layer of the residual module of the pre-trained electron-dense material deposition position recognition model to obtain a preliminary feature extraction result; the feature extraction module processing result and the preliminary feature extraction result are added through the residual connection mechanism of the residual module to obtain the residual module processing result.
[0048] Through the residual connection mechanism of the residual module, multiple residual blocks of the residual module are skipped, and the processing results of the feature extraction module and the preliminary feature extraction results are added, which helps to solve the gradient vanishing and degradation problems in the deep network, so that the electron-dense material deposition location recognition model can more effectively learn the features of different parts of the image.
[0049] S240, performing dimensionality reduction processing on the residual module processing result through the global average pooling module of the pre-trained electron-dense material deposition position recognition model to obtain the global average pooling module processing result.
[0050] The global average pooling module calculates the average value of all elements in each pooling window as the output value of the area, realizes the dimensionality reduction processing of the residual module processing result, and obtains the global average pooling module processing result. The global average pooling module processing result can be a feature vector containing global features.
[0051] S250, performing feature mapping on the processing result of the global average pooling module through the classification module of the pre-trained electron-dense material deposition position recognition model to obtain the electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be recognized.
[0052] In this embodiment, the model task of the pre-trained electron-dense material deposition location recognition model is a multi-label classification task. The classification module maps the processing results of the global average pooling module to the dimensional space corresponding to the renal structure area, and outputs the probability distribution of each category through the activation function, that is, the probability of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
[0053] like Figure 3As shown in the figure, the user uploads the electron microscope image of the kidney to be identified. After identification by the pre-trained electron-dense deposition location recognition model, the probability of the presence of electron-dense deposition in different renal structural regions, namely, the mesangial region, subepithelial region, intramembranous region, and subendothelial region, is obtained. Figure 3 The values are 0.041, 1, 1 and 0.004 respectively, and the presence or absence of the value is determined by giving a preset threshold. Figure 3 The judgment threshold of the corresponding mesangial area is 0.505, and 0.041 is less than 0.505, so there is no electron-dense deposition in the mesangial area in the electron-dense deposition position identification result.
[0054] In an optional embodiment, feature mapping is performed on the processing results of the global average pooling module through a classification module of a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified. The linear transformation result can be obtained by performing a linear transformation on the processing results of the global average pooling module through a classification module of a pre-trained electron-dense material deposition position recognition model; the linear transformation result is output as a probability vector through an activation function of the classification module, and the probability vector includes a probability value of the presence of electron-dense material deposition in each renal structural area of the renal biopsy electron microscope image to be identified; according to the numerical relationship between the probability value and a preset threshold, the probability vector is converted into a binary label, and the binary label is used as the electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified.
[0055] The classification module performs a linear transformation on the result of the global average pooling module, that is, the feature vector obtained by the global average pooling module, and maps the feature vector to the dimensional space corresponding to each renal structural region. The linear transformation result is converted into a probability vector through the activation function Sigmoid, and each element in the vector represents the probability value of the electron-dense material deposition in the corresponding renal structural region.
[0056] Each probability value in the probability vector is compared with a preset threshold value, and a value greater than or equal to the threshold value is marked as 1 (deposition exists), and a value less than the threshold value is marked as 0 (no deposition exists), and the obtained binary label is used as the electron dense material deposition location recognition result of the renal biopsy electron microscope image to be identified. Figure 3 The recognition result of the image in is (0,1,1,0), that is, there is no electron-dense deposition in the mesangial area and under the endothelium, and there is electron-dense deposition under the epithelium and in the basement membrane.
[0057] The technical solution of this embodiment is as follows: acquiring an electron microscopic image of a renal biopsy to be identified; inputting the electron microscopic image of the renal biopsy to be identified into a pre-trained electron-dense material deposition position recognition model; performing feature extraction on the renal biopsy electron microscopic image to be identified by a feature extraction module of the pre-trained electron-dense material deposition position recognition model, and obtaining a processing result of the feature extraction module; performing feature extraction and feature fusion on the processing result of the feature extraction module by a residual module of the pre-trained electron-dense material deposition position recognition model, and obtaining a processing result of the residual module; performing dimensionality reduction processing on the processing result of the residual module by a global average pooling module of the pre-trained electron-dense material deposition position recognition model, and obtaining a processing result of the global average pooling module; performing feature mapping on the processing result of the global average pooling module by a classification module of the pre-trained electron-dense material deposition position recognition model, and obtaining an electron-dense material deposition position recognition result of the renal biopsy electron microscopic image to be identified. The technical solution of the embodiment of the present invention solves the problem that the current judgment of the location of electron-dense material deposits relies on low efficiency manual judgment. The presence of electron-dense material deposits in the renal structure area can be identified through a pre-trained electron-dense material deposit location recognition model, thereby reducing manual dependence on electron-dense material location recognition, and further improving recognition efficiency and accuracy through a residual module.
[0058] Figure 4 A flowchart of a method for renal biopsy electron microscopy image recognition provided by an embodiment of the present invention, this embodiment and the method for renal biopsy electron microscopy image recognition in the above embodiment belong to the same inventive concept, and further describes the process of training the electron dense object deposition location recognition model. The method can be executed by a renal biopsy electron microscopy image recognition device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.
[0059] like Figure 4 As shown, the training process of the electron-dense material deposition position recognition model in the renal biopsy electron microscope image recognition method of this embodiment includes the following steps:
[0060] S310, obtaining a renal biopsy electron microscopy sample image and a sample binary label corresponding to the renal biopsy electron microscopy sample image.
[0061] Renal pathological electron microscopy images were selected as sample images for training the electron-dense object location recognition model, that is, renal biopsy electron microscopy sample images, and professional assessors were used to evaluate the electron-dense object deposition sites (mesangial area, subepithelial area, basement membrane, and subendothelial area). If there was electron-dense object deposition, it was 1, and if there was no electron-dense object deposition, it was 0. Figure 5 As shown, the sample binary label corresponding to the renal biopsy electron microscope sample image is obtained as (0, 1, 1, 0).
[0062] When selecting renal biopsy electron microscopy sample images, the magnification of the kidney pathology electron microscopy images collected for diagnosis is selected to be 4000-8000, and the resolution is 2048×2048. In order to meet the training requirements and optimize the use of computing resources, the image resolution can be adjusted from the original 2048×2048 to 700×700. For example, the number of renal biopsy electron microscopy sample images can be 5100.
[0063] S320, based on a cross-validation training method, the initial model for electron-dense object position recognition is trained using renal biopsy electron microscopy sample images and sample binary labels, and when the loss function value meets a preset convergence threshold, an electron-dense object position recognition model is obtained.
[0064] The renal biopsy electron microscopy sample images were divided into training set, test set and validation set. The deep learning model was trained and validated using the training set and test set, and the model recognition results were obtained by testing on the validation set.
[0065] For example, the cross validation is a 10-fold cross validation, and the initial model for electron-dense object position recognition is trained 10 times. When the loss function value meets the preset convergence threshold, the electron-dense object position recognition model is obtained. Among them, the loss function is a binary cross entropy loss function with a logical value, that is, the BCEWithLogitsLoss loss function. BCEWithLogitsLoss can better measure the difference between the model prediction result and the true label, and improve the prediction effect of the model.
[0066] The performance of the model is evaluated through accuracy, precision, recall, F1 score and ROC curve.
[0067] After the electron-dense object location recognition model was trained, the differences in recognition between pathologists and the electron-dense object location recognition model were compared, and 430 electron microscope images were randomly selected from the validation set for diagnostic testing. The pathologists included 2 electron microscope pathologists and 2 renal pathology comprehensive report doctors, and the differences in evaluation between pathologists and the electron-dense object deposition location recognition model were compared. The electron-dense object deposition location recognition model evaluated the electron-dense object deposition site 1010 times faster than pathologists, which can greatly improve the efficiency of identifying the electron-dense object deposition site.
[0068] In an optional embodiment, a first number of training cycles are preset before the training of the electron-dense material deposition position identification model, and the learning rate of each of the first number of training cycles is determined by a linear function, a preset initial learning rate, and a preset target learning rate; a second number of training cycles are preset after the training of the electron-dense material deposition position identification model, and the learning rate of each of the second number of training cycles is determined by a cosine annealing formula, a preset target learning rate, and a preset minimum learning rate; wherein the preset first number is less than the preset second number.
[0069] For example, the first number is preset to 10, and the second number is preset to 90. All the initial models for electron-dense object position recognition participating in the training have undergone 100 training cycles. Specifically, the learning rate is set in a staged manner. In the first 10 training cycles, the parameters such as the preset initial learning rate, the preset target learning rate, the current number of steps, and the number of warm-up steps are substituted into the linear function, so that the learning rate is gradually increased in a linear manner to the preset target learning rate of 0.0001.
[0070] In the remaining 90 training cycles, the preset target learning rate and the preset minimum learning rate are substituted into the cosine annealing formula to determine the learning rate for each preset second number of training cycles. The preset minimum learning rate is the lower limit of the learning rate that can be reached during the annealing process. The preset target learning rate is the maximum learning rate, which is the initial learning rate at the beginning of annealing. The input cosine annealing formula also includes the current number of training steps and the total number of steps. The strategy of adjusting the learning rate by linear increase in the early stage and cosine annealing in the later stage is adopted in stages. The linear increase in the early stage prevents gradient explosion and stabilizes the training. The cosine annealing in the later stage finely adjusts the parameters to improve the convergence accuracy, which can significantly improve the training effect.
[0071] The technical solution of this embodiment is to obtain a renal biopsy electron microscopy sample image and a sample binary label corresponding to the renal biopsy electron microscopy sample image; based on a cross-validation training method, train the electron-dense object position recognition initial model through the renal biopsy electron microscopy sample image and the sample binary label, and obtain the electron-dense object position recognition model when the loss function value meets the preset convergence threshold; wherein the loss function is a binary cross entropy loss function with a logical value. The technical solution of the embodiment of the present invention solves the problem that the current judgment of the electron-dense object deposition position relies on low efficiency of manual judgment, and can fully train the electron-dense object deposition position recognition initial model, and select the optimal model as the electron-dense object position recognition model through the evaluation value, thereby improving the model performance and generalization ability, and improving the efficiency and accuracy of recognition.
[0072] Figure 6 The present invention provides a schematic diagram of the structure of a renal biopsy electron microscope image recognition device, which can be applied to the scene of renal biopsy electron microscope image recognition. The renal biopsy electron microscope image recognition device can be implemented by software and / or hardware, and integrated into a computer terminal device with application development function.
[0073] like Figure 6 As shown, the renal biopsy electron microscope image recognition device includes: an image acquisition module 410 and a deposition position recognition module 420.
[0074] Among them, the image acquisition module 410 is used to acquire the electron microscopic image of the renal biopsy to be identified; the deposition position recognition module 420 is used to input the electron microscopic image of the renal biopsy to be identified into the pre-trained electron-dense material deposition position recognition model to obtain the electron-dense material deposition position recognition result of the renal biopsy electron microscopic image to be identified; wherein the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is the recognition result of the presence of electron-dense material deposition in each renal structural area of the renal biopsy electron microscopic image to be identified.
[0075] The technical solution of this embodiment is to obtain an electron microscopic image of a renal biopsy to be identified; input the electron microscopic image of the renal biopsy to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscopic image to be identified; wherein the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is an identification result of the presence of electron-dense material deposition in each renal structural region of the renal biopsy electron microscopic image to be identified. The technical solution of the embodiment of the present invention solves the problem of low efficiency in the current reliance on manual judgment for the determination of the location of electron-dense material deposition, and can identify the presence of electron-dense material deposition in the renal structural region through a pre-trained electron-dense material deposition position recognition model, thereby reducing the manual reliance on the identification of the location of electron-dense material and improving the efficiency and accuracy of the identification.
[0076] In an optional implementation, the deposition location identification module 420 is specifically used for:
[0077] The renal biopsy electron microscope image to be identified is input into a pre-trained electron-dense material deposition position recognition model, and the feature extraction module of the pre-trained electron-dense material deposition position recognition model is used to extract features of the renal biopsy electron microscope image to be identified, so as to obtain the processing result of the feature extraction module; the residual module of the pre-trained electron-dense material deposition position recognition model is used to extract features and fuse features on the processing result of the feature extraction module, so as to obtain the processing result of the residual module; the global average pooling module of the pre-trained electron-dense material deposition position recognition model is used to perform dimensionality reduction processing on the processing result of the residual module, so as to obtain the processing result of the global average pooling module; the classification module of the pre-trained electron-dense material deposition position recognition model is used to perform feature mapping on the processing result of the global average pooling module, so as to obtain the electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified.
[0078] In an optional implementation, the deposition location identification module 420 is further configured to:
[0079] The convolutional layer of the residual module of the pre-trained electron-dense material deposition location recognition model is used to extract features from the processing results of the feature extraction module to obtain preliminary feature extraction results. The processing results of the feature extraction module and the preliminary feature extraction results are added together through the residual connection mechanism of the residual module to obtain the processing results of the residual module.
[0080] In an optional implementation, the deposition location identification module 420 is further configured to:
[0081] The linear transformation result is obtained by performing a linear transformation on the processing result of the global average pooling module through the classification module of the pre-trained electron-dense material deposition position recognition model; the linear transformation result is output as a probability vector through the activation function of the classification module, and the probability vector includes the probability value of the existence of electron-dense material deposition in each renal structural area of the renal biopsy electron microscopy image to be identified; according to the numerical relationship between the probability value and the preset threshold, the probability vector is converted into a binary label, and the binary label is used as the electron-dense material deposition position recognition result of the renal biopsy electron microscopy image to be identified.
[0082] In an optional embodiment, the device further comprises:
[0083] The model training module is used to obtain renal biopsy electron microscopy sample images and sample binary labels corresponding to the renal biopsy electron microscopy sample images; based on the cross-validation training method, the initial model for electron-dense object position recognition is trained through the renal biopsy electron microscopy sample images and sample binary labels, and when the loss function value meets the preset convergence threshold, the electron-dense object position recognition model is obtained; wherein the loss function is a binary cross entropy loss function with a logical value.
[0084] In an optional implementation, the model training module is further used to:
[0085] A first preset number of training cycles before the training of the electron-dense material deposition position identification model is performed, and the learning rate of each of the first preset number of training cycles is determined by a linear function, a preset initial learning rate, and a preset target learning rate; a second preset number of training cycles after the training of the electron-dense material deposition position identification model is performed, and the learning rate of each of the second preset number of training cycles after the training is performed is determined by a cosine annealing formula, a preset target learning rate, and a preset minimum learning rate; wherein the preset first number is less than the preset second number.
[0086] The renal biopsy electron microscopic image recognition device provided in the embodiment of the present invention can execute the renal biopsy electron microscopic image recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0087] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 7A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.
[0088] like Figure 7 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).
[0089] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0090] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.
[0091] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0092] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.
[0093] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RFID systems, tape drives, and data backup storage systems.
[0094] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, implementing the renal biopsy electron microscope image recognition method provided by the embodiment of the present invention, the method comprising:
[0095] Acquire electron microscope images of renal biopsy to be identified;
[0096] Inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified;
[0097] Among them, the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is the recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
[0098] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for renal biopsy electron microscope image recognition provided by any embodiment of the present invention is implemented. The method comprises:
[0099] Acquire electron microscope images of renal biopsy to be identified;
[0100] Inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified;
[0101] Among them, the electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is the recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
[0102] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0103] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0104] Computer program code for performing the operation of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, Python, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0105] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the renal biopsy electron microscopy image recognition method provided in any embodiment of the present application.
[0106] In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, Python, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0107] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0108] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for renal biopsy electron microscope image recognition, characterized in that: include: Acquire electron microscope images of renal biopsy to be identified; Inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model to obtain an electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be identified; The electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is a recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
2. The method according to claim 1, characterized in that The step of inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position identification model to obtain an electron-dense material deposition position identification result of the renal biopsy electron microscope image to be identified includes: Inputting the renal biopsy electron microscope image to be identified into a pre-trained electron-dense material deposition position recognition model, performing feature extraction on the renal biopsy electron microscope image to be identified through a feature extraction module of the pre-trained electron-dense material deposition position recognition model, and obtaining a feature extraction module processing result; Performing feature extraction and feature fusion on the processing result of the feature extraction module through the residual module of the pre-trained electron-dense material deposition position recognition model to obtain the residual module processing result; Performing dimensionality reduction processing on the processing result of the residual module through the global average pooling module of the pre-trained electron-dense material deposition position recognition model to obtain the processing result of the global average pooling module; The classification module of the pre-trained electron-dense material deposition position recognition model performs feature mapping on the processing result of the global average pooling module to obtain the electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be recognized.
3. The method according to claim 2, characterized in that The method of performing feature extraction and feature fusion on the processing result of the feature extraction module through the residual module of the pre-trained electron dense material deposition position recognition model to obtain the processing result of the residual module includes: Performing feature extraction on the processing result of the feature extraction module through the convolution layer of the residual module of the pre-trained electron-dense material deposition position recognition model to obtain a preliminary feature extraction result; The feature extraction module processing result and the preliminary feature extraction result are added together through the residual connection mechanism of the residual module to obtain the residual module processing result.
4. The method according to claim 2, characterized in that: The classification module of the pre-trained electron-dense material deposition position recognition model performs feature mapping on the processing result of the global average pooling module to obtain the electron-dense material deposition position recognition result of the renal biopsy electron microscope image to be recognized, including: Performing a linear transformation on the processing result of the global average pooling module through the classification module of the pre-trained electron-dense material deposition position recognition model to obtain a linear transformation result; Outputting the linear transformation result as a probability vector through the activation function of the classification module, wherein the probability vector includes a probability value of the presence of electron-dense material deposition in each renal structural region of the renal biopsy electron microscope image to be identified; According to the numerical relationship between the probability value and the preset threshold, the probability vector is converted into a binary label, and the binary label is used as the electron-dense material deposition position identification result of the renal biopsy electron microscope image to be identified.
5. The method according to any one of claims 1 to 4, characterized in that: The training process of the electron dense material deposition position recognition model includes: Acquire a renal biopsy electron microscopy sample image and a sample binary label corresponding to the renal biopsy electron microscopy sample image; Based on a cross-validation training method, the electron-dense object position recognition initial model is trained using the renal biopsy electron microscope sample image and the sample binary label, and when the loss function value meets a preset convergence threshold, the electron-dense object position recognition model is obtained; The loss function is a binary cross entropy loss function with logical values.
6. The method according to claim 5, characterized in that The method also includes: A first preset number of training cycles before training the electron dense material deposition location recognition model, determining a learning rate for each of the first preset number of training cycles by using a linear function, a preset initial learning rate and a preset target learning rate; After a second number of training cycles of the electron dense material deposition position recognition model training, the learning rate of each of the second number of training cycles is determined by a cosine annealing formula, the preset target learning rate, and the preset minimum learning rate; Wherein, the preset first number is smaller than the preset second number.
7. A renal biopsy electron microscope image recognition device, characterized in that: include: An image acquisition module, used for acquiring an electron microscope image of a renal biopsy to be identified; A deposition position recognition module, used for inputting the renal biopsy electron microscope image to be recognized into a pre-trained electron-dense object deposition position recognition model to obtain an electron-dense object deposition position recognition result of the renal biopsy electron microscope image to be recognized; The electron-dense material deposition position recognition model is a deep residual network model, and the electron-dense material deposition position recognition result is a recognition result of the presence of electron-dense material deposition in each renal structure area of the renal biopsy electron microscope image to be identified.
8. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the renal biopsy electron microscope image recognition method as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for renal biopsy electron microscope image recognition as described in any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method for recognizing renal biopsy electron microscopy images as described in any one of claims 1 to 6.