Basilar membrane lesion analysis method and device for renal needle biopsy transmission electron microscope image and readable storage medium

Abstract

The invention provides a basilar membrane lesion analysis method and device for a renal needle biopsy transmission electron microscope image and a readable storage medium, and provides a new method based on RADS-Net, and end-to-end automation from segmentation to thickness measurement is realized in combination with Transform and CNN. Comprising the steps of data preprocessing, segmentation, segmentation result optimization and key area screening, thickness calculation, data storage, result output and the like, and detailed elaboration is conducted on specific methods and algorithms of all the steps. The method can effectively improve the analysis efficiency, the segmentation accuracy and the reliability of pathological diagnosis, and has a wide application prospect.

Description

Technical Field

[0001] The present application relates to the field of medical diagnosis technology, and in particular to a method, device and readable storage medium for analyzing basement membrane lesions in transmission electron microscope images of renal puncture biopsy. Background Art

[0002] The thickness and ultrastructural changes of the glomerular basement membrane (GBM) are of key significance in the diagnosis of kidney diseases. For example, the diagnosis of diseases such as diabetic nephropathy, thin basement membrane nephropathy and Alport syndrome often depends on the accurate analysis of GBM. In the process of pathological diagnosis, high-resolution electron microscopy images are usually needed to observe the morphology and structure of GBM.

[0003] However, the current analysis technology for the glomerular basement membrane has many shortcomings. First, most existing studies focus on a single task, such as segmentation or thickness calculation, and lack a fully automated process that can achieve everything from image segmentation to effective area screening and thickness calculation. This requires a lot of manual intervention in the analysis process, which is not only inefficient, but also prone to human errors, and cannot meet the needs of modern medicine for efficient and accurate diagnosis.

[0004] Secondly, the basement membrane in glomerular electron microscopy images has fuzzy boundaries with surrounding structures (such as endothelial cells, podocytes, and interstitial regions), and the local features are highly similar, which poses a great challenge to accurate segmentation. Traditional segmentation methods are prone to mis-segmentation and incomplete segmentation, which affects subsequent diagnostic results.

[0005] In addition, existing thickness calculation methods usually assume that the image contains GBM areas suitable for measurement, and lack the step of automatically screening valid areas. This results in the inclusion of non-compliant areas (such as mesenchymal areas and electron-dense deposits) in actual measurements, making the measurement results inaccurate and requiring manual confirmation or adjustment, further increasing the workload and error risk.

[0006] Finally, since GBM's high-resolution images require large-scale computing resources, existing methods are often limited by memory when processing high-resolution full images. They can usually only perform small-scale local analysis and cannot fully utilize the global information of the image, thus affecting the accuracy and reliability of the analysis results.

[0007] In summary, the existing technology has many problems in the analysis of glomerular basement membrane, and there is an urgent need for a method that can realize full process automation, improve segmentation accuracy, ensure the effectiveness of regional screening, and optimize thickness calculation to meet the needs of clinical diagnosis and research. Summary of the invention

[0008] The embodiments of the present application provide a method, device and readable storage medium for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy, in order to address the problems that the current technology of glomerular basement membrane analysis technology lacks a fully automated process, has defects such as difficulty in segmentation, manual reliance on measurement area screening, and limited computing resources and efficiency.

[0009] The core technology of this invention is mainly a fully automated glomerular basement membrane analysis method based on RADS-Net, which combines Transformer and CNN to achieve an end-to-end automated process from segmentation to thickness measurement.

[0010] In a first aspect, the present application provides a method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy, the method comprising the following steps:

[0011] S00, taking the electron microscope image as input, generating image blocks adapted to the input requirements of the deep learning model by overlapping and cropping;

[0012] S10, RADS-Net network combining Transformer and CNN architecture is used to segment the glomerular basement membrane and electron-dense deposits in the image blocks;

[0013] S20, generating an uncertainty heat map of the segmentation result by fusing global and local information through the uncertainty assessment module, screening out high-confidence areas, and eliminating areas that do not meet the measurement requirements using a positioning module based on a classification network;

[0014] S30. Extract the centerline skeleton image from the screened high-confidence area segmentation area, use the refinement algorithm in the skeletonization algorithm developed by OpenCV to obtain the accurate skeleton path, use the regional growing algorithm to calculate the vertical thickness and generate the thickness distribution map and the complete maximum skeleton line, and then calculate the thickness value of the centerline skeleton image. After excluding the electron-dense sediment area, use the statistical screening algorithm to select representative measurement results.

[0015] Furthermore, in step S10, the RADS-Net network includes multi-layer feature extraction and a U-Net decoder.

[0016] Furthermore, in step S20, the classification network used by the positioning module is ResNet-34, and the OCR algorithm is used to dynamically crop a suitable field of view according to the image scale, analyze the image region by region, and mark the area suitable for thickness measurement.

[0017] Furthermore, in step S00, the resolution of the electron microscope image is at least 3296×2563 pixels.

[0018] Furthermore, in step S00, the input image is cropped by overlapping to generate an image block of 512×512 pixels to adapt to the input requirements of the deep learning model.

[0019] Furthermore, in step S20, the specific method of the uncertainty assessment module generating an uncertainty heat map of the segmentation result by fusing global and local information is as follows:

[0020] For the input image block, the preliminary segmentation result is obtained through the RADS-Net network;

[0021] Calculate the global features of the segmentation results and the local features of the segmentation results;

[0022] The global features and local features are concatenated to obtain a fused feature vector;

[0023] The fused feature vector is mapped to an uncertainty value through a fully connected layer to generate an uncertainty heat map of the segmentation result, which is used to indicate the confidence of the segmentation result.

[0024] Furthermore, the method further includes step S40, storing all segmentation results and thickness distribution maps in a database in a preset standardized format, and outputting a visualization image and thickness distribution map of the segmented area according to requirements.

[0025] In a second aspect, the present application provides a device for analyzing basement membrane lesions in transmission electron microscope images of renal puncture biopsy, comprising:

[0026] The data preprocessing module takes the electron microscope image as input and generates image blocks that meet the input requirements of the deep learning model through overlapping cropping;

[0027] The segmentation module uses the RADS-Net network that combines the Transformer and CNN architectures to segment the glomerular basement membrane and electron-dense deposits in the image blocks;

[0028] The segmentation result optimization and key area screening module generates an uncertainty heat map of the segmentation result by fusing global and local information through the uncertainty assessment module, screens out high-confidence areas, and uses the positioning module based on the classification network to eliminate areas that do not meet the measurement requirements;

[0029] The thickness calculation module extracts the centerline skeleton image from the screened high-confidence area segmentation area, uses the region growing algorithm to calculate the vertical thickness and generate the thickness distribution map and the complete maximum skeleton line, and then calculates the thickness value of the centerline skeleton image. After excluding the electron-dense sediment area, the statistical screening algorithm is used to select representative measurement results;

[0030] The storage and result output module stores all segmentation results and thickness distribution maps in the database in a preset standardized format, and outputs the visual image and thickness distribution map of the segmented area according to requirements.

[0031] In a third aspect, the present application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned basement membrane lesion analysis method for transmission electron microscopy images of renal puncture biopsy.

[0032] In a fourth aspect, the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the above-mentioned method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy.

[0033] The main contributions and innovations of the present invention are as follows:

[0034] 1. Achieve full process automation: The existing technology lacks a complete automated process and requires manual intervention. The method proposed in the present invention can achieve end-to-end automation from segmentation of glomerular basement membrane (GBM) and electron-dense deposits (EDD), screening of suitable measurement areas, to GBM thickness calculation, without manual intervention, greatly improving diagnostic efficiency and consistency, and meeting the needs of high-throughput pathology detection.

[0035] 2. Improve segmentation accuracy: When segmenting the glomerular basement membrane, traditional methods are prone to mis-segmentation and missed segmentation due to the blurred boundaries between the glomerular basement membrane and the surrounding structures. This paper innovatively introduces the Transformer architecture into the GBM ultrastructure segmentation task, combines the characteristics of CNN, develops the RADS-Net network, and effectively optimizes the segmentation results through the uncertainty evaluation module (UE module), reduces mis-segmentation and missed segmentation, and achieves high-precision semantic segmentation of complex ultrastructures.

[0036] 3. Ensure the accuracy of area screening: The existing thickness calculation method cannot automatically screen the effective area, resulting in inaccurate measurement results. The positioning module of the present invention uses dynamic sliding window clipping and ResNet-34 classification network to accurately classify and label the segmented area, eliminate the intermediate area and other areas that are not suitable for measurement, ensure the effectiveness of the measurement area, and thus improve the reliability of thickness calculation.

[0037] 4. Optimize thickness calculation: Based on centerline skeleton extraction and region growing algorithm, the present invention designs a new GBM thickness calculation method. At the same time, it combines the region exclusion strategy and statistical distribution screening, which can not only ensure the accuracy of the thickness calculation results, but also meet the needs of pathological diagnosis for thickness distribution details.

[0038] 5. Reduce resource dependence: Existing methods face memory limitations when processing high-resolution images, and the cost of manual annotation is high. This paper reduces the annotation cost through a semi-automatic dataset construction strategy, and uses an optimized network design to reduce the demand for computing resources, making the method easier to promote and apply.

[0039] In summary, the present invention has significant advantages in the analysis of glomerular basement membrane lesions, can effectively solve the problems existing in the prior art, and provides efficient and accurate technical support for pathological research and diagnosis.

[0040] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 is a flow chart of a method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to an embodiment of the present application;

[0043] Figure 2 is a thickness calculation flow chart of a method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to an embodiment of the present application;

[0044] Figure 3 is a diagram of the generation result of a transmission electron microscope image of a renal puncture biopsy according to an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of the process of using the segmentation module in RADS-Net according to an embodiment of the present application;

[0046] Figure 5 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0048] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0049] The existing glomerular basement membrane analysis technology lacks a fully automated process and has defects such as difficulty in segmentation, manual reliance on measurement area screening, and limited computing resources and efficiency.

[0050] Based on this, the present invention is based on RADS-Net to solve the problems existing in the prior art.

[0051] Embodiment 1

[0052] This application aims to propose a method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy. Specifically, a workstation equipped with an RTX3090 GPU or more advanced models is used for model reasoning and analysis to ensure that the computing resources meet the needs of large-scale segmentation and thickness calculation. Figure 1 , the method comprising:

[0053] S00, Data Preprocessing

[0054] 1) Sample preparation:

[0055] High-resolution (5K-20K magnification) electron microscope images of glomeruli were collected as input data for glomerular ultrastructural analysis.

[0056] 2) Image Cropping:

[0057] The entire image is segmented into 512×512 pixel blocks using overlapping cropping technology with an overlap size of 256, and then standardized to adapt to the input requirements of the RADS-Net segmentation model.

[0058] S10, segmentation and optimization

[0059] 1) Split execution:

[0060] Combining the characteristics of Transformer and CNN, using the segmentation module in RADS-Net (using multi-layer feature extraction and U-Net decoder), such as Figure 4As shown in Figure 1, the encoder extracts multi-resolution features through a hierarchical improved Transformer module, from coarse-grained to fine-grained, to provide support for high-precision segmentation. The self-attention of the Transformer module is improved, and the calculation formula is as shown in Formula 1. The sequence compression technology is used to reduce the calculation complexity of self-attention O(N 2 ) is O(N 2 / R 2 ). The cropped image blocks are segmented with high precision for glomerular basement membrane (GBM) and electron dense deposits (EDD), generating Fine-GBM-mask (finely refined GBM mask) and EDD-mask (electron dense mask).

[0061] Among them, Transformer is a deep learning model architecture based on the attention mechanism, which was originally used for natural language processing tasks such as machine translation. Its core mechanism is shown in Formula 1:

[0062]

[0063] Among them, Q, K, and V are three core concepts, representing queries respectively. The representation of each patch is transformed to generate a Query vector, which is used to indicate which other patches this patch needs to "pay attention to". The Key vector is a representation of the features of each patch, providing an "identity identifier". The Value vector is the actual information contained in the patch. After the model calculates the similarity between the Query and the Key, it will perform a weighted summation on the Value using the weight to generate the feature update of the current patch.

[0064] CNN is a type of neural network specifically designed to process grid-structured data such as images. U-Net is a convolutional neural network architecture for image segmentation, which is characterized by a symmetrical U-shaped structure.

[0065] 2) Global optimization:

[0066] To solve the problem of global information loss caused by large input image size, the Refine-Net module is designed. Figure 1 ), processes the semantic information of the whole image and supplements the global consistency of the segmentation results, such as Figure 1The macroscopic GBM mask in . It supplements the global consistency of the segmentation results by extracting and fusing multi-scale features of the whole image, improving the accuracy and completeness of the segmentation. The key part is the uncertainty assessment module (U-E module) to generate a confidence heat map, remove low-confidence areas, and only retain segmentation results suitable for further analysis. The principle is as shown in Formula 2. It generates an uncertainty heat map of the segmentation results by fusing global and local information, thereby screening out high-confidence segmentation areas and improving the accuracy and stability of the segmentation. The specific steps are:

[0067] First, for the input image block, the RADS-Net network is used to obtain the preliminary segmentation result. Then, the global features of the segmentation result and the features of the local fine segmentation are calculated to check whether the segmented shape matches: the degree of overlap between the preliminary segmentation and the global segmentation is calculated (similar to comparing the overlap area of ​​two figures).

[0068] Secondly, compare whether the pixel values ​​are close: compare the activation probabilities of the two segmentation masks pixel by pixel to evaluate whether their boundaries and textures are consistent. Then, concatenate the global features and local features to obtain the fused feature vector. The sigmoid function can be used to limit the output value between 0 and 1. The closer the value is to 1, the higher the confidence, and the closer it is to 0, the greater the uncertainty.

[0069] Wherein, formula 2 is as follows:

[0070] U_Gi=λ*U_Si+(1-λ)*(1-U_Pi)

[0071] U_Gi: Comprehensive uncertainty assessment value of basement membrane (GBM) i.

[0072] U_Si: Based on the uncertainty of shape feature matching, it is obtained by calculating the IOU between contours.

[0073] U_Pi: Matching uncertainty based on pixel-level edges,

[0074] according to:

[0075] A gi represents the sum of the pixel areas of the basement membrane i, p F 、p R Represents the probability value of the pixel after activating segmentation inference for Fmapi and Rmapj. is a subset of the non-zero points in Fmapi of the original inference graph in F, In the mask, the horizontal and vertical coordinates of each pixel are represented by k and l.

[0076] λ: A hyperparameter of fusion depth that controls the weight of the two.

[0077] 4) Localization of glomerular basement membrane area:

[0078] Based on the ResNet-34 classification network pre-trained for the GBM positioning task, the actual size of the cropped field of view is 2.5μm×2.5μm, and the overlap sampling is 50%. The OCR algorithm dynamically crops the appropriate field of view area according to the image scale, not a fixed pixel size. The segmentation results are classified by region, and areas that do not meet the measurement requirements (such as interstitial or other interference areas) are eliminated. Combined with the sliding window cropping strategy, the image is analyzed region by region, and the areas suitable for thickness measurement are marked.

[0079] Among them, ResNet-34 is a deep residual network, a type of convolutional neural network. It solves the gradient vanishing and gradient exploding problems in the deep neural network training process by introducing residual connections, making the network easier to train and achieve better performance.

[0080] S20, thickness calculation (such as Figure 2 )

[0081] 1) Central skeleton calculation:

[0082] The centerline skeleton image is extracted from the segmented GBM area, and the refinement algorithm in the skeletonization algorithm developed by OpenCV is used to obtain the precise skeleton path. The specific algorithm pseudo code 1 is as follows Table 1:

[0083]

[0084]

[0085] 2) Longest skeleton line search (calculation of GBM contour line):

[0086] First, the region growing algorithm is used to generate a complete maximum skeleton line. The specific steps are as follows:

[0087] 1. Each pixel on the refined centerline skeleton is taken as a seed point, the skeleton image is represented as Mapt, and its non-zero pixels are represented by Pi.

[0088] 2. Apply the region growing method criterion to find a set of endpoints denoted as Pe that satisfies: N(Pi) means counting the number of pixels in the eight neighborhoods of pixel Pi.

[0089] 3. We use a stack data structure to store the coordinates of points and growing parent points.

[0090] 4. By backtracking, find the data points consisting of the longest skeleton as the central skeleton of the basement membrane, recorded as the set Mapsk, as the GBM contour line.

[0091] 3) Thickness sampling calculation

[0092] Calculate the thickness of the entire GBM. According to the points on Mapsk, sample the thickness of each point at equal intervals n, where n is a variable value. The thickness calculation method is based on the fusion of vertical line fitting and distance constraints. Specifically: select the point to be measured as Pj, fit means to perform binomial fitting on the pixel point to obtain the tangent slope of the measurement point, and then calculate the slope of the vertical line kj to obtain the measurement point where the quadratic polynomial fitting curve is located, expressed as:

[0093] kj = -1 / fit(Pj-25, Pj-24, ..., Pj+24, Pj+25), Pj∈Mapsk. At the same time, the invalid area overlapping with EDD is eliminated to obtain the thickness distribution.

[0094] 4) Thickness measurement area selection

[0095] After obtaining the thickness distribution, select a representative measurement area based on the uniformity of the thickness distribution. The specific steps are as follows:

[0096] First, a thickness distribution histogram is drawn based on the calculated thickness data;

[0097] Then, calculate the mean μ and standard deviation σ of the histogram;

[0098] Select the area with thickness value in the range of [μ-kσ,μ+kσ] as the candidate area, where k is an adjustable parameter, for example, k=2;

[0099] Finally, the largest and continuous region is selected from the candidate regions as the final representative measurement region.

[0100] S30, result splicing and output

[0101] 1) Stitching field of view:

[0102] Multiple segmentation results are stitched together to restore the overall segmentation information of the original image and ensure the consistency of the segmentation area and thickness distribution throughout the image.

[0103] 2) Visualization generation:

[0104] The grayscale value and thickness distribution of the segmentation results are mapped to different color channels to generate visual images to assist pathologists in intuitive analysis. Figure 3 shown.

[0105] S40, data preservation:

[0106] The segmentation results, thickness distribution map and original image are saved separately to facilitate subsequent in-depth analysis and verification.

[0107] Through the above steps, the present invention realizes the fully automated analysis of the glomerular basement membrane. The complete process from image segmentation to thickness calculation ensures high efficiency and accuracy, and provides reliable data support and technical tools for pathological research and diagnosis.

[0108] Embodiment 2

[0109] Based on the same concept, the present application also proposes a basement membrane lesion analysis device for renal puncture biopsy transmission electron microscope images, comprising:

[0110] 1. Data preprocessing module

[0111] The present invention uses high-resolution electron microscope images (3296×2563 pixels) as input data and designs cropping and preprocessing strategies based on the complexity of ultrastructure. The input image is cropped to generate 512×512 pixel image blocks through overlapping, which is suitable for the input requirements of the deep learning model.

[0112] 2. Segmentation module

[0113] Model Architecture:

[0114] The present invention adopts RADS-Net network, combines the characteristics of Transformer and CNN, utilizes multi-layer feature extraction and U-Net decoder, and realizes high-precision segmentation of glomerular basement membrane (GBM) and electron-dense deposits (EDD).

[0115] Global optimization:

[0116] To address the problem of global information loss caused by large input image size, a Refine-Net module was designed to process the semantic information of the entire image and supplement the global consistency of the segmentation results.

[0117] 3. Segmentation result optimization and key area screening module

[0118] Uncertainty Assessment Module (U-E Module):

[0119] The present invention designs an uncertainty assessment module, which generates an uncertainty heat map of the segmentation result by fusing global and local information, screens out high-confidence segmentation areas, and improves the accuracy and stability of segmentation.

[0120] Positioning module:

[0121] The positioning module is based on the ResNet-34 classification network, which classifies the segmentation results into regions and removes regions that do not meet the measurement requirements (such as interstitial or other interference regions). Combined with the sliding window cropping strategy, the image is analyzed region by region to mark the areas suitable for thickness measurement.

[0122] 4.Thickness calculation module

[0123] Centerline skeleton extraction:

[0124] From the filtered GBM segmentation area, the centerline skeleton graph is extracted, and the precise centerline path is obtained through the refinement algorithm.

[0125] Area growth and thickness distribution calculation:

[0126] Based on the centerline skeleton, the region growing algorithm is used to calculate the vertical thickness of each pixel and generate a thickness distribution map. The EDD area is automatically excluded to ensure the representativeness and accuracy of the thickness calculation. Finally, the statistical screening algorithm is used to select the area with uniform thickness as the representative measurement result.

[0127] 5. Storage and result output module

[0128] Storage and Management:

[0129] All segmentation results and thickness distribution maps are stored in a database in a standardized format to facilitate subsequent analysis.

[0130] Result output:

[0131] The output of the segmented area visualization image and thickness distribution map provides pathologists with an intuitive auxiliary diagnosis tool and supports seamless integration with existing pathology analysis systems.

[0132] 6. Hardware and parallel computing optimization module

[0133] Hardware Platform:

[0134] The model of the present invention is carried out on an i9-13900K—RTX3090 GPU, RAM128G workstation (the same type of AMD R9 9950X with RTX4090 is also OK, the stronger the performance, the faster the processing speed), ensuring large-scale parallel processing capabilities.

[0135] Computational optimization:

[0136] The multi-threaded parallel processing technology is used to accelerate the cropping, segmentation, screening and thickness calculation processes, significantly shortening the processing time of a single image and meeting the needs of efficient medical record analysis.

[0137] The present invention is suitable for various pathology laboratories, hospitals and research institutions, and has broad application prospects in the following fields:

[0138] 1. Kidney disease diagnosis: Through automated segmentation and thickness measurement of the glomerular basement membrane (GBM), diseases such as thin basement membrane disease and Alport syndrome can be accurately identified, providing a reliable basis for early diagnosis of kidney disease and formulation of treatment plans.

[0139] 2. Electron microscopy pathology analysis: Achieve efficient segmentation and quantitative analysis of ultrastructures, reduce manual operations, and improve the efficiency and accuracy of electron microscopy in pathology diagnosis.

[0140] 3. Pathology big data processing: Supports automated analysis of large-scale high-resolution pathology images, and provides technical support for artificial intelligence research and big data mining in the field of pathology.

[0141] 4. Basic medical research: Provide convenient tools for the study of renal pathology and ultrastructural biology, and promote in-depth exploration of the pathogenesis of basement membrane-related diseases.

[0142] 5. Clinical trials and drug development: In drug development and clinical trials, it supports the quantitative evaluation of changes in basement membrane thickness, and assists in drug target analysis and efficacy monitoring.

[0143] 6. Education and training: Provide automated auxiliary tools for pathology teaching and training, reduce the learning curve of complex microsurgery operations, and improve teaching efficiency.

[0144] The present invention is based on efficient and accurate automation technology, and is widely applicable to various application scenarios in the medical and scientific research fields, with important practical value and promotion prospects.

[0145] Embodiment 3

[0146] This embodiment also provides an electronic device, referring to Figure 5 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0147] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0148] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). In appropriate circumstances, the ROM can be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH) or a combination of two or more of the above. Under appropriate circumstances, the RAM can be a static random access memory (Static Random-Access Memory, abbreviated as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, abbreviated as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory 404 (Fast Page Mode Dynamic Random Access Memory, abbreviated as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, abbreviated as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, abbreviated as SDRAM), etc.

[0149] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0150] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the basement membrane lesion analysis methods for renal puncture biopsy transmission electron microscopy images in the above embodiments.

[0151] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0152] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0153] Input / output devices 408 are used to input or output information.

[0154] Embodiment 4

[0155] This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the basement membrane lesion analysis method for renal puncture biopsy transmission electron microscopy images according to embodiment one.

[0156] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0157] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0158] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0159] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy, characterized in that: The following steps are involved: S00, taking the electron microscope image as input, generating image blocks adapted to the input requirements of the deep learning model by overlapping and cropping; S10, RADS-Net network combining Transformer and CNN architecture is used to segment the glomerular basement membrane and electron-dense deposits in the image blocks; S20, generating an uncertainty heat map of the segmentation result by fusing global and local information through the uncertainty assessment module, screening out high-confidence areas, and eliminating areas that do not meet the measurement requirements using a positioning module based on a classification network; S30. Extract the centerline skeleton image from the screened high-confidence area segmentation area, use the refinement algorithm in the skeletonization algorithm developed by OpenCV to obtain the accurate skeleton path, use the regional growing algorithm to calculate the vertical thickness and generate the thickness distribution map and the complete maximum skeleton line, and then calculate the thickness value of the centerline skeleton image. After excluding the electron-dense sediment area, use the statistical screening algorithm to select representative measurement results.

2. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to claim 1, characterized in that: In step S10, the RADS-Net network includes multi-layer feature extraction and U-Net decoder.

3. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to claim 1, characterized in that: In step S20, the classification network used by the positioning module is ResNet-34, and the OCR algorithm is used to dynamically cut out a suitable field of view according to the image scale, analyze the image region by region, and mark the area suitable for thickness measurement.

4. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to claim 1, characterized in that: In step S00, the resolution of the electron microscope image is at least 3296×2563 pixels.

5. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy as claimed in claim 4, characterized in that: In step S00, the input image is cropped through overlapping to generate image blocks of 512×512 pixels to adapt to the input requirements of the deep learning model.

6. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to claim 1, characterized in that: In step S20, the uncertainty assessment module generates an uncertainty heat map of the segmentation result by fusing global and local information in the following specific method: For the input image block, the preliminary segmentation result is obtained through the RADS-Net network; Calculate the global features of the segmentation results and the local features of the segmentation results; The global features and local features are concatenated to obtain a fused feature vector; The fused feature vector is mapped to an uncertainty value through a fully connected layer to generate an uncertainty heat map of the segmentation result, which is used to indicate the confidence of the segmentation result.

7. A method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to any one of claims 1 to 6, characterized in that: The method further includes step S40, storing all segmentation results and thickness distribution maps in a database in a preset standardized format, and outputting a visualization image and thickness distribution map of the segmented area according to requirements.

8. A device for analyzing basement membrane lesions in transmission electron microscope images of renal puncture biopsy, characterized in that: include: The data preprocessing module takes the electron microscope image as input and generates image blocks that meet the input requirements of the deep learning model through overlapping cropping; The segmentation module uses the RADS-Net network that combines the Transformer and CNN architectures to segment the glomerular basement membrane and electron-dense deposits in the image blocks; The segmentation result optimization and key area screening module generates an uncertainty heat map of the segmentation result by fusing global and local information through the uncertainty assessment module, screens out high-confidence areas, and uses the positioning module based on the classification network to eliminate areas that do not meet the measurement requirements; The thickness calculation module extracts the centerline skeleton image from the screened high-confidence area segmentation area, uses the region growing algorithm to calculate the vertical thickness and generate the thickness distribution map and the complete maximum skeleton line, and then calculates the thickness value of the centerline skeleton image. After excluding the electron-dense sediment area, the statistical screening algorithm is used to select representative measurement results; The storage and result output module stores all segmentation results and thickness distribution maps in the database in a preset standardized format, and outputs the visual image and thickness distribution map of the segmented area according to requirements.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the basement membrane lesion analysis method for renal puncture biopsy transmission electron microscopy images according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes a method for analyzing basement membrane lesions in transmission electron microscopy images of renal puncture biopsy according to any one of claims 1 to 7.

Images