Heterogeneous data-based kidney image processing method and apparatus, and electronic device
Through the renal image processing method based on heterogeneous data, the characteristics of early images of kidney disease are extracted using convolutional neural networks and recurrent neural networks, the problem of low classification accuracy in early kidney disease is solved, the comprehensiveness and accuracy of detection are improved, and the overall accuracy of detection is reduced. The work pressure of the doctor.
Patent Information
- Application Number
- CN202411183183.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the early stage of kidney disease, when renal image classification is performed through artificial intelligence, the extracted renal pathological feedback results are not obvious, resulting in low classification accuracy and increasing the work pressure of doctors and the risk of misjudgment.
Using a renal image processing method based on heterogeneous data, pre-processing is performed by acquiring heterogeneous data (including renal medical images, blood oxygen level-dependent imaging images, arterial spin imaging images, diffusion-weighted imaging images and diffusion tensor imaging images). and feature extraction, combined with convolutional neural network and recurrent neural network model, extract more refined features and perform correlation processing to improve classification accuracy.
It improves the accuracy of early-stage image classification of kidney disease, reduces the work pressure of doctors, reduces the risk of misjudgment, and enhances the comprehensiveness and accuracy of detection.
Smart Images

Figure CN118967655A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of medical data processing, and in particular to a method, device and electronic device for processing kidney images based on heterogeneous data. Background Art
[0002] The kidney is an important organ of the human body. Its basic function is to produce urine, remove metabolic products and certain wastes and poisons in the body, and maintain water, electrolyte and acid-base balance. If the kidney is diseased, it will not be able to effectively play the important function of "filtering" waste substances, resulting in the accumulation of many toxins and harmful substances in the body. With the development of MRI (Magnetic Resonance Imaging) technology, the morphology, size, structure and blood flow of the kidney can be observed, and lesions such as abnormal kidney structure, tumors, and cysts can be detected. When the number of patients is huge, doctors may make misjudgments due to anxiety or fatigue. Therefore, it is proposed to use artificial intelligence to classify images first and then show them to doctors, which can relieve the pressure on doctors. However, in the early stage of kidney disease, when renal images are classified by artificial intelligence, the classification accuracy is low because the extracted renal pathology feedback results are not obvious. Summary of the invention
[0003] The present application provides a kidney image processing method, device, electronic device and readable storage medium based on heterogeneous data, which can improve the accuracy of classification results.
[0004] The technical solution of the embodiment of the present application is as follows: In a first aspect, an embodiment of the present application provides a method for processing a kidney image based on heterogeneous data, the method comprising: Acquiring heterogeneous data, wherein the heterogeneous data includes a renal medical image to be detected, a blood oxygen level dependent imaging image, an arterial spin imaging image, a diffusion weighted imaging image, and a diffusion tensor imaging image; Preprocessing the renal medical image to obtain image preprocessing features, and fusion preprocessing the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image, and the diffusion tensor imaging image to obtain fusion preprocessing features; The image preprocessing features and the fusion preprocessing features are input into a preset target detection model, and the following processing is performed: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; Using the convolutional neural network module to perform feature extraction on the image preprocessing feature and the fusion preprocessing feature to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature; Using the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain a second fusion feature; Classify the second fusion feature and output an image classification result.
[0005] In the above technical scheme, heterogeneous data are firstly obtained, and the heterogeneous data include renal medical images to be detected, blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images. By obtaining heterogeneous data of different indicators, the detection is made more comprehensive and the classification accuracy is improved; the renal medical images are preprocessed to obtain image preprocessing features, and the main feature components are extracted, which can reduce feature calculations; the blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images are fused preprocessed to obtain fused preprocessing features. By performing image preprocessing, the representation ability of the fused preprocessing features is made stronger, and more comprehensive information is captured. Combining data from different aspects for processing can improve the classification accuracy; the image preprocessing features and the fused preprocessing features are input into the preset In the target detection model, the following processing is performed: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; the convolutional neural network module is used to extract features from the image preprocessing features and the fusion preprocessing features to obtain kidney image features corresponding to the image preprocessing features and the first fusion features corresponding to the fusion preprocessing features. On the basis of preprocessing, feature extraction is performed again to extract more refined features; the recurrent neural network module is used to perform correlation processing on the kidney image features and the first fusion features to obtain the second fusion features. Heterogeneous extraction can extract the contextual relationship between pixels of each image in the heterogeneous data, and integrate the features of various aspects together, which is conducive to improving the detection accuracy; the second fusion feature is classified and processed to output the image classification result. For images of early kidney disease, the classification accuracy is relatively high.
[0006] In some embodiments of the present application, the convolutional neural network module includes a plurality of residual blocks and a first convolution layer, each residual block includes an activation function layer and at least one second convolution layer; The method of using the convolutional neural network module to extract features from the image preprocessing features and the fusion preprocessing features to obtain a renal image feature corresponding to the image preprocessing features and a first fusion feature corresponding to the fusion preprocessing features includes: Using the second convolution layer to extract the image preprocessing features and the fusion preprocessing features, and using the activation function layer to perform nonlinear processing to obtain a first feature corresponding to the image preprocessing features and a second feature corresponding to the fusion preprocessing features; The first convolutional layer is used to perform feature extraction processing on the first feature and the second feature respectively to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature, wherein the convolution kernel of the first convolutional layer is 1×1.
[0007] In the above technical solution, by utilizing the residual block, not only can the network depth be increased and feature information be fully extracted, but also the gradient disappearance can be avoided.
[0008] In some embodiments of the present application, the fusion preprocessing of the blood oxygen level dependence imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image to obtain the fusion preprocessing features includes: Performing feature vector conversion on the blood oxygen level dependence imaging image to obtain a first feature vector; Performing feature vector conversion on the arterial spin imaging image to obtain a second feature vector; Performing feature vector conversion on the diffusion weighted imaging image to obtain a third feature vector; Performing feature vector conversion on the diffusion tensor imaging image to obtain a fourth feature vector; Perform an attention mechanism fusion on the first feature vector, the second feature vector, the third feature vector and the fourth feature vector to obtain the fused preprocessing feature.
[0009] In the above technical solution, each image data is first converted into a feature vector to conform to the data input format, and then the attention mechanism is integrated, which can focus on features that are more conducive to classification and improve the accuracy of classification detection.
[0010] In some embodiments of the present application, performing an attention mechanism fusion on the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector to obtain the fused preprocessing feature includes: Obtaining a preset first weight value, a preset second weight value, a preset third weight value, and a preset fourth weight value, wherein the sum of the first weight value, the second weight value, the third weight value, and the fourth weight value is 1; Multiply the first eigenvector by the first weight value to obtain a first product, multiply the second eigenvector by the second weight value to obtain a second product, multiply the third eigenvector by the third weight value to obtain a third product, and multiply the fourth eigenvector by the fourth weight value to obtain a fourth product; The first product, the second product, the third product and the fourth product are connected and normalized to obtain the fusion preprocessing feature.
[0011] In the above technical solution, through the fusion of the attention mechanism, it is possible to focus on features that are more conducive to classification and improve the accuracy of classification detection.
[0012] In some embodiments of the present application, the using the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain the second fusion feature includes: Performing feature connection on the renal image feature and the first fusion feature to obtain a third fusion feature; The recurrent neural network module is used to perform feature extraction processing on the third fusion feature to obtain the second fusion feature.
[0013] In the above technical solution, the renal image feature is first fused with the first fusion feature through feature splicing, which can make the feature more expressive. Then, the recurrent neural network is used for feature extraction to obtain the correlation between each image in the heterogeneous image, which is conducive to improving the classification accuracy.
[0014] In some embodiments of the present application, before inputting the image preprocessing features and the fusion preprocessing features into a preset target detection model, the method further includes: Acquire a heterogeneous data set, the heterogeneous data set comprising a plurality of training renal images, training blood oxygen level dependent imaging images, training arterial spin imaging images, training diffusion weighted imaging images and training diffusion tensor imaging images corresponding to each of the training renal images, and target classifications corresponding to each of the training renal images; Preprocessing each of the training renal images to obtain a first preprocessing feature set, and fusing the training blood oxygen level dependent imaging image, the training arterial spin imaging image, the training diffusion weighted imaging image, and the training diffusion tensor imaging image corresponding to each of the training renal images to obtain a second preprocessing feature set; Inputting the first preprocessing feature set and the second preprocessing feature set into a preset initial detection model, and performing the following processing: the initial detection model includes a preset initial convolutional neural network module and a preset initial recurrent neural network module; Using the initial convolutional neural network module to perform feature extraction on the first preprocessing feature set and the second preprocessing feature set to obtain an image feature set corresponding to the first preprocessing feature set and a fusion feature set corresponding to the second preprocessing feature set; Using the initial recurrent neural network module to perform correlation processing on the image feature set and the fusion feature set to obtain an output feature set; Performing classification processing on the output feature set and outputting a plurality of predicted image classifications; The value of the loss function is calculated according to each predicted image classification and the target classification, and the value of the loss function is used to adjust the parameters of the initial detection model to obtain the target detection model.
[0015] In the above technical solution, by training the initial detection model, the obtained target detection model has good generalization, which is conducive to the classification detection of different types of data.
[0016] In some embodiments of the present application, the value of the loss function is calculated according to the predicted image classification and the target classification, and the value of the loss function is used to adjust the parameters of the initial detection model to obtain the target detection model, including: Using a preset loss function to perform loss calculation on the predicted image classification and the target classification to obtain a value of the loss function; In the first round of training, the value of the loss function is used to adjust the parameters of the initial recurrent neural network module and the initial convolutional neural network module in turn. In the second round of training, the value of the loss function is used to adjust the parameters of the initial recurrent neural network module, and the initial convolutional neural network module is not adjusted. In the third round of training, the initial recurrent neural network module is not adjusted, and the value of the loss function is used to adjust the parameters of the initial convolutional neural network module. In multiple rounds of training, the first round, the second round, and the third round are cycled to adjust the parameters of the initial recurrent neural network module and the initial convolutional neural network module to obtain the target detection model.
[0017] In the above technical solution, by adjusting the parameters of the initial recurrent neural network module and the initial convolutional neural network module in turn, the features extracted by the convolutional neural network module and the features extracted by the recurrent neural network module have a mutual correlation, which can improve the classification accuracy of the target detection model.
[0018] In a second aspect, an embodiment of the present application provides a kidney image processing device based on heterogeneous data, the device comprising: A data acquisition module, used for acquiring heterogeneous data, wherein the heterogeneous data includes a renal medical image to be detected, a blood oxygen level dependent imaging image, an arterial spin imaging image, a diffusion weighted imaging image, and a diffusion tensor imaging image; A data preprocessing module is used to preprocess the renal medical image to obtain image preprocessing features, and to perform fusion preprocessing on the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image to obtain fusion preprocessing features; A model processing module, used for inputting the image preprocessing features and the fusion preprocessing features into a preset target detection model, and performing the following processing: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; A first processing module is used to perform feature extraction on the image preprocessing feature and the fusion preprocessing feature by using the convolutional neural network module to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature; A second processing module is used to use the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain a second fusion feature; The classification detection module is used to classify the second fusion feature and output an image classification result.
[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods provided in the first aspect above.
[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any one of the methods provided in the first aspect is executed.
[0021] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Due to the use of heterogeneous data reflecting different levels, the heterogeneous data is first preprocessed to achieve rough feature extraction, extract the main characteristic components of the image, and process them into the same format, which is conducive to the subsequent target detection model processing. Then, different neural network models are used to extract features from the preprocessed data to extract finer features, and then feature fusion is performed to obtain the correlation between the extracted features, and then classification processing is performed to obtain the technical means of the detection results. According to the data at different levels, different information is captured, the correlation between different information is extracted, and the accuracy of detection is improved.
[0022] 2. By using residual blocks, not only can the network depth be increased and feature information fully extracted, but also the gradient disappearance can be avoided.
[0023] 3. Through the fusion of attention mechanism, we can focus on features that are more conducive to classification and improve the accuracy of classification detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 is a flow chart of a method for processing kidney images based on heterogeneous data provided by an embodiment of the present application; Figure 2 yes Figure 1 A schematic flow chart of a sub-step of step S131; Figure 3 yes Figure 1 A schematic flow chart of a sub-step of step S132; Figure 4 is a flow chart of a method for processing kidney images based on heterogeneous data provided by another embodiment of the present application; Figure 5 is a schematic structural diagram of a kidney image processing device based on heterogeneous data provided by an embodiment of the present application; Figure 6 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.
[0026] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.
[0027] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.
[0028] The embodiments of the present application provide a method, device and electronic device for processing renal images based on heterogeneous data. The method for processing renal images based on heterogeneous data first obtains heterogeneous data, wherein the heterogeneous data includes renal medical images to be detected, blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images. By obtaining heterogeneous data of different indicators, the detection is made more comprehensive and the classification accuracy is improved. The renal medical images are preprocessed to obtain image preprocessing features, and the main feature components are extracted, which can reduce feature calculation. The blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images are fused preprocessed to obtain fused preprocessing features. By performing image preprocessing, the representation ability of the fused preprocessing features is made stronger, and more comprehensive information is captured. Data from different aspects are combined for processing, which can improve the classification accuracy. The image preprocessing features and the fusion preprocessing features are input into the preset target detection model, and the following processing is performed: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; the image preprocessing features and the fusion preprocessing features are extracted by using the convolutional neural network module to obtain the renal image features corresponding to the image preprocessing features and the first fusion features corresponding to the fusion preprocessing features. On the basis of preprocessing, feature extraction is performed again to extract more refined features; the recurrent neural network module is used to perform correlation processing on the renal image features and the first fusion features to obtain the second fusion features, and the heterogeneous extraction can extract the contextual relationship between the pixels of each image in the heterogeneous data, and the features of various aspects are integrated together, which is conducive to improving the detection accuracy; the second fusion feature is classified and processed, and the image classification result is output. For the image in the early stage of renal disease, the classification accuracy is higher. Compared with the prior art in the early stage of renal disease, when renal images are classified by artificial intelligence, the extracted renal pathology feedback results are not obvious and the classification accuracy is low, the embodiment of the present application can improve the accuracy of classification detection.
[0029] It should be noted that the renal image processing method based on heterogeneous data is applied in medical equipment. For the misjudgment that may be caused by early nephrology doctors under work pressure, a classification screening is first performed through the renal image processing method based on heterogeneous data to reduce the pressure on doctors.
[0030] The technical solution provided in the embodiments of the present application is further described below in conjunction with the accompanying drawings.
[0031] Reference Figure 1 , Figure 11 is a flow chart of a method for processing renal images based on heterogeneous data provided by an embodiment of the present application. The method for processing renal images based on heterogeneous data is applied to a renal image processing device based on heterogeneous data, and the method for processing renal images based on heterogeneous data is performed by an electronic device or a computer-readable storage medium. The method for processing renal images based on heterogeneous data includes steps S110, S120, and S130, and step S130 includes steps S131, S132, and S133.
[0032] Step S110, obtaining heterogeneous data, wherein the heterogeneous data includes a renal medical image to be detected, a blood oxygen level dependent imaging image, an arterial spin imaging image, a diffusion weighted imaging image, and a diffusion tensor imaging image.
[0033] In one embodiment, heterogeneous data is data in multiple representation forms, and the heterogeneous data includes renal medical images to be detected, blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images, and diffusion tensor imaging images. The renal medical images to be detected are obtained by collecting computer tomography or magnetic resonance imaging of patients, and are used to observe and evaluate the anatomical structure, morphology, and lesions of the kidney; on the basis of nuclear magnetic resonance images, blood oxygen level dependent imaging combined with functional magnetic resonance imaging imaging methods are used to observe and evaluate the oxygenation state and metabolic activity of specific areas in the organ; arterial spin imaging images use the contrast between spin-labeled water molecules and unlabeled water molecules to quantitatively measure blood flow, and are used to measure cerebral blood flow imaging images; diffusion weighted imaging images are based on magnetic resonance imaging to detect and evaluate renal lesions and inflammation; and diffusion tensor imaging images use magnetic resonance imaging to measure and analyze the diffusion direction and degree of water molecules in tissues, and can observe the diffusion direction and fiber structure information of water molecules in renal tissues. The above-mentioned images are stored, and the heterogeneous data are obtained using the preset image reading results. Data of different indicator forms can be obtained through heterogeneous data, making the detection more comprehensive.
[0034] Step S120, preprocessing the renal medical image to obtain image preprocessing features, and performing fusion preprocessing on the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image to obtain fusion preprocessing features.
[0035] In one embodiment, the renal medical image can be converted into a feature vector by means of statistical feature extraction to obtain image preprocessing features, or pixels can be arranged according to the renal medical image to obtain image preprocessing features. By converting the image to a feature vector, it is beneficial to subsequently input the renal medical image into a target detection model for classification processing.
[0036] In another embodiment, the blood oxygen level dependence imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image are fused preprocessed to obtain fusion preprocessing features, including but not limited to: the blood oxygen level dependence imaging image can be subjected to feature vector conversion using a statistical feature extraction method or a pixel arrangement method to obtain a first feature vector; the arterial spin imaging image can be subjected to feature vector conversion using a statistical feature extraction method or a pixel arrangement method to obtain a second feature vector; the diffusion weighted imaging image can be subjected to feature vector conversion using a statistical feature extraction method or a pixel arrangement method to obtain a third feature vector; the diffusion tensor imaging image can be subjected to feature vector conversion using a statistical feature extraction method or a pixel arrangement method to obtain a fourth feature vector; obtaining the first feature vector, the second feature vector, the third feature vector and the fourth feature vector in the above manner is conducive to obtaining the fusion preprocessing features subsequently.
[0037] In another embodiment, a preset attention mechanism is used to perform feature fusion on the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector to obtain fusion preprocessing features so as to input the target detection model and improve classification accuracy. The first feature vector, the second feature vector, the third feature vector, and the fourth feature vector are fused by the attention mechanism to obtain fusion preprocessing features, including but not limited to: obtaining a preset first weight value, a preset second weight value, a preset third weight value, and a preset fourth weight value, wherein the sum of the first weight value, the second weight value, the third weight value, and the fourth weight value is 1, and the preset weight value is set by professionals based on experience, and the weight value is read through a parameter reading function. By setting different weight values, the larger the weight value, the higher the corresponding attention, and the focus is on the input subset during subsequent feature extraction, so that the characteristics of the early stage of kidney disease can be reflected.
[0038] Exemplarily, the first weight value may be set to 0.4, the second weight value may be set to 0.3, the third weight value may be set to 0.2, and the fourth weight value may be set to 0.1. Other proportional values may also be used, and the values are set according to the network classification detection result, which will not be elaborated here.
[0039] In another embodiment, the first feature vector is multiplied by the first weight value to obtain a first product, so that the first feature vector has a set degree of attention; the second feature vector is multiplied by the second weight value to obtain a second product, so that the second feature vector has a set degree of attention; the third feature vector is multiplied by the third weight value to obtain a third product, so that the third feature vector has a set degree of attention; the fourth feature vector is multiplied by the fourth weight value to obtain a fourth product, so that the fourth feature vector has a set degree of attention, and then the first product, the second product, the third product and the fourth product are connected according to a preset connection method to fuse the features. The fused features have a strong representation ability and can capture different information, which is conducive to improving the detection results. The preset connection method can be vector concatenation or vector addition. The fused features are normalized, the data is restricted to a unified dimension, and the fused preprocessing features are obtained, which is conducive to improving the accuracy of classification.
[0040] Step S130, input the image preprocessing features and the fusion preprocessing features into a preset target detection model, and perform the following processing: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module.
[0041] In one embodiment, the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module, the preset convolutional neural network module can extract global features and local features of image features, and the preset recurrent neural network module can also be a variant of the recurrent neural network module, which can extract the correlation between pixels. The image preprocessing features and the fusion preprocessing features are input into the preset target detection model, so that the classification can be comprehensively predicted and the classification detection results are more accurate.
[0042] Step S131, using a convolutional neural network module to extract features from the image preprocessing features and the fusion preprocessing features, to obtain kidney image features corresponding to the image preprocessing features and first fusion features corresponding to the fusion preprocessing features.
[0043] In one embodiment, the convolutional neural network module includes a plurality of residual blocks and a first convolution layer, each residual block includes an activation function layer and at least one second convolution layer. Figure 2 As shown, the image preprocessing feature and the fusion preprocessing feature are extracted using a convolutional neural network module to obtain a renal image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature, including but not limited to the following steps: Step S1311, using the second convolution layer to extract the image preprocessing features and the fusion preprocessing features, and using the activation function layer to perform nonlinear processing to obtain a first feature corresponding to the image preprocessing features and a second feature corresponding to the fusion preprocessing features.
[0044] In some embodiments of the present application, the second convolution layer can be a 3×3 convolution kernel or a 5×5 convolution kernel. The size of the convolution kernel is set according to the needs and will not be described here. The second convolution layer is used to extract features of the image preprocessing features and the fusion preprocessing features, and the local features of the image preprocessing features and the fusion preprocessing features are extracted respectively, which is conducive to the subsequent acquisition of the first fusion feature. The extracted features are processed nonlinearly using the activation function layer, which can reduce the amount of calculation, and obtain the first feature corresponding to the image preprocessing feature and the second feature corresponding to the fusion preprocessing feature, which is conducive to the subsequent acquisition of the first fusion feature. The activation function of the activation function layer can be a relu() function or a tanh() function.
[0045] In other embodiments of the present application, the second convolutional layer can be 2 layers or 3 layers, and the number of residual blocks is 3. Since the gradient disappears when the network is deepened, this problem is solved by the skip connection of the residual block. The image preprocessing features and the fusion preprocessing features are respectively extracted by using the 2-layer or 3-layer second convolutional layer in the residual block, and then processed by other residual blocks in turn, and the first feature and the second feature are output, which is conducive to the subsequent renal image feature and the first fusion feature. Through the processing of the above-mentioned residual block, compared with the preprocessing, the more refined features in the image can be extracted, and the network gradient can be avoided from disappearing. Through the feature coarse processing and feature fine processing in turn, it is conducive to improving the classification accuracy.
[0046] Step S1312, using the first convolution layer to perform feature extraction processing on the first feature and the second feature respectively, to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature, wherein the convolution kernel of the first convolution layer is 1×1.
[0047] In some embodiments of the present application, the convolution kernel of the first convolutional layer is 1×1. According to step S1311, the first convolutional layer is used to perform feature extraction processing on the first feature and the second feature respectively, which can make the renal image feature and the first fusion feature linearly distributed, making it convenient to input them into the recurrent neural network module for processing later, thereby improving the accuracy of detection.
[0048] In some possible embodiments of the present application, the first convolutional layer can also be replaced by a fully connected network layer, which can transform the dimensions of the first feature and the second feature respectively, and can subsequently be input into a recurrent neural network. The fully connected layer is not described in detail here.
[0049] Step S132, using a recurrent neural network module to perform correlation processing on the kidney image feature and the first fusion feature to obtain a second fusion feature.
[0050] like Figure 3 As shown, the recurrent neural network module is used to perform correlation processing on the renal image feature and the first fusion feature to obtain the second fusion feature, including but not limited to the following steps: Step S1321, feature-connecting the kidney image feature and the first fusion feature to obtain a third fusion feature.
[0051] In some possible embodiments of the present application, the attention mechanism can be used to connect the renal image feature and the first fusion feature to obtain the third fusion feature, or the renal image feature can be directly added to the first fusion feature to obtain the third fusion feature, which is conducive to the subsequent recurrent neural network to extract the correlation between pixel features. Feature addition refers to combining and connecting the features, splicing the first fusion feature after the renal image feature, and the input data is more comprehensive, which improves the classification accuracy.
[0052] Step S1322: Use a recurrent neural network module to perform feature extraction processing on the third fusion feature to obtain a second fusion feature.
[0053] In some possible embodiments of the present application, according to the third fusion feature obtained in step S1321, the third fusion feature is subjected to feature extraction processing using a recurrent neural network module, so that the correlation between the pixels of the renal image can be obtained, and the correlation between the renal image and other imaging images can also be obtained, which can better reflect the renal structure and the pathological condition in the image or image, and obtain the second fusion feature, so as to facilitate the subsequent classification processing according to the second fusion feature.
[0054] Step S133, classify the second fusion feature and output the image classification result.
[0055] In one embodiment, the second fusion feature is classified using a preset classification algorithm, and an image classification result is output, which helps doctors make diagnoses based on the classification results and reduces the workload of doctors. The classification algorithm can be softmax(), which is used to calculate the second fusion feature. According to the calculated probability, the category with a probability greater than a preset threshold is classified as risky and requires doctor observation, and the category with a probability less than or equal to the preset threshold is classified as risk-free and does not require doctor inspection, which can effectively reduce the workload of doctors.
[0056] like Figure 4 As shown, before the image preprocessing features and the fusion preprocessing features are input into the preset target detection model, the kidney image processing method based on heterogeneous data also includes but is not limited to the following steps: Step S210, obtaining a heterogeneous data set, the heterogeneous data set including a plurality of training renal images, training blood oxygen level dependent imaging images corresponding to each training renal image, training arterial spin imaging images, training diffusion weighted imaging images and training diffusion tensor imaging images, and target classifications corresponding to each training renal image.
[0057] In one embodiment, the heterogeneous data set includes a plurality of training renal images, training blood oxygen level dependent imaging images corresponding to each training renal image, training arterial spin imaging images, training diffusion weighted imaging images and training diffusion tensor imaging images, and target classifications corresponding to each training renal image. First, the training renal images are collected. The training renal images can be collected in a hospital. It should be noted that the collection obtains the consent of the person being collected, and the training blood oxygen level dependent imaging images, training arterial spin imaging images, training diffusion weighted imaging images and training diffusion tensor imaging images corresponding to each training renal image are collected from the same person, and the target classifications are obtained by manual annotation to form a heterogeneous data set and store it. The stored heterogeneous data set is read through a preset data reading interface to provide data support for subsequent network training.
[0058] Step S220, preprocessing each training renal image to obtain a first preprocessing feature set, and fusion preprocessing is performed on the training blood oxygen level dependence imaging image, training arterial spin imaging image, training diffusion weighted imaging image and training diffusion tensor imaging image corresponding to each training renal image to obtain a second preprocessing feature set.
[0059] In one embodiment, each training renal image can be converted into a feature vector by means of statistical feature extraction to obtain a first preprocessing feature set, or pixels can be arranged according to the training renal image to obtain a first preprocessing feature set. By performing the image-to-feature vector conversion, it is beneficial to subsequently input the training renal image into a target detection model for classification processing. According to the processing method of the training renal image, the training blood oxygen level dependent imaging image, the training arterial spin imaging image, the training diffusion weighted imaging image, and the training diffusion tensor imaging image corresponding to each training renal image are first converted into a feature vector, and then the converted feature vector is fused and preprocessed using an attention mechanism to obtain a second preprocessing feature set, which is beneficial to subsequently input the first preprocessing feature set and the second preprocessing feature set into the initial detection model for model training.
[0060] It should be noted that the use of the attention mechanism for feature fusion is similar to the above-mentioned fusion preprocessing method for blood oxygen level dependence imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images. Fusion preprocessing is performed on each data set, which will not be repeated here.
[0061] Step S230, input the first preprocessing feature set and the second preprocessing feature set into a preset initial detection model, and perform the following processing: the initial detection model includes a preset initial convolutional neural network module and a preset initial recurrent neural network module.
[0062] In one embodiment, the initial detection model includes a preset initial convolutional neural network module and a preset initial recurrent neural network module. The preset initial recurrent neural network module can also be a variant of the recurrent neural network module. The first preprocessing feature set and the second preprocessing feature set are input into the preset initial detection model according to the preset batches, which can comprehensively extract features in all aspects, so that the classification detection using the trained model in the future is more accurate. Among them, the preset batches can be 2, 4 or 8, which will not be repeated here.
[0063] Step S231, using the initial convolutional neural network module to extract features from the first preprocessing feature set and the second preprocessing feature set to obtain an image feature set corresponding to the first preprocessing feature set and a fusion feature set corresponding to the second preprocessing feature set.
[0064] In one embodiment, the initial convolutional neural network module includes multiple residual blocks and a first convolutional layer, each residual block includes an activation function layer and at least one second convolutional layer, and the initial convolutional neural network module is used to perform feature extraction on the first preprocessing feature set and the second preprocessing feature set to obtain an image feature set corresponding to the first preprocessing feature set and a fusion feature set corresponding to the second preprocessing feature. Similar to step S1311 and step S1312, each feature vector of the feature set needs to be processed, which is not repeated here.
[0065] Step S232, using the initial recurrent neural network module to perform correlation processing on the image feature set and the fusion feature set to obtain an output feature set.
[0066] In one embodiment, the initial recurrent neural network module is used to extract the correlation between the image feature set and the fusion feature set to obtain the output feature set. Similar to step S1321 and step S1322, each feature vector of the feature set needs to be processed, which will not be described in detail here.
[0067] Step S233, classify the output feature set and output multiple predicted image classifications.
[0068] In one embodiment, a preset classification algorithm is used to classify each output feature in the output feature set, and multiple predicted image classifications are output, which helps doctors make diagnoses based on the classification results and reduces the workload of doctors. The classification algorithm can be softmax(), which is used to calculate the probability of the output feature set. For multiple training data, the probabilities corresponding to each training data are weighted and summed and then averaged to obtain a comprehensive probability value. The categories with a comprehensive probability greater than a preset threshold are classified as risky, and the categories with a comprehensive probability less than or equal to the preset threshold are classified as risk-free.
[0069] Step S234, calculating the value of the loss function according to each predicted image classification and target classification, and using the value of the loss function to adjust the parameters of the initial detection model to obtain the target detection model.
[0070] In one embodiment, the target classification is manually labeled, and a preset loss function is used to calculate the loss of the predicted image classification and the target classification to obtain the value of the loss function, which is conducive to subsequent back propagation according to the value of the loss function to perform model training. The preset loss function can be a logarithmic function, a cross entropy function, etc.
[0071] In another embodiment, since the initial detection model includes a preset initial convolutional neural network module and a preset initial recurrent neural network module, the parameters of the initial detection model are adjusted using the value of the loss function as follows: through multiple rounds of training, each round of training is counted, and in the first round of training, the parameters of the initial recurrent neural network module and the initial convolutional neural network module are adjusted in turn using the value of the loss function; in the second round of training, the parameters of the initial recurrent neural network module are adjusted using the value of the loss function, and the initial convolutional neural network module is not adjusted; in the third round of training, the initial recurrent neural network module is not adjusted, and the parameters of the initial convolutional neural network module are adjusted using the value of the loss function. The fourth round corresponds to the adjustment method of the first round, the fifth round corresponds to the adjustment method of the second round, and the sixth round corresponds to the adjustment method of the third round. The parameters of the initial detection model are adjusted in a cycle of three rounds to obtain a target detection model, so that the model can fully fit the training data. By continuously adjusting the parameters, the generalization of the target detection model can be better, thereby improving the accuracy of detection.
[0072] like Figure 5As shown, an embodiment of the present application provides a renal image processing device 100 based on heterogeneous data. The device 100 acquires heterogeneous data through a data acquisition module 110. The heterogeneous data includes renal medical images to be detected, blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images. By acquiring heterogeneous data of different indicators, the detection is made more comprehensive and the classification accuracy is improved; then the data preprocessing module 120 is used to preprocess the renal medical images to obtain image preprocessing features, and the blood oxygen level dependent imaging images, arterial spin imaging images, diffusion weighted imaging images and diffusion tensor imaging images are fused preprocessed to obtain fused preprocessing features. By performing image preprocessing, the representation ability of the fused preprocessing features is made stronger and more comprehensive information is captured; through model processing The processing module 130 inputs the image preprocessing features and the fusion preprocessing features into a preset target detection model, and performs the following processing: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; the first processing module 131 uses the convolutional neural network module to perform feature extraction on the image preprocessing features and the fusion preprocessing features, and obtains kidney image features corresponding to the image preprocessing features and first fusion features corresponding to the fusion preprocessing features; the second processing module 132 uses the recurrent neural network module to perform correlation processing on the kidney image features and the first fusion features, and obtains the second fusion features, which can extract the contextual relationship of pixels and is beneficial to improving the detection accuracy; the classification detection module 133 is used to classify the second fusion features, and output the image classification results, and the classification accuracy is high.
[0073] It should be noted that the data acquisition module 110 is connected to the data preprocessing module 120, the data preprocessing module 120 is connected to the model processing module 130, the model processing module 130 includes a first processing module 131, a second processing module 132 and a classification detection module 133, the first processing module 131 is connected to the second processing module 132, and the second processing module 132 is connected to the classification detection module 133. The above-mentioned kidney image processing method based on heterogeneous data is applied to the kidney image processing device 100 based on heterogeneous data. The kidney image processing device 100 based on heterogeneous data uses heterogeneous data reflecting different levels, first preprocesses the heterogeneous data, and then uses different neural network models to extract features from the preprocessed data, and then performs feature fusion to obtain the detection result, and captures different information according to the data at different levels to improve the accuracy of the detection.
[0074] It should also be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0075] The present application also discloses an electronic device. Figure 6 , Figure 6 The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .
[0076] The communication bus 502 is used to realize the connection and communication between these components.
[0077] The user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.
[0078] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0079] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.
[0080] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. Refer to Figure 6 , the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a kidney image processing method based on heterogeneous data.
[0081] exist Figure 6In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 501 can be used to call an application program stored in the memory 505 for a method for processing kidney images based on heterogeneous data. When executed by one or more processors 501, the electronic device 500 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described order of actions, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.
[0082] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0083] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0086] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.
[0087] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.
[0088] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A kidney image processing method based on heterogeneous data, characterized in that: The method comprises: Acquiring heterogeneous data, wherein the heterogeneous data includes a renal medical image to be detected, a blood oxygen level dependent imaging image, an arterial spin imaging image, a diffusion weighted imaging image, and a diffusion tensor imaging image; Preprocessing the renal medical image to obtain image preprocessing features, and fusion preprocessing the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image, and the diffusion tensor imaging image to obtain fusion preprocessing features; The image preprocessing features and the fusion preprocessing features are input into a preset target detection model, and the following processing is performed: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; Using the convolutional neural network module to perform feature extraction on the image preprocessing feature and the fusion preprocessing feature to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature; Using the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain a second fusion feature; Classify the second fusion feature and output an image classification result.
2. The method according to claim 1, characterized in that The convolutional neural network module includes a plurality of residual blocks and a first convolution layer, each residual block includes an activation function layer and at least one second convolution layer; The method of using the convolutional neural network module to extract features from the image preprocessing features and the fusion preprocessing features to obtain a renal image feature corresponding to the image preprocessing features and a first fusion feature corresponding to the fusion preprocessing features includes: Using the second convolution layer to extract the image preprocessing features and the fusion preprocessing features, and using the activation function layer to perform nonlinear processing to obtain a first feature corresponding to the image preprocessing features and a second feature corresponding to the fusion preprocessing features; The first convolutional layer is used to perform feature extraction processing on the first feature and the second feature respectively to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature, wherein the convolution kernel of the first convolutional layer is 1×1.
3. The method according to claim 1, characterized in that The step of performing fusion preprocessing on the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image to obtain fusion preprocessing features includes: Performing feature vector conversion on the blood oxygen level dependence imaging image to obtain a first feature vector; Performing feature vector conversion on the arterial spin imaging image to obtain a second feature vector; Performing feature vector conversion on the diffusion weighted imaging image to obtain a third feature vector; Performing feature vector conversion on the diffusion tensor imaging image to obtain a fourth feature vector; Perform an attention mechanism fusion on the first feature vector, the second feature vector, the third feature vector and the fourth feature vector to obtain the fused preprocessing feature.
4. The method according to claim 3, characterized in that The performing attention mechanism fusion on the first feature vector, the second feature vector, the third feature vector and the fourth feature vector to obtain the fusion preprocessing feature includes: Obtaining a preset first weight value, a preset second weight value, a preset third weight value, and a preset fourth weight value, wherein the sum of the first weight value, the second weight value, the third weight value, and the fourth weight value is 1; Multiply the first eigenvector by the first weight value to obtain a first product, multiply the second eigenvector by the second weight value to obtain a second product, multiply the third eigenvector by the third weight value to obtain a third product, and multiply the fourth eigenvector by the fourth weight value to obtain a fourth product; The first product, the second product, the third product and the fourth product are connected and normalized to obtain the fusion preprocessing feature.
5. The method according to claim 1, characterized in that The using the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain a second fusion feature includes: Performing feature connection on the renal image feature and the first fusion feature to obtain a third fusion feature; The recurrent neural network module is used to perform feature extraction processing on the third fusion feature to obtain the second fusion feature.
6. The method according to claim 1, characterized in that Before inputting the image preprocessing features and the fusion preprocessing features into a preset target detection model, the method further includes: Acquire a heterogeneous data set, the heterogeneous data set comprising a plurality of training renal images, training blood oxygen level dependent imaging images, training arterial spin imaging images, training diffusion weighted imaging images and training diffusion tensor imaging images corresponding to each of the training renal images, and target classifications corresponding to each of the training renal images; Preprocessing each of the training renal images to obtain a first preprocessing feature set, and fusing the training blood oxygen level dependent imaging image, the training arterial spin imaging image, the training diffusion weighted imaging image, and the training diffusion tensor imaging image corresponding to each of the training renal images to obtain a second preprocessing feature set; Inputting the first preprocessing feature set and the second preprocessing feature set into a preset initial detection model, and performing the following processing: the initial detection model includes a preset initial convolutional neural network module and a preset initial recurrent neural network module; Using the initial convolutional neural network module to perform feature extraction on the first preprocessing feature set and the second preprocessing feature set to obtain an image feature set corresponding to the first preprocessing feature set and a fusion feature set corresponding to the second preprocessing feature set; Using the initial recurrent neural network module to perform correlation processing on the image feature set and the fusion feature set to obtain an output feature set; Performing classification processing on the output feature set and outputting a plurality of predicted image classifications; The value of the loss function is calculated according to each predicted image classification and the target classification, and the value of the loss function is used to adjust the parameters of the initial detection model to obtain the target detection model.
7. The method according to claim 6, characterized in that The step of calculating a loss function value according to the predicted image classification and the target classification, and adjusting parameters of the initial detection model using the loss function value to obtain the target detection model includes: Using a preset loss function to perform loss calculation on the predicted image classification and the target classification to obtain a value of the loss function; In the first round of training, the value of the loss function is used to adjust the parameters of the initial recurrent neural network module and the initial convolutional neural network module in turn. In the second round of training, the value of the loss function is used to adjust the parameters of the initial recurrent neural network module, and the initial convolutional neural network module is not adjusted. In the third round of training, the initial recurrent neural network module is not adjusted, and the value of the loss function is used to adjust the parameters of the initial convolutional neural network module. In multiple rounds of training, the first round, the second round, and the third round are cycled to adjust the parameters of the initial recurrent neural network module and the initial convolutional neural network module to obtain the target detection model.
8. A kidney image processing device based on heterogeneous data, characterized in that: The device comprises: A data acquisition module (110) is used to acquire heterogeneous data, wherein the heterogeneous data includes a renal medical image to be detected, a blood oxygen level dependent imaging image, an arterial spin imaging image, a diffusion weighted imaging image, and a diffusion tensor imaging image; A data preprocessing module (120) is used to preprocess the renal medical image to obtain image preprocessing features, and to perform fusion preprocessing on the blood oxygen level dependent imaging image, the arterial spin imaging image, the diffusion weighted imaging image and the diffusion tensor imaging image to obtain fusion preprocessing features; A model processing module (130) is used to input the image preprocessing features and the fusion preprocessing features into a preset target detection model to perform the following processing: the target detection model includes a preset convolutional neural network module and a preset recurrent neural network module; A first processing module (131) is used to perform feature extraction on the image preprocessing feature and the fusion preprocessing feature using the convolutional neural network module to obtain a kidney image feature corresponding to the image preprocessing feature and a first fusion feature corresponding to the fusion preprocessing feature; A second processing module (132) is used to use the recurrent neural network module to perform correlation processing on the renal image feature and the first fusion feature to obtain a second fusion feature; A classification detection module (133) is used to perform classification processing on the second fusion feature and output an image classification result.
9. An electronic device, characterized in that: The electronic device (500) comprises a processor (501), a memory (505), a user interface (503), a communication bus (502) and a network interface (504), wherein the processor (501), the memory (505), the user interface (503) and the network interface (504) are respectively connected to the communication bus (502), the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device (500) executes the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.
