Model-based lesion image recognition method, device and electronic device
Through the cascade network architecture and feature pyramid network structure, the multi-parameter MRI image is segmented, which solves the problem of low accuracy of segmentation in the lesion area, improves the accuracy and robustness of segmentation, and provides more accurate lesion judgment and treatment plan. .
Patent Information
- Application Number
- CN202411252147.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-09
AI Technical Summary
The prior art has problems such as low accuracy, insufficient adaptability and limited multimodal data processing capabilities in lesion area segmentation, especially in complex organ tissues and multi-lesion cases, which are prone to missed or missed detection.
Using a cascading network architecture, including the first segmentation network and the second segmentation network, multi-parameter MRI images are segmented through a joint convolutional neural network and a multi-layer perceptron, using feature pyramid network structure and multiple repeat semantic extraction module, Combining morphological operations and regional merging improves segmentation accuracy and robustness.
It significantly improves the segmentation accuracy of the lesion area and the reliability of segmentation results, provides more accurate judgments on the location, range and morphology of the lesion, helps to formulate more effective treatment plans, and improves the accuracy of diagnosis and treatment.
Smart Images

Figure CN118762048B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent medical control, and in particular to a model-based lesion image recognition method, device and electronic equipment. Background Art
[0002] With the rapid development of medical imaging technology, locating the lesion area through medical imaging has become an important means of auxiliary treatment.
[0003] Currently, the lesion area is mainly determined by deep learning models and other methods to achieve automatic segmentation of organs and lesions.
[0004] However, this method has the problem of low accuracy in lesion area segmentation. Summary of the invention
[0005] The present application provides a model-based lesion image recognition method, device and electronic device to solve the problem of low lesion area segmentation accuracy.
[0006] In a first aspect, the present application provides a model-based lesion image recognition method, comprising:
[0007] Acquire an initial image; the initial image represents an image directly acquired by a medical device;
[0008] The initial image is processed using a first segmentation model to identify the position of the organ image region in the initial image; wherein the first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder; the encoder is used to convert the initial image into a first feature image of different resolutions; the bottleneck layer is used to determine the position of features related to the organ image region in the first feature images of different resolutions through semantic recognition; the decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images of different resolutions;
[0009] According to the position of the organ image region in the initial image obtained by identification, the initial image is cropped to obtain a cropped image including the organ image region;
[0010] Processing the cropped image using a second segmentation model to identify the location of the lesion image region in the organ image region; wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has multiple repeated semantic extraction modules;
[0011] According to the identified positions of the organ image region and the lesion image region, marking processing is performed in the initial image to obtain a final lesion image.
[0012] In a second aspect, the present application provides a model-based lesion image recognition device, comprising:
[0013] An acquisition module, used for acquiring an initial image; the initial image represents an image directly acquired by a medical device;
[0014] A segmentation module, used for processing the initial image using a first segmentation model to identify the position of the organ image region in the initial image; wherein the first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder; the encoder is used to convert the initial image into a first feature image of different resolutions; the bottleneck layer is used to determine the position of features related to the organ image region in the first feature images of different resolutions through semantic recognition; the decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images of different resolutions; according to the position of the organ image region in the initial image obtained by identification, the initial image is cropped to obtain a cropped image containing the organ image region; the cropped image is processed using a second segmentation model to identify the position of the lesion image region in the organ image region; wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has a semantic extraction module repeated multiple times;
[0015] The marking module is used to perform marking processing in the initial image according to the positions of the organ image area and the lesion image area obtained by identification, so as to obtain a final lesion image.
[0016] In a third aspect, the present application provides an electronic device, including: a memory and a processor;
[0017] The memory is used to store a computer program; the processor is used to execute the model-based lesion image recognition method in the first aspect and any possible design of the first aspect according to the computer program stored in the memory.
[0018] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the model-based lesion image recognition method in the first aspect and any possible design of the first aspect.
[0019] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When at least one processor of an electronic device executes the computer program, the electronic device executes the model-based lesion image recognition method in the first aspect and any possible design of the first aspect.
[0020] The model-based lesion image recognition method, device and electronic device provided in the present application obtain an initial image by acquiring and collecting; by inputting the initial image into the first segmentation model, the organ image area in the initial image is recognized and the position of the organ image area is determined; according to the position of the organ image area, a cropped image containing the organ image area is cropped from the initial image; by inputting the cropped image into the second segmentation model, the lesion image area in the cropped image is recognized; according to the position of the organ image area and the position of the lesion image area, the organ image area and the lesion image area are marked in the initial image to obtain a means of obtaining a lesion image, thereby achieving the effect of improving the segmentation accuracy of the lesion area and improving the reliability of the segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0022] Figure 1 A flowchart of an execution of a model-based lesion image recognition method provided in an embodiment of the present application;
[0023] Figure 2 A flow chart of a model-based lesion image recognition method provided in one embodiment of the present application;
[0024] Figure 3 A flow chart of a model-based lesion image recognition method provided in one embodiment of the present application;
[0025] Figure 4 A schematic diagram of the structure of a lesion image processing model provided in one embodiment of the present application;
[0026] Figure 5 A schematic diagram of the structure of a semantic extraction module provided in one embodiment of the present application;
[0027] Figure 6 A schematic diagram of the structure of a supplementary attention module provided in one embodiment of the present application;
[0028] Figure 7A schematic diagram of the structure of a model-based lesion image recognition device provided in one embodiment of the present application;
[0029] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0031] With the rapid development of medical imaging technology, locating the lesion area through medical imaging has become an important means of auxiliary treatment. At present, the existing lesion area segmentation method based on multiparametric magnetic resonance imaging (mpMRI) is to use the Faster-RCNN network to locate organs and lesions on 68Ga-PSMA-11 PET / MR images, and then use the U-Net network to perform fine segmentation of lesions, thereby achieving organ and lesion segmentation. This method combines multimodal data of T2 weighted imaging (T2WI), diffusion weighted imaging (DWI), apparent diffusion coefficient (ADC) and positron emission tomography (PET).
[0032] Although these methods have made some progress in the segmentation of lesion areas, there are still some shortcomings in the segmentation accuracy. For example, for complex and changeable organ tissues, although the use of deep learning models can improve the segmentation accuracy, in some cases, it is still easy to produce false positives and lead to misdiagnosis. In addition, the processing capacity of multimodal data in the existing technology is limited. In the process of information fusion and comprehensive utilization between modalities such as T2WI, DWI, and ADC, it is often impossible to give full play to the advantages of each modality, resulting in unsatisfactory segmentation effect. In addition, the existing technology has poor adaptability to different lesion morphologies. The morphology and size of the lesion area vary greatly. Existing deep learning models often show insufficient adaptability when dealing with this diversity. For example, when facing multi-lesion cases, the segmentation results of some models are not stable enough, and missed detection or misdetection are prone to occur.
[0033] In view of the above technical problems, the model-based lesion image recognition method provided in the embodiment of the present application designs a cascade network architecture by combining multiple MRI parameters such as T2-weighted images, diffusion-weighted images, and apparent diffusion coefficient images. The cascade network architecture includes a first segmentation network and a second segmentation network. Among them, the first segmentation network is used to segment the organ image area. For example, the organ image area can be the area of the organ where the lesion is located. Among them, the second segmentation network is used to segment the lesion image area. For example, the lesion image area can be the area where the lesion is located. The use of the two-stage segmentation network in the cascade network architecture improves the accuracy and robustness of the segmentation.
[0034] Specifically, the process can be as follows Figure 1 As shown. The electronic device can input the initial image into the first segmentation network. The first segmentation network first segments the organ image area in the initial image through a segmentation architecture of a joint convolutional neural network (CNN) and a multilayer perceptron (MLP). The first segmentation network can also use multimodal MRI images to extract the mask of the organ image area. The first segmentation network can also cut out a cropped image containing the organ image area based on the mask. The cropped image is the region of interest (ROI) of the first segmentation network. The electronic device can input the cropped image into the second segmentation network, effectively reducing the interference of non-related areas in the image, concentrating computing resources on the area of the organ where the lesion is located, and improving the accuracy of subsequent segmentation. The second segmentation network uses a deeper CNN-MLP network to segment the lesion image area based on the ROI. The network further uses multi-parameter MRI images to refine and segment the lesion image area, thereby enhancing the recognition ability of the lesion image area. Based on the segmentation results of the first segmentation network and the second segmentation network, the electronic device can fuse the final lesion image. Moreover, during the fusion process, the electronic device can further optimize the accuracy and consistency of the segmentation results through post-processing such as morphological operations and region merging.
[0035] The first segmentation network and the second segmentation network of the present application both adopt a feature pyramid network structure. In addition, the present application makes full use of the segmentation results of the first segmentation network, and improves the accuracy of lesion segmentation by processing the segmentation results in the second segmentation model. The present application fuses the segmentation results of the first segmentation model and the second segmentation model through operations such as morphological operations and regional merging, and the lesion segmentation image finally generated provides a reliable imaging basis and high-quality reference for clinical diagnosis and treatment. The present application helps doctors to more accurately judge the location, range and morphology of lesions through the high-precision lesion segmentation image, so as to formulate a more effective treatment plan, significantly improve the patient's prognosis and quality of life, and improve the accuracy of diagnosis and treatment.
[0036] The electronic device is used as the execution subject to execute the model-based lesion image recognition method of the following embodiment. Specifically, the execution subject can be a hardware device of the electronic device, or a software application that implements the following embodiment in the electronic device, or a computer-readable storage medium installed with the software application that implements the following embodiment, or a code that implements the software application of the following embodiment.
[0037] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0038] Figure 2 FIG. 1 is a flow chart of a model-based lesion image recognition method provided by an embodiment of the present application. Figure 2 As shown, with the electronic device as the execution subject, the method of this embodiment may include the following steps:
[0039] S101, acquiring an initial image. The initial image represents an image directly acquired by a medical device.
[0040] In this embodiment, the electronic device can acquire one or more initial images. The initial image may include an organ image area. The organ image area may include a lesion image area. The initial image may be an image directly acquired by a medical device. Optionally, the electronic device may be a medical device that acquires the initial image. Alternatively, the electronic device may acquire the initial image from a medical device that acquires the initial image. For example, the initial image may be a multi-parameter MRI image acquired before surgery. For example, the organs included in the initial image may be prostate tissue, liver tissue, stomach tissue, or intestinal tissue.
[0041] S102, using the first segmentation model to process the initial image, and identify the position of the organ image region in the initial image. The first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder. The encoder is used to convert the initial image into first feature images of different resolutions. The bottleneck layer is used to determine the position of features related to the organ image region in the first feature images of different resolutions through semantic recognition. The decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images of different resolutions.
[0042] In this embodiment, a first segmentation model may be pre-stored in the electronic device. Optionally, the first segmentation model is a trained artificial intelligence model. The electronic device may recognize the organ image region in the initial image by inputting the initial image into the first segmentation model. Optionally, the electronic device may obtain a mask of the organ image region by identifying the organ image region. Optionally, the electronic device may also obtain a position of the organ image region by identifying the organ image region.
[0043] Among them, the first segmentation model can specifically include three parts: an encoder, a bottleneck layer and a decoder. The electronic device can realize the recognition of the organ image area in the initial image by using the encoder, the bottleneck layer and the decoder. Optionally, the first segmentation model can use CNN-MLP to segment the multi-parameter MRI image. For example, when the organ is a prostate, the electronic device can obtain the entire prostate tissue area through the first segmentation model. For another example, when the organ is a liver, the electronic device can obtain the entire liver tissue area through the first segmentation model.
[0044] Specifically, the encoder is used to convert the initial image into first feature images of different resolutions. Optionally, the encoder can use a CNN network architecture. The encoder can include multiple convolution kernels of different sizes. The use of the convolution kernels of different sizes can achieve the extraction of different features in the initial image. The encoder can be preset with a convolution step size. For example, the convolution step size can be 2. By using the encoder to perform multiple convolutions, the electronic device can obtain first feature images of different resolutions.
[0045] The bottleneck layer is used to determine the location of features related to the organ image region in the first feature image of different resolutions through semantic recognition. The multi-layer perceptron (MLP) can be used to perceive the features related to the organ image region in the first feature image and determine the location of these features. Optionally, the MLP can implement semantic recognition.
[0046] The decoder is used to determine the position of the organ image region in the initial image according to the position of the features related to the organ image region in the first feature images of different resolutions. The setting of the decoder can also use the CNN network architecture. The activation function and convolution kernel in the decoder correspond to the encoder settings.
[0047] S103 , according to the position of the organ image region in the initial image obtained by identification, the initial image is cropped to obtain a cropped image including the organ image region.
[0048] In this embodiment, after determining the position of the organ image area in the initial image, the electronic device can locate the organ image area in the initial image according to the position. The electronic device can crop the cropped image containing the organ image area from the electronic device according to a preset cropping rule. For example, the electronic device can obtain the minimum circumscribed rectangle of the organ image area, and crop the cropped image according to the minimum circumscribed rectangle. Alternatively, the electronic device can determine the boundary of the organ image area through the mask of the organ image area, and crop the organ image area according to the boundary. The electronic device can add the cropped organ image area to a blank picture of a preset size to obtain the cropped image.
[0049] In one example, a specific process of the electronic device cropping to obtain a cropped image may include:
[0050] Step 11: Determine the boundary of the organ image region according to the position of the organ image region in the initial image obtained by identification.
[0051] In this step, the electronic device can identify the position of the organ image region in the initial image through the first segmentation model. The electronic device can determine the ROI region based on the position of the organ image region. For example, the ROI region can be a region containing prostate tissue. Or, the ROI region can be a region containing liver tissue. The electronic device can determine the boundary of the organ image region in the initial image based on the ROI region.
[0052] Step 12: Expand the boundary of the organ image region according to preset parameters, wherein the preset parameters are determined according to the anatomical structure of the organ image region and the resolution of the initial image.
[0053] In this step, in order to ensure that the organ image area is completely contained in the ROI and reduce the impact of the boundary effect, the electronic device can appropriately expand the ROI boundary determined in step 11. Specifically, the electronic device can determine the preset parameters according to the anatomical structure of the organ image area and the resolution of the initial image, and according to the preset parameters, realize the expansion of the boundary. For example, for the prostate area, the preset parameters can be adjusted according to the anatomical structure of the prostate and the image resolution. For example, for the liver area, the preset parameters can be adjusted according to the anatomical structure of the liver and the image resolution. Optionally, the preset parameters can be used to determine the scope of expansion.
[0054] Step 13: According to the boundary of the expanded organ image region, the initial image is cropped to obtain a cropped image corresponding to the organ image region.
[0055] In this step, the electronic device can crop the initial image according to the boundary of the expanded organ image area to obtain a cropped image corresponding to the organ image area. This step can effectively reduce the interference of non-related areas in the image, so that the subsequent second segmentation model can focus on the fine segmentation of the organ image area.
[0056] Optionally, after obtaining the cropped image, the electronic device can also align the multimodal cropped image. Specifically, since the initial image of the present application is a multimodal image including T2WI, DWI, ADC and other modalities, in order to ensure the use effect of the cropped image, the electronic device can also align the multimodal image by unifying the resolution to obtain the final cropped image.
[0057] S104, using a second segmentation model to process the cropped image and identify the position of the lesion image region in the organ image region, wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has multiple repeated semantic extraction modules.
[0058] In this embodiment, a second segmentation model may be pre-stored in the electronic device. Optionally, the second segmentation model is a trained artificial intelligence model. The model architecture of the second segmentation model is the same as the first segmentation model, including three parts: an encoder, a bottleneck layer, and a decoder. The model parameters of the second segmentation model are different from those of the first segmentation model. In addition, the bottleneck layer of the second segmentation model may include a semantic extraction module that is repeated multiple times. Optionally, the bottleneck layer uses a semantic extraction module that is repeated 12 times. This design enables the second segmentation model to extract features at a deeper level, thereby better identifying and segmenting the lesion image area. Optionally, the lesion image area is the lesion area.
[0059] The electronic device can input the cropped image into the second segmentation model to realize the recognition of the lesion image area in the cropped image. Optionally, the electronic device can obtain the mask of the lesion image area by recognizing the lesion image area. Optionally, the electronic device can also obtain the position of the lesion image area by recognizing the lesion image area.
[0060] S105 , marking the initial image according to the positions of the organ image region and the lesion image region obtained by recognition, to obtain a final lesion image.
[0061] In this embodiment, the electronic device can fuse the organ image area identified by the first segmentation network and the lesion image area identified by the second segmentation network. Specifically, the electronic device can mark the organ image area in the initial image according to the position of the organ image area. The electronic device can mark the lesion image area in the initial image according to the position of the lesion image area. Optionally, the marks of the organ image area and the lesion image area can be covered with different colors to achieve marking. Alternatively, the marks of the organ image area and the lesion image area can be circled with different colors to achieve marking. The electronic device can mark the final lesion image. The electronic device can display the lesion image. The display of the lesion image can provide a reliable reference for clinical diagnosis and treatment, and assist doctors in completing treatment. The fusion mark of the position of the organ image area and the position of the lesion image area further optimizes the accuracy and consistency of the segmentation results.
[0062] The model-based lesion image recognition method provided in the present application can be used for an electronic device to acquire an initial image. The electronic device can recognize the organ image area in the initial image and determine the position of the organ image area by inputting the initial image into the first segmentation model. The electronic device can crop a cropped image containing the organ image area from the initial image according to the position of the organ image area. The electronic device can recognize the lesion image area in the cropped image by inputting the cropped image into the second segmentation model. The electronic device can mark the organ image area and the lesion image area in the initial image according to the position of the organ image area and the position of the lesion image area to obtain a lesion image. In the present application, by using a multimodal MRI image as the initial image and using the first segmentation network and the second segmentation network for cascade segmentation, the segmentation accuracy of the lesion area is improved and the reliability of the segmentation result is improved.
[0063] Figure 3 FIG. 1 is a flow chart of a model-based lesion image recognition method provided by an embodiment of the present application. Figure 2 Based on the embodiment shown, Figure 3As shown, with the electronic device as the execution subject, the specific process of the first segmentation model completing the position recognition of the organ image area may include the following steps:
[0064] S201, inputting an initial image into an encoder, and extracting first feature images of different resolutions through a dual convolution module in the encoder. The dual convolution module includes an activation function and a plurality of convolution kernels of different sizes. The first feature image represents a feature map after the initial image is processed using the convolution kernel and the activation function.
[0065] In this embodiment, the electronic device can input the initial image into the encoder. The encoder may include a dual convolution module. The dual convolution module may include an activation function and a plurality of convolution kernels of different sizes. Optionally, the activation function may be a GELU activation function. Optionally, the dual convolution module can realize the extraction of shallow features of the initial image. Optionally, the dual convolution module may be provided with a fixed step size. Optionally, the step size of the dual convolution module may be 2. Optionally, the setting of the convolution kernel of this size and the step size can expand the receptive field of the network and strengthen the semantic parsing, thereby generating a first feature map of different spatial resolutions.
[0066] For example, Figure 4 As shown, the input image can be an initial image. The encoder can include 4 convolution kernels of different sizes. Each convolution kernel can correspond to a coding layer. The encoder can perform 4 convolutions on the initial image through the 4 convolution kernels. When the resolution of the initial image is, when passing through the first convolution kernel, the electronic device can convolve with a step size of 2 to obtain a first feature image with a resolution of. When the first feature image with a resolution of passes through the second convolution kernel, the electronic device can convolve with a step size of 2 to obtain a second first feature image with a resolution of. When the first feature image with a resolution of passes through the third convolution kernel, the electronic device can convolve with a step size of 2 to obtain a third first feature image with a resolution of. When the first feature image with a resolution of passes through the third convolution kernel, the electronic device can convolve with a step size of 2 to obtain a fourth first feature image with a resolution of. Optionally, the number of channels of can be set to 32, 64, 128 and 256 respectively.
[0067] S202: Input the first feature images with different resolutions into the bottleneck layer, and extract the second feature images through the semantic extraction module in the bottleneck layer. The second feature images have information on the positions of features related to the organ image region.
[0068] In this embodiment, the electronic device can input first feature images of different resolutions into the bottleneck layer. The bottleneck layer can first merge features of different resolutions into a first recognition feature map of uniform resolution by downsampling. Secondly, the bottleneck layer can obtain a second recognition feature map of uniform resolution by performing a semantic extraction module on the feature map of uniform resolution. The second recognition feature map includes information on the location of features related to the organ image area. The electronic device can split the features in the second recognition feature map according to the resolution of the first feature image to obtain multiple second feature images of different resolutions. The second feature image has information on the location of features related to the organ image area. Optionally, the bottleneck layer may include a semantic extraction module. Optionally, the semantic extraction module may be composed of a tokenized MLP layer.
[0069] In one example, the semantic extraction module includes multiple sequentially cascaded multi-layer perception units.
[0070] In one example, the specific execution process of the bottleneck layer may include the following steps. Figure 5 shown.
[0071] Step 21: Input the first feature image into a plurality of multi-layer perception units cascaded in sequence, and obtain the perceptual feature map output by each multi-layer perception unit, wherein the perceptual feature map output by the previous multi-layer perception unit is the input of the next multi-layer perception unit.
[0072] In this step, the electronic device may input the first feature image into a plurality of sequentially cascaded multi-layer perception units. The multi-layer perception unit may be a tokenized multi-layer perception (MLP) module. For example, Figure 5 The semantic extraction module shown may include three different tokenized MLP modules connected in sequence. The semantic extraction module can process the first feature image in sequence through the three MLP modules. Among them, the perceptual feature map output by the previous multi-layer perception unit is the input of the next multi-layer perception unit. The electronic device can obtain the perceptual feature map output by each multi-layer perception unit. This sequential processing ensures that the feature information can be effectively integrated and refined at different stages.
[0073] Step 22: Process the perceptual feature map output by each multi-layer perception unit through an upsampling unit so that the dimension of the perceptual feature map output by each multi-layer perception unit is consistent with the dimension of the perceptual feature map output by the first multi-layer perception unit.
[0074] In this step, the electronic device can upsample the perceptual feature map output by the multi-layer perception unit through the upsampling unit. Figure 5As shown, the electronic device can upsample the perceptual feature map output by the second multi-layer perception unit by ×2, and the electronic device can upsample the perceptual feature map output by the third multi-layer perception unit by ×4. Optionally, during the upsampling process, the electronic device can also perform convolution processing on the perceptual feature map output by the multi-layer perception unit. The dimensions of the perceptual feature maps output by the three multi-layer perception units after processing are the same. Moreover, the same dimension is determined according to the dimension of the perceptual feature map output by the first multi-layer perception unit.
[0075] Step 23: Fuse the perceptual feature maps output by the multi-layer perception units to obtain a second feature image.
[0076] In this step, the electronic device can superimpose and fuse the perceptual feature maps of the same dimension output by multiple multi-layer perception units to obtain a second feature image. The accumulation operation can enhance the complementarity of feature information and reduce the information loss that may be caused by the displacement operation. The second feature contains information about the location of features related to the organ image area.
[0077] S203, using a decoder to decode the second feature image, and obtain the position of the organ image region according to the position of the feature related to the organ image region in the decoded second feature image. The second feature image represents the feature map output by the bottleneck layer with the position of the feature related to the organ image region.
[0078] In this embodiment, the electronic device may use a decoder to decode the second feature image. The decoder may include a dual convolution module with a ReLU activation function. The decoder can be used to restore the feature details in the second feature image. During the decoding process, the electronic device may use a convolution kernel of the same size as the convolution kernel corresponding to the second feature image for decoding. During the decoding process, the electronic device can increase the resolution of the feature map by two times by upsampling. During the decoding process, the electronic device can fuse the second feature image from the MLP module with the decoder output to enhance the global understanding ability of the network.
[0079] In one example, the process of decoding the second feature image by the decoder may include the following steps:
[0080] Step 31: sequentially set a plurality of decoding layers in the decoder, wherein the decoding layers have convolution kernels, and the setting of the convolution kernels in the decoding layers corresponds to the use order of the convolution kernels in the dual convolution module.
[0081] In this step, the electronic device may be provided with multiple decoding layers. The setting of the decoding layers may correspond to the decoding layers of the encoder. The convolution kernels in the decoding layers may correspond to the convolution kernels in the decoding layers. For example, Figure 5As shown, four decoding layers may be arranged in the electronic device from bottom to top, and each decoding layer may correspond to one encoding layer.
[0082] Step 32: Use the convolution kernel of the decoding layer to decode the output feature map of the previous decoding layer and the second feature image corresponding to the decoding layer to obtain the output feature map of the decoding layer.
[0083] In this step, the electronic device can obtain the feature map output by the previous decoding layer and the second feature image corresponding to the decoding layer. The electronic device can use the convolution kernel of the decoding layer to decode the feature map output by the previous decoding layer and the second feature image corresponding to the decoding layer to obtain the output feature map of the decoding layer.
[0084] For example, Figure 4 As shown, the first decoding layer of the lower network can decode and upsample the second feature image with a resolution of and obtain an output feature map. The second decoding layer can decode and upsample the second feature image with a resolution of and the output feature map output by the first decoding layer to obtain an output feature map. The third decoding layer can decode and upsample the second feature image with a resolution of and the output feature map output by the second decoding layer to obtain an output feature map. The fourth decoding layer can decode and upsample the second feature image with a resolution of and the output feature map output by the third decoding layer to obtain an output feature map. The resolution of the output feature map of the fourth decoding layer is.
[0085] Step 33, obtaining the output feature map of the last decoding layer, and processing the output feature map using the activation function corresponding to the double convolution module to obtain the location of the features related to the organ image area. The resolution of the output feature map of the last decoding layer is the same as that of the initial image.
[0086] In this step, after obtaining the output feature map of the last decoding layer, the electronic device can use an activation function to process the output feature map to obtain the position of the feature related to the organ image area. Optionally, the activation function can be a Sigmoid activation function. Optionally, the electronic device can also perform convolution processing on the output feature map of the last decoding layer using a convolution kernel.
[0087] Step 34: Determine the organ image region in the initial image according to the position of the features related to the organ image region in the output feature map of the last decoding layer.
[0088] In this step, the electronic device can obtain the position of the features related to the organ image area from the output feature map of the last decoding layer, and determine the corresponding position of the features related to the organ image area in the initial image, thereby determining the organ image area in the initial image.
[0089] In one example, the first segmentation model also includes a supplementary attention module. The decoder can also introduce Figure 6 The supplementary attention module shown. The supplementary attention module can strengthen the focus on the organ image area and improve the segmentation accuracy. The supplementary attention module can be set at the output end of each decoding layer. The use of the supplementary attention module significantly improves the feature response of the low activation area during the recognition process, while keeping attention on the area with active response, enhancing the feature expression and information utilization efficiency of the entire network. Taking the supplementary attention module of a decoding layer as an example, the specific execution process of the decoding layer may include:
[0090] Step 41: Obtain a first feature image corresponding to the decoding layer.
[0091] In this step, the electronic device may obtain the first feature map from the encoding layer corresponding to the decoding layer according to the currently calculated decoding layer.
[0092] Step 42: construct a weighted attention map and a supplementary weight matrix according to the output feature map of the decoding layer and the first feature map.
[0093] In this step, the electronic device can obtain the decoded output feature map. The electronic device can process the output feature map through global average pooling, ReLU activation function and function to calculate the channel attention. The electronic device can calculate the product of the channel attention and the first feature map to obtain the first intermediate feature map.
[0094] The electronic device may also process the first intermediate feature map by average pooling, convolution and function to calculate the spatial attention. The electronic device may use the spatial attention to form a weighted attention map.
[0095] The electronic device can also process the output feature map to calculate a supplementary weight matrix.
[0096] Step 43: The electronic device may calculate the product of the first feature image and the weighted attention map to obtain a first interaction attention map.
[0097] Step 44: The electronic device may calculate the product of the supplementary weight matrix and the first interaction attention map to obtain a second interaction attention map.
[0098] Step 45: The electronic device may superimpose the second interactive attention map onto the output feature map of the decoding layer to obtain a new output feature map.
[0099] The model-based lesion image recognition method provided by the present application is that the electronic device can input the initial image into the encoder, and extract the first feature images of different resolutions through the dual convolution module in the encoder. The electronic device can input the first feature images of different resolutions into the bottleneck layer. The bottleneck layer can obtain the second recognition feature map by semantic extraction of the first feature map. The electronic device can decode the second feature image using a decoder to obtain the location of the features related to the organ image area in the second feature image, and then determine the location of the organ image area. In the present application, through the characteristics of CNN and MLP, a balance is achieved in the fusion of global and local features, thereby improving the accuracy and robustness of segmentation.
[0100] Based on the above embodiments, with the electronic device as the execution subject, the training process of the first segmentation model and the second segmentation model in this embodiment may include the following steps:
[0101] S301 , obtaining training data, wherein the training data includes a lesion image with position marks of an organ image region and a lesion image region.
[0102] In this embodiment, the electronic device needs to obtain a training data set. The training data set may include multiple lesion graphics. The lesion image may include a multimodal MRI image. Each lesion image includes at least multiple MRI parameter images such as T2WI, DWI, ADC, etc. The lesion image also needs to include the position of the organ image area and the position mark of the lesion image area. Optionally, the position of the organ image area and the position mark of the lesion image area may be a manually annotated segmentation mask.
[0103] Optionally, the electronic device may preprocess the training data. The preprocessing may include image normalization, data enhancement, etc. Among them, data enhancement may include rotation, translation, scaling, etc. The execution of the preprocessing may enable the trained model to have better robustness and generalization ability.
[0104] S302: Use training data to train a first segmentation model and a second segmentation model.
[0105] In this embodiment, the electronic device can construct a first segmentation model and a second segmentation model based on the above structure. The electronic device can input the preprocessed training data into the first segmentation model to obtain the position of the organ image area. The electronic device can segment the organ image area according to the position of the organ image area to obtain a cropped image. The electronic device can input the cropped image into the second segmentation model. The second segmentation model can perform fine segmentation based on the cropped image to obtain the position of the lesion image area.
[0106] The electronic device can use the Dice loss function or the cross entropy loss function to measure the difference between the segmentation result and the true mask, thereby guiding the update of the model parameters. Optionally, the true mask is a manually annotated segmentation mask. The electronic device can use the Adam optimizer or other suitable optimization algorithms to iteratively update the model parameters and gradually improve the segmentation accuracy. After each training cycle, the electronic device can use the validation set data to evaluate the model performance, adjust the hyperparameters and network structure, and avoid overfitting and underfitting.
[0107] Optionally, after the training is completed, the electronic device can also use the test set data to perform a final evaluation on the model, calculate indicators such as the Dice coefficient, precision, and recall, and comprehensively evaluate the segmentation performance of the first segmentation model and the second segmentation model. Optionally, the electronic device can use the first segmentation model and the second segmentation model after the segmentation performance of the first segmentation model and the second segmentation model meets the requirements.
[0108] On the basis of the above embodiment, taking cancer tissue segmentation as an example, the execution process of the above model-based lesion image recognition method may include the following steps:
[0109] S401. Multi-parameter MRI image acquisition.
[0110] In this embodiment, before performing cancer lesion detection, the electronic device may collect a multimodal MRI image of the patient as an initial image, which may include T2WI, DWI, ADC, and the like.
[0111] S402: Organ region segmentation.
[0112] In this embodiment, the electronic device can segment the initial image to obtain the organ region of the cancer lesion. The electronic device can use the trained first segmentation network to segment the collected initial image to obtain the organ region. The organ region is the above-mentioned organ image region. Among them, the first segmentation network uses a combination of CNN and MLP to enhance the accuracy of feature extraction and segmentation.
[0113] S403: Obtain a cropped image.
[0114] In this embodiment, the electronic device can extract the ROI region of the organ containing the cancer lesion according to the segmentation result. The electronic device can ensure that the organ is completely contained in the ROI region by appropriately expanding the boundary of the initially segmented organ. The T2WI, DWI, ADC and other modalities in the cropped image obtained by cropping are spatially aligned.
[0115] S404. Organ and tissue segmentation of cancer lesions.
[0116] In this embodiment, the electronic device inputs the cropped image into a second segmentation network at a deeper level. The second segmentation network is a CNN-MLP network, which is used to perform fine segmentation on the lesions of cancerous lesions in the organ. The lesions are cancerous lesion tissues. The cancerous lesion tissues are the lesion image areas in the above embodiments.
[0117] S405: Segmentation result fusion and output.
[0118] In this embodiment, the electronic device can fuse the segmentation results of the first segmentation model and the second segmentation model to generate a final lesion image. The lesion image is a segmentation image of cancer lesion tissue. The electronic device can also visualize the lesion image. The lesion image can also provide a reliable reference for clinical diagnosis and treatment. For example, when the cancer is prostate cancer, the lesion image can be applied to a prostate puncture robot to achieve high-precision automatic segmentation of prostate cancer tissue, providing strong technical support for clinical diagnosis and surgical treatment.
[0119] Figure 7 FIG. 1 shows a schematic diagram of a structure of a model-based lesion image recognition device provided in an embodiment of the present application, such as Figure 7 As shown, the model-based lesion image recognition device 10 of this embodiment is used to implement the operation corresponding to the electronic device in any of the above method embodiments. The model-based lesion image recognition device 10 of this embodiment includes:
[0120] The acquisition module 11 is used to acquire an initial image; the initial image represents an image directly acquired by a medical device;
[0121] The segmentation module 12 is used to process the initial image using the first segmentation model to identify the position of the organ image region in the initial image; wherein the first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder; the encoder is used to convert the initial image into a first feature image of different resolutions; the bottleneck layer is used to determine the position of features related to the organ image region in the first feature images of different resolutions through semantic recognition; the decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images of different resolutions; according to the position of the organ image region in the initial image obtained by the identification, the initial image is cropped to obtain a cropped image containing the organ image region; the cropped image is processed using the second segmentation model to identify the position of the lesion image region in the organ image region; wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has a semantic extraction module repeated multiple times;
[0122] The marking module 13 is used to perform marking processing in the initial image according to the positions of the organ image area and the lesion image area obtained by identification, so as to obtain a final lesion image.
[0123] Optionally, the segmentation module 12 is used to:
[0124] The initial image is input into the encoder, and the first feature images of different resolutions are extracted through the double convolution module in the encoder; wherein the double convolution module includes an activation function and a plurality of convolution kernels of different sizes; the first feature image represents the feature map after the initial image is processed using the convolution kernel and the activation function.
[0125] Optionally, the segmentation module 12 is used to:
[0126] The first feature images with different resolutions are input into the bottleneck layer, and the second feature image is extracted by the semantic extraction module in the bottleneck layer; the second feature image has the information of the position of the features related to the organ image area.
[0127] Optionally, the semantic extraction module includes a plurality of sequentially cascaded multi-layer perception units; a cutting module 12, which is used to:
[0128] Inputting the first feature image into a plurality of multi-layer perception units cascaded in sequence, and obtaining a perceptual feature map output by each multi-layer perception unit; wherein the perceptual feature map output by the previous multi-layer perception unit is the input of the next multi-layer perception unit;
[0129] Processing the perceptual feature map output by each multi-layer perception unit through an upsampling unit so that the dimension of the perceptual feature map output by each multi-layer perception unit is consistent with the dimension of the perceptual feature map output by the first multi-layer perception unit;
[0130] The perceptual feature maps output by the multi-layer perception units are fused to obtain the second feature image.
[0131] Optionally, the segmentation module 12 is used to:
[0132] The second feature image is decoded using a decoder, and the position of the organ image area is obtained according to the position of the features related to the organ image area in the decoded second feature image; the second feature image represents a feature map of the position of the features related to the organ image area output by the bottleneck layer.
[0133] Optionally, the segmentation module 12 is used to:
[0134] A plurality of decoding layers are sequentially arranged in the decoder, each of the decoding layers having a convolution kernel, and the arrangement of the convolution kernel in the decoding layer corresponds to the use order of the convolution kernel in the dual convolution module;
[0135] Using the convolution kernel of the decoding layer, the output feature map of the previous decoding layer and the second feature image corresponding to the decoding layer are decoded to obtain the output feature map of the decoding layer;
[0136] Obtaining the output feature map of the last decoding layer, and processing the output feature map using the activation function corresponding to the double convolution module to obtain the location of features related to the organ image region; wherein the resolution of the output feature map of the last decoding layer is the same as that of the initial image;
[0137] The organ image region is determined in the initial image according to the position of the features related to the organ image region in the output feature map of the last decoding layer.
[0138] Optionally, the first segmentation model further includes a supplementary attention module; a segmentation module 12, configured to:
[0139] Obtaining a first feature image corresponding to a decoding layer;
[0140] According to the output feature map of the decoding layer and the first feature map, a weight attention map and a supplementary weight matrix are constructed;
[0141] Calculate the product of the first feature image and the weighted attention map to obtain the first interaction attention map.
[0142] Calculate the product of the supplementary weight matrix and the first interactive attention map to obtain a second interactive attention map;
[0143] The second interactive attention map is superimposed on the output feature map of the decoding layer to obtain a new output feature map.
[0144] Optionally, the segmentation module 12 is used to:
[0145] Determine the boundary of the organ image region according to the position of the organ image region in the initial image obtained by identification;
[0146] According to preset parameters, the boundary of the organ image region is expanded; wherein the preset parameters are determined according to the anatomical structure of the organ image region and the resolution of the initial image;
[0147] According to the boundary of the expanded organ image region, a cropped image corresponding to the organ image region is obtained by cropping the initial image.
[0148] Optionally, a training module is used to:
[0149] Acquire training data, wherein the training data includes a lesion image with position marks of an organ image region and a lesion image region;
[0150] The first segmentation model and the second segmentation model are trained using the training data.
[0151] The model-based lesion image recognition device 10 provided in the embodiment of the present application can execute the above method embodiment. Its specific implementation principles and technical effects can be found in the above method embodiment, and this embodiment will not be repeated here.
[0152] Figure 8 FIG. 1 shows a hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 20 is used to implement the operations corresponding to the electronic device in any of the above method embodiments. The electronic device 20 of this embodiment may include: a memory 21, a processor 22 and a communication interface 24.
[0153] The memory 21 is used to store computer programs. The memory 21 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a disk, or an optical disk.
[0154] The processor 22 is used to execute the computer program stored in the memory to implement the model-based lesion image recognition method in the above embodiment. For details, please refer to the relevant description in the above method embodiment. The processor 22 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.
[0155] Optionally, the memory 21 may be independent or integrated with the processor 22 .
[0156] When the memory 21 is a device independent of the processor 22, the electronic device 20 may further include a bus 23. The bus 23 is used to connect the memory 21 and the processor 22. The bus 23 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0157] The communication interface 24 may be connected to the processor 21 via the bus 23. The processor 22 may control the communication interface 24 to implement the function of communicating with the medical device that collects the initial image.
[0158] The electronic device provided in this embodiment can be used to execute the above-mentioned model-based lesion image recognition method. Its implementation method and technical effect are similar and will not be described in detail in this embodiment.
[0159] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the methods provided in the various embodiments described above.
[0160] Among them, the computer-readable storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, a computer-readable storage medium is coupled to a processor so that the processor can read information from the computer-readable storage medium and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can be located in an application-specific integrated circuit (ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the computer-readable storage medium can also exist in a communication device as discrete components.
[0161] Specifically, the computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium may be any available medium that can be accessed by a general or special-purpose computer.
[0162] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of a device can read the computer program from the computer-readable storage medium, and at least one processor executes the computer program so that the device implements the methods provided in the various embodiments described above.
[0163] An embodiment of the present application also provides a chip, which includes a memory and a processor, the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.
[0164] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0165] Among them, each module can be physically separated, for example, installed in different locations of a device, or installed on different devices, or distributed on multiple network units, or distributed on multiple processors. Each module can also be integrated together, for example, installed in the same device, or integrated in a set of codes. Each module can exist in the form of hardware, or can also exist in the form of software, or can also be implemented in the form of software plus hardware. The present application can select some or all of the modules according to actual needs to achieve the purpose of the present embodiment.
[0166] When each module is implemented as an integrated module in the form of a software function module, it can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of each embodiment of the present application.
[0167] It should be understood that, although the various steps in the flowcharts in the above-described embodiments are sequentially displayed according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily to be carried out sequentially, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A model-based lesion image recognition method, characterized in that: The method comprises: Acquire an initial image; the initial image represents an image directly acquired by a medical device; The initial image is processed using a first segmentation model to identify the position of the organ image region in the initial image; wherein the first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder; the encoder is used to convert the initial image into a first feature image of different resolutions; the bottleneck layer is used to determine the position of features related to the organ image region in the first feature images of different resolutions through semantic recognition; the decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images of different resolutions; According to the position of the organ image region in the initial image obtained by identification, the initial image is cropped to obtain a cropped image including the organ image region; The cropped image is processed using a second segmentation model to identify the position of the lesion image region in the organ image region; wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has multiple repeated semantic extraction modules, and the network structure of the second segmentation model is a feature pyramid network structure; According to the identified positions of the organ image region and the lesion image region, marking processing is performed in the initial image to obtain a final lesion image.
2. The method according to claim 1, characterized in that The converting the initial image into a first feature image with different resolutions comprises: The initial image is input into an encoder, and a first feature image of different resolutions is extracted through a dual convolution module in the encoder; wherein the dual convolution module includes an activation function and a plurality of convolution kernels of different sizes; and the first feature image represents a feature map after the initial image is processed using the convolution kernel and the activation function.
3. The method according to claim 1, characterized in that The determining, by semantic recognition, the positions of features related to the organ image region in the first feature images of different resolutions comprises: The first feature images of different resolutions are input into the bottleneck layer, and the second feature images are extracted by the semantic extraction module in the bottleneck layer; the second feature images have information on the positions of features related to the organ image area.
4. The method according to claim 3, characterized in that The semantic extraction module includes a plurality of sequentially cascaded multi-layer perception units; the first feature images of different resolutions are input into the bottleneck layer, and the second feature images are extracted by the semantic extraction module in the bottleneck layer, including: Input the first feature image into a plurality of the multi-layer perception units cascaded in sequence, and obtain a perceptual feature map output by each of the multi-layer perception units; wherein the perceptual feature map output by the previous multi-layer perception unit is the input of the next multi-layer perception unit; Processing the perceptual feature map output by each of the multi-layer perception units through an upsampling unit so that the dimension of the perceptual feature map output by each of the multi-layer perception units is consistent with the dimension of the perceptual feature map output by the first multi-layer perception unit; The perceptual feature maps output by the multi-layer perception unit are fused to obtain a second feature image.
5. The method according to claim 1, characterized in that Determining the position of the organ image region in the initial image according to the position of the feature related to the organ image region in the first feature images with different resolutions includes: The second feature image is decoded using a decoder, and the position of the organ image area is obtained based on the position of the features related to the organ image area in the decoded second feature image; the second feature image represents a feature map of the position of the features related to the organ image area output by the bottleneck layer.
6. The method according to claim 5, characterized in that Decoding the second feature image using a decoder, and obtaining the position of the organ image region according to the position of the feature related to the organ image region in the decoded second feature image, comprising: A plurality of decoding layers are sequentially arranged in the decoder, wherein the decoding layers have convolution kernels, and the arrangement of the convolution kernels in the decoding layers corresponds to the use order of the convolution kernels in the dual convolution module; Using the convolution kernel of the decoding layer, decoding the output feature map of the previous decoding layer and the second feature image corresponding to the decoding layer, to obtain the output feature map of the decoding layer; Obtaining an output feature map of the last decoding layer, and processing the output feature map using an activation function corresponding to the double convolution module to obtain the position of features related to the organ image region; wherein the resolution of the output feature map of the last decoding layer is the same as that of the initial image; The organ image region is determined in the initial image according to the position of the feature related to the organ image region in the output feature map of the last decoding layer.
7. The method according to claim 6, characterized in that The first segmentation model also includes a supplementary attention module; the method further includes: Acquire a first feature image corresponding to the decoding layer; Constructing a weighted attention map and a supplementary weight matrix according to the output feature map of the decoding layer and the first feature map; Calculating the product of the first feature image and the weighted attention map to obtain a first interactive attention map; Calculating the product of the supplementary weight matrix and the first interactive attention map to obtain a second interactive attention map; The second interactive attention map is superimposed on the output feature map of the decoding layer to obtain a new output feature map.
8. The method according to any one of claims 1 to 7, characterized in that The step of obtaining an organ image region in the initial image according to the identification and cropping the initial image to obtain a cropped image corresponding to the organ image region includes: Determining a boundary of the organ image region according to the position of the organ image region in the initial image obtained by identification; According to preset parameters, the boundary of the organ image region is expanded; wherein the preset parameters are determined according to the anatomical structure of the organ image region and the resolution of the initial image; According to the boundary of the expanded organ image region, a cropped image corresponding to the organ image region is obtained by cropping the initial image.
9. The method according to any one of claims 1 to 7, characterized in that: The method comprises: Acquire training data, wherein the training data includes a lesion image with position marks of the organ image region and the lesion image region; The first segmentation model and the second segmentation model are trained using the training data.
10. A model-based lesion image recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring an initial image; the initial image represents an image directly acquired by a medical device; A segmentation module, used for processing the initial image using a first segmentation model to identify the position of the organ image region in the initial image; wherein the first segmentation model completes the identification of the organ image region through an encoder, a bottleneck layer and a decoder; the encoder is used to convert the initial image into a first feature image with different resolutions; the bottleneck layer is used to determine the position of features related to the organ image region in the first feature images with different resolutions through semantic recognition; the decoder is used to determine the position of the organ image region in the initial image according to the position of features related to the organ image region in the first feature images with different resolutions; according to the position of the organ image region in the initial image obtained by identification, the initial image is cropped to obtain a cropped image containing the organ image region; the cropped image is processed using a second segmentation model to identify the position of the lesion image region in the organ image region; wherein the second segmentation model has different model parameters from the first segmentation model, and the bottleneck layer of the second segmentation model has a semantic extraction module repeated multiple times, and the network structure of the second segmentation model is a feature pyramid network structure; The marking module is used to perform marking processing in the initial image according to the positions of the organ image area and the lesion image area obtained by identification, so as to obtain a final lesion image.
11. An electronic device, characterized in that: The device comprises: a memory and a processor; The memory is used to store a computer program; the processor is used to implement the method according to any one of claims 1 to 9 according to the computer program stored in the memory.
12. A computer-readable storage medium / computer program product, characterized in that: The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 9; Furthermore, the computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.
