Cancer focus segmentation model, method, device and medium

Abstract

The invention provides a cancer focus segmentation model, method and device and a medium. The model comprises a U-shaped network composed of a plurality of encoders and a plurality of decoders which are symmetrical; the encoder extracts multi-scale features by using different types of convolution; the encoder and the decoder are in jump connection through an attention module, and the attention module is used for capturing long-range dependence along a first space direction and position information along a second space direction, and obtaining a region of interest according to the long-range dependence and the position information. The model has a multi-scale receptive field, so that lesion information of different scales can be captured, and the detection and recognition capability on lesions of different sizes is improved; cross-channel information, direction sensing information and position sensing information can be captured, interested targets can be accurately recognized, and the method has the advantages of being light in weight and rapid in reasoning.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to cancer lesion segmentation models, methods, devices and media. Background Art

[0002] Neural network models applied to the medical field have a large amount of computation, so they are often complex in structure and have a large number of parameters, making it difficult to deploy applications on embedded devices or mobile terminals. On the one hand, the large number of parameters in complex models places high demands on the GPU memory of mobile devices. On the other hand, many actual application scenarios require low latency and fast response for network models, and the more complex the network model, the slower the response speed. Therefore, the large number of parameters and computational complexity of these neural network models in the medical field make them unsuitable for mobile medical applications. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The purpose of the present application is to solve one of the technical problems existing in the related art to at least a certain extent. The embodiments of the present application provide a cancer lesion segmentation model, method, device and medium, which have the advantages of lightweight and fast reasoning.

[0005] An embodiment of the first aspect of the present application, a cancer lesion segmentation model, comprises:

[0006] A U-shaped network consisting of multiple symmetrical encoders and multiple decoders; the encoder is used to extract multi-scale features using different types of convolutions; the encoder and the decoder are jump-connected via an attention module, the attention module is used to capture long-range dependencies along a first spatial direction and position information along a second spatial direction, and a region of interest is obtained based on the long-range dependencies and the position information, and the first spatial direction is different from the second spatial direction.

[0007] According to certain embodiments of the first aspect of the present application, the number of channels of the plurality of encoders gradually increases along the forward direction.

[0008] According to certain embodiments of the first aspect of the present application, the encoder includes a multi-scale feature module, the multi-scale feature module includes a first branch, and the first branch includes a plurality of serial standard convolution blocks.

[0009] According to certain embodiments of the first aspect of the present application, the multi-scale feature module includes a second branch, which includes a plurality of serial atrous convolution blocks.

[0010] According to certain embodiments of the first aspect of the present application, the multi-scale feature module includes a third branch, and the third branch includes a plurality of standard convolution blocks and a plurality of hole convolution blocks in series.

[0011] According to an embodiment of the first aspect of the present application, the sum of the output features of the first branch and the output features of the second branch is convolved to obtain a first convolution feature, and the first convolution feature and the output features of the second branch are added to obtain the output feature of the multi-scale feature module.

[0012] According to an embodiment of the first aspect of the present application, the attention module encodes the input tensor along the first spatial direction to obtain a first encoded feature, encodes it along the second spatial direction to obtain a second encoded feature, aggregates the first encoded feature and the second encoded feature to obtain an aggregated feature, performs a first linear transformation on the aggregated feature along the first spatial direction to capture long-range dependencies to obtain a first perceptual attention map, and performs a second linear transformation on the aggregated feature along the second spatial direction to capture position information to obtain a second perceptual attention map; the first perceptual attention map and the second perceptual attention map include the region of interest.

[0013] An embodiment of the second aspect of the present application, a method for segmenting a cancer lesion, comprises:

[0014] Acquire cancer images;

[0015] The cancer image is input into the cancer lesion segmentation model as described in the embodiment of the first aspect of the present application to perform image segmentation to obtain a segmentation result.

[0016] An embodiment of the third aspect of the present application is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the cancer lesion segmentation method as described in the embodiment of the second aspect of the present application is implemented.

[0017] An embodiment of the fourth aspect of the present application is a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the cancer lesion segmentation method as described in the embodiment of the second aspect of the present application.

[0018] The above scheme has at least the following beneficial effects: by extracting features of different scales, the model can have a multi-scale receptive field, so that lesion information of different scales can be captured. This helps to improve the model's ability to detect and identify lesions of different sizes. At the same time, it can also enhance feature expression and improve the robustness of the model. Not only can it capture cross-channel information, but it can also capture direction perception and position perception information, which can help the model locate and identify targets of interest more accurately with fewer parameters. It has the advantages of lightweight and fast reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0020] Figure 1 is a schematic diagram of the structure of a cancer lesion segmentation model provided in an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of the structure of a multi-scale feature module provided in an embodiment of the present application;

[0022] Figure 3 It is a structural diagram of the attention module provided in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0025] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0026] An embodiment of the present application provides a cancer lesion segmentation model.

[0027] Reference Figure 1The cancer lesion segmentation model includes: a U-shaped network composed of multiple symmetrical encoders and multiple decoders; the encoder is used to extract multi-scale features using different types of convolutions; the encoder and the decoder are jump-connected through an attention module, and the attention module is used to capture the long-range dependency along the first spatial direction and the position information along the second spatial direction, and the region of interest is obtained according to the long-range dependency and the position information, and the first spatial direction is different from the second spatial direction.

[0028] The cancer lesion segmentation model has a simple structure and has the advantages of lightweight and fast inference.

[0029] In contrast to the simple skip connections in UNet, the U-Net of the cancer lesion segmentation model includes attention modules for each stage between the encoder and decoder. By integrating these advanced modules, the cancer lesion segmentation model significantly reduces the parameters and computational load while improving the segmentation performance.

[0030] The number of channels of multiple encoders gradually increases along the forward direction. The encoder consists of six stages, and the number of channels in each stage is {8, 16, 24, 32, 48, 64}. The first three stages use ordinary convolutions with kernel sizes of k=1, k=2, and k=3, and the last three stages use multi-scale feature modules to extract multi-scale features from different angles.

[0031] Reference Figure 2 The encoder includes a multi-scale feature module, the multi-scale feature module includes a first branch, the first branch includes a plurality of serial standard convolution blocks. The multi-scale feature module includes a second branch, the second branch includes a plurality of serial hole convolution blocks. The multi-scale feature module includes a third branch, the third branch includes a plurality of serial standard convolution blocks and a plurality of hole convolution blocks. The output feature of the first branch and the output feature of the second branch are added to obtain a first convolution feature through convolution, and the first convolution feature and the output feature of the second branch are added to obtain the output feature of the multi-scale feature module.

[0032] Specifically, the first standard convolution block of the first branch uses a 3x3 convolution operation, followed by batch normalization and ReLU activation function. The second standard convolution block uses a 3x3 convolution operation, followed by batch normalization and ReLU activation function.

[0033] The first dilated convolution block of the second branch uses a 3x3 dilated convolution operation followed by batch normalization and ReLU activation. The second dilated convolution block uses a 3x3 dilated convolution operation followed by batch normalization and ReLU activation. The dilation rates of the first and second dilated convolution blocks are different.

[0034] The first dilated convolution block of the third branch uses a 1x3 dilated convolution operation, followed by batch normalization and ReLU activation. The second standard convolution block uses a 3x1 convolution operation, followed by batch normalization and ReLU activation. The first dilated convolution block uses a 1x3 dilated convolution operation, followed by batch normalization and ReLU activation. The second dilated convolution block uses a 3x1 dilated convolution operation, followed by batch normalization and ReLU activation.

[0035] The sum of the output features of the first branch and the output features of the second branch is processed by 1x1 convolution, batch normalization and ReLU activation function to obtain the first convolution feature. The first convolution feature and the output features of the second branch are added to obtain the output features of the multi-scale feature module.

[0036] Using different types of convolution blocks to extract multi-scale features enables it to better understand the global and local features in the image, while also being able to extract rich features and improve the generalization ability of the model. Dilated convolution will enable the model to explore a wider feature space with fewer parameters and retain more spatial information. Asymmetric convolutions have the same receptive field as standard convolutions. Compared with standard convolutions, asymmetric convolutions may require fewer parameters when processing specific types of data. For data with obvious directionality, using asymmetric convolutions can reduce the number of parameters by sharing weights, thereby improving the efficiency of the model.

[0037] By extracting features of different scales, the model can have a multi-scale receptive field, so that it can capture lesion information of different scales. This helps to improve the model's ability to detect and identify lesions of different sizes. At the same time, it can also enhance feature expression and improve the robustness of the model.

[0038] Reference Figure 3 The attention module encodes the input tensor along a first spatial direction to obtain a first encoded feature, encodes it along a second spatial direction to obtain a second encoded feature, aggregates the first encoded feature and the second encoded feature to obtain an aggregated feature, performs a first linear transformation on the aggregated feature along the first spatial direction to capture long-range dependencies to obtain a first perceptual attention map, and performs a second linear transformation on the aggregated feature along the second spatial direction to capture position information to obtain a second perceptual attention map; the first perceptual attention map and the second perceptual attention map contain regions of interest.

[0039] Specifically, global pooling is used for channel attention to encode global spatial information into channel descriptors. However, it loses position information, which is crucial for capturing spatial structure in visual tasks. Therefore, position information needs to be preserved so that the network can understand the positional relationship of objects in the image, thereby better performing spatial reasoning and task execution. Global pooling is decomposed into a one-to-one one-dimensional feature encoding operation.

[0040] The first spatial direction is the x-axis direction, and the second spatial direction is the y-axis direction.

[0041] The input tensor of the attention module is 4 different inputs. In particular, the original input is added at each stage, which helps to better preserve and utilize the details of the input image.

[0042] For the input tensor, the pooling kernel is used to average pool and encode each channel along the x-axis direction to obtain the first encoded feature; the pooling kernel is used to average pool and encode each channel along the y-axis direction to obtain the second encoded feature. The first encoded feature and the second encoded feature are concatenated and two-dimensionally convolved, and then batch normalized and nonlinearly transformed to aggregate the features to obtain the aggregated feature.

[0043] The aggregated features are subjected to a first linear transformation and a Sigmoid activation function along the first spatial direction to capture long-range dependencies to obtain a first perceptual attention map. The aggregated features are subjected to a second linear transformation and a Sigmoid activation function along the second spatial direction to capture position information to obtain a second perceptual attention map. The first perceptual attention map and the second perceptual attention map are reweighted.

[0044] The first linear transformation and the second linear transformation allow the attention module to capture long-range dependencies along one spatial direction and precise position information along another spatial direction, which helps the network to locate the object of interest more accurately.

[0045] In order to reduce the number of parameters, after splitting into two separate tensors along the spatial dimension, the first linear transformation and the second linear transformation are applied with bilinear interpolation instead of convolution operation.

[0046] The weights generated by the Sigmoid activation function are multiplied by the original input tensor to obtain a perceptual attention map containing the area of ​​interest.

[0047] It can not only capture cross-channel information, but also direction-aware and position-aware information, which can help the model locate and identify targets of interest more accurately with fewer parameters.

[0048] An embodiment of the present application provides a cancer lesion segmentation method.

[0049] The cancer lesion segmentation method comprises the following steps:

[0050] Step S100, acquiring a cancer image;

[0051] Step S200: Input the cancer image into the cancer lesion segmentation model to perform image segmentation to obtain a segmentation result.

[0052] Specifically, the cancer image is an X-ray image of breast cancer. The X-ray image of breast cancer is input into the cancer lesion segmentation model for image segmentation, and the breast cancer area and the background image area can be segmented.

[0053] An embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above cancer lesion segmentation method when executing the computer program.

[0054] The electronic device may be any intelligent terminal including a computer.

[0055] In general, for the hardware structure of the electronic device, the processor can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., to execute related programs to implement the technical solutions provided in the embodiments of the present application.

[0056] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory, and the processor is called to execute the methods of the embodiments of this application.

[0057] The input / output interface is used to realize information input and output.

[0058] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WI FI, Bluetooth, etc.).

[0059] The bus transmits information between the various components of the device (such as the processor, memory, input / output interface, and communication interface). The processor, memory, input / output interface, and communication interface communicate with each other within the device through the bus.

[0060] An embodiment of the present application provides a computer storage medium storing computer executable instructions for executing the cancer lesion segmentation method as described above.

[0061] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the above description of this specification, the description of reference terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0062] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0063] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0066] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0067] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A cancer lesion segmentation model, characterized in that: include: A U-shaped network consisting of a plurality of symmetrical encoders and a plurality of decoders; the encoders are used to extract multi-scale features using different types of convolutions; The encoder and the decoder are jump-connected via an attention module, and the attention module is used to capture long-range dependencies along a first spatial direction and position information along a second spatial direction, and obtain a region of interest based on the long-range dependencies and the position information. The first spatial direction is different from the second spatial direction.

2. The cancer lesion segmentation model according to claim 1, characterized in that: The number of channels of the plurality of encoders gradually increases along the forward direction.

3. The cancer lesion segmentation model according to claim 1, characterized in that: The encoder includes a multi-scale feature module, which includes a first branch, and the first branch includes a plurality of serial standard convolution blocks.

4. The cancer lesion segmentation model according to claim 3, characterized in that: The multi-scale feature module includes a second branch, which includes a plurality of serial dilated convolution blocks.

5. The cancer lesion segmentation model according to claim 4, characterized in that: The multi-scale feature module includes a third branch, and the third branch includes a plurality of standard convolution blocks and a plurality of hole convolution blocks in series.

6. The cancer lesion segmentation model according to claim 5, characterized in that: The output feature of the first branch and the output feature of the second branch are added to obtain a first convolution feature through convolution, and the first convolution feature and the output feature of the second branch are added to obtain the output feature of the multi-scale feature module.

7. The cancer lesion segmentation model according to claim 1, characterized in that: The attention module encodes the input tensor along the first spatial direction to obtain a first encoded feature, encodes it along the second spatial direction to obtain a second encoded feature, aggregates the first encoded feature and the second encoded feature to obtain an aggregated feature, performs a first linear transformation on the aggregated feature along the first spatial direction to capture long-range dependencies to obtain a first perceptual attention map, and performs a second linear transformation on the aggregated feature along the second spatial direction to capture position information to obtain a second perceptual attention map; the first perceptual attention map and the second perceptual attention map include the region of interest.

8. A cancer lesion segmentation method, characterized in that: include: Acquire cancer images; The cancer image is input into the cancer lesion segmentation model according to any one of claims 1 to 7 to perform image segmentation to obtain a segmentation result.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cancer lesion segmentation method according to claim 8 when executing the computer program.

10. A computer storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the cancer lesion segmentation method according to claim 8.

Images