Image segmentation method and device of mammary gland duct, electronic equipment and storage medium
By constructing a mammary duct image segmentation model consisting of an encoder, connector, and decoder, the problem of low accuracy in mammary duct segmentation using deep learning networks is solved, achieving higher-precision automatic segmentation of duct regions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA NORMAL UNIV
- Filing Date
- 2023-03-30
- Publication Date
- 2026-05-29
Smart Images
Figure CN116703937B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image segmentation method, apparatus, electronic device, and storage medium for mammary ducts. Background Technology
[0002] Image segmentation refers to the techniques and processes for extracting regions of interest from an image; it is a crucial step in the transition from image processing to image analysis. With the development of the information age and the increasing number of images to be processed, image segmentation methods using deep learning networks have demonstrated powerful performance and convenience. Deep learning networks, through continuous training, extract image features and segment the regions of interest. Therefore, deep learning networks have broad research value and significance in the field of image segmentation.
[0003] In related technologies, due to the significant morphological differences between different mammary ducts, with the area of large and small ducts differing by as much as a hundredfold, deep learning networks have an insignificant effect on the segmentation of mammary duct regions and low segmentation accuracy. Summary of the Invention
[0004] Based on this, the purpose of this application is to provide an image segmentation method, apparatus, electronic device, and storage medium for mammary ducts, which can improve the image segmentation accuracy of mammary ducts.
[0005] According to a first aspect of the embodiments of this application, a method for image segmentation of mammary ducts is provided, comprising the following steps:
[0006] Obtain a dataset of sample images of mammary ducts;
[0007] An image segmentation model for mammary ducts is constructed. The image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence. The encoder is used for feature extraction, the connector is used for feature weight calculation, establishing global relationships, and suppressing invalid features, and the decoder is used for feature fusion.
[0008] The encoder comprises several encoding modules, each including a first convolutional layer, a second convolutional layer, an average pooling layer, a max pooling layer, and a first perceptron network. The first convolutional layer, the average pooling layer, the max pooling layer, and the first perceptron network constitute the channel attention module of the encoder, and the second convolutional layer constitutes the spatial attention module of the encoder. The connector includes a first linear fitting layer, a multi-head cross-attention network layer, a second linear fitting layer, and a second perceptron network. The decoder includes a first global average pooling layer, a second global average pooling layer, a first dimensionality reduction fitting layer, a second dimensionality reduction fitting layer, and a transpose layer.
[0009] Based on the sample image dataset, the encoder, connector, and decoder are trained to obtain a trained image segmentation model of mammary ducts;
[0010] The image of the mammary duct to be segmented is input into the trained image segmentation model of the mammary duct to obtain the image of the mammary duct region.
[0011] According to a second aspect of the embodiments of this application, an image segmentation apparatus for mammary ducts is provided, comprising:
[0012] The dataset acquisition module is used to acquire a dataset of sample images of mammary ducts;
[0013] The model building module is used to construct an image segmentation model for mammary ducts. The image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence. The encoder is used for feature extraction, the connector is used for feature weight calculation, establishing global relationships, and suppressing invalid features, and the decoder is used for feature fusion.
[0014] The encoder comprises several encoding modules, each including a first convolutional layer, a second convolutional layer, an average pooling layer, a max pooling layer, and a first perceptron network. The first convolutional layer, the average pooling layer, the max pooling layer, and the first perceptron network constitute the channel attention module of the encoder, and the second convolutional layer constitutes the spatial attention module of the encoder. The connector includes a first linear fitting layer, a multi-head cross-attention network layer, a second linear fitting layer, and a second perceptron network. The decoder includes a first global average pooling layer, a second global average pooling layer, a first dimensionality reduction fitting layer, a second dimensionality reduction fitting layer, and a transpose layer.
[0015] The model training module is used to train the encoder, connector, and decoder based on the sample image dataset to obtain a trained image segmentation model of mammary ducts.
[0016] The region image acquisition module is used to input the mammary duct image to be segmented into the trained mammary duct image segmentation model to obtain the mammary duct region image.
[0017] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed as in any of the above-described methods for image segmentation of mammary ducts.
[0018] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image segmentation method for mammary ducts as described in any of the above.
[0019] This application embodiment obtains a sample image dataset of mammary ducts; constructs an image segmentation model for mammary ducts; the image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence; the encoder is used for feature extraction, the connector is used for feature weight calculation, and the decoder is used for feature fusion; based on the sample image dataset, the encoder, connector, and decoder are trained to obtain a trained image segmentation model for mammary ducts; the image of the mammary duct to be segmented is input into the trained image segmentation model for mammary ducts to obtain an image of the mammary duct region, thereby improving the image segmentation accuracy of mammary ducts.
[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.
[0021] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description
[0022] Figure 1 A flowchart illustrating an image segmentation method for mammary ducts provided in one embodiment of this application;
[0023] Figure 2 This is a structural block diagram of an image segmentation device for mammary ducts provided in one embodiment of this application;
[0024] Figure 3 This is a schematic block diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0026] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0027] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0028] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0029] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0030] Example 1
[0031] Please see Figure 1 This is a flowchart illustrating an image segmentation method for mammary ducts provided in one embodiment of this application. The image segmentation method for mammary ducts provided in this embodiment includes the following steps:
[0032] S10: Obtain a dataset of sample images of mammary ducts.
[0033] In this embodiment of the application, web crawler technology can be used to download several sample images of mammary ducts from the Internet to form a sample image dataset of mammary ducts.
[0034] S20: Construct an image segmentation model for mammary ducts; the image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence; the encoder is used for feature extraction, the connector is used for feature weight calculation, and the decoder is used for feature fusion.
[0035] In the embodiments of this application, the encoder may include several encoding modules based on the joint attention mechanism, the connector is a CCT skip connection structure based on multi-scale training, CCT is a channel cross-fusion converter, and the decoder is a TCA structure based on the attention mechanism, TCA is a transposed cross-attention mechanism.
[0036] S30: Based on the sample image dataset, train the encoder, connector, and decoder to obtain a trained image segmentation model for mammary ducts.
[0037] In this embodiment, the sample image dataset is sequentially input into the encoder, connector, and decoder, and the image segmentation result is output. Based on the image segmentation result and the segmentation label of each sample image in the sample image dataset, the loss function value is calculated, and the encoder, connector, and decoder are trained to obtain a trained image segmentation model of mammary ducts.
[0038] S40: Input the image of the mammary duct to be segmented into the trained image segmentation model of the mammary duct to obtain the image of the mammary duct region.
[0039] In this embodiment of the application, after obtaining the trained image segmentation model of mammary ducts, the image of the mammary ducts to be segmented is input into the trained image segmentation model of mammary ducts, and the image of the mammary duct region can be obtained automatically and quickly.
[0040] By applying the embodiments of this application, a sample image dataset of mammary ducts is obtained; an image segmentation model of mammary ducts is constructed; the image segmentation model of mammary ducts includes an encoder, a connector, and a decoder connected in sequence; the encoder is used for feature extraction, the connector is used for feature weight calculation, and the decoder is used for feature fusion; the encoder, connector, and decoder are trained according to the sample image dataset to obtain a trained image segmentation model of mammary ducts; the image of the mammary duct to be segmented is input into the trained image segmentation model of mammary ducts to obtain an image of the mammary duct region, thereby improving the image segmentation accuracy of mammary ducts.
[0041] In an optional embodiment, after step S10, step S101 is included, as follows:
[0042] S101: Preprocess the sample image dataset; wherein, the preprocessing includes: dividing the sample image dataset into datasets and performing image compression, label normalization and image enhancement on each sample image in the sample image dataset.
[0043] In this embodiment of the application, the sample image dataset of mammary ducts consists of 230 images, each with a resolution of 2000. 2000 pixels. The labels of the sample images of mammary ducts were normalized and compressed to 512. The dataset is 512 pixels in size. Then, the images are flipped, mirrored, and subjected to a series of image enhancement operations. Finally, the sample image dataset is divided into training, validation, and test sets in a 6:2:2 ratio.
[0044] In an optional embodiment, the sample image dataset includes several sample images and segmentation labels for each sample image. Step S30 includes steps S31 to S35, as follows:
[0045] S31: Input each sample image in the sample image dataset into the encoder to obtain the encoded feature vector of each sample image.
[0046] The segmentation label for each sample image includes 0 and 1. 1 represents the region of mammary ducts in the sample image, and 0 represents the region of the sample image that is not a mammary duct, i.e., the background region.
[0047] In this embodiment of the application, each sample image in the sample image dataset is input to the encoder for feature extraction to obtain the encoded feature vector of each sample image.
[0048] S32: Input the encoded feature vector of each sample image into the connector to obtain the connection feature vector of each sample image.
[0049] In this embodiment of the application, the encoded feature vector of each sample image is input to the connector to calculate the feature weights and obtain the connection feature vector of each sample image.
[0050] S33: Input the connection feature vector and the encoded feature vector of each sample image into the decoder to obtain the segmentation result of each sample image.
[0051] In this embodiment of the application, the connection feature vector and the encoded feature vector of each sample image are input into the decoder for feature fusion to obtain the segmentation result of each sample image.
[0052] S34: Determine the loss function of the image segmentation model for mammary ducts based on the segmentation results and segmentation labels of each sample image.
[0053] S35: Iteratively train the image segmentation model of mammary ducts according to the loss function until the predetermined training termination condition is met, and obtain the trained image segmentation model of mammary ducts.
[0054] In the embodiments of this application, the predetermined training termination condition may be that the number of training sessions meets a preset threshold, the loss function converges, or the loss function value fluctuates within a preset threshold range.
[0055] By training the encoder, connector, and decoder, a trained image segmentation model of mammary ducts can be obtained automatically and quickly.
[0056] In an optional embodiment, the encoder includes several encoding modules, each encoding module including a first convolutional layer, a second convolutional layer, an average pooling layer, a max pooling layer, and a first perceptron network. Step S31 includes steps S311 to S318, as follows:
[0057] S311: Traverse each encoding module, input each sample image in the sample image dataset into the first convolutional layer of the current encoding module, and obtain the first feature vector of each sample image.
[0058] In this embodiment, the encoder includes four encoding modules, with the first convolutional layer being 3... The first convolutional layer has a kernel size of 3, and the second convolutional layer has a kernel size of 7. A convolution kernel of 7.
[0059] S312: Input the first feature vector of each sample image into the average pooling layer of the current encoding module to obtain the second feature vector of each sample image;
[0060] S313: Input the first feature vector of each sample image into the max pooling layer of the current encoding module to obtain the third feature vector of each sample image;
[0061] S314: Input the second and third feature vectors of each sample image into the first perceptron network of the current encoding module to obtain the fourth feature vector of each sample image;
[0062] S315: Multiply the fourth feature vector of each sample image with the first feature vector to obtain the fifth feature vector of each sample image;
[0063] S316: Calculate the average and maximum values of each vector element in the fifth feature vector of each sample image, and concatenate the average and maximum values to obtain the sixth feature vector of each sample image.
[0064] S317: Input the sixth feature vector of each sample image into the second convolutional layer of the current encoding module to obtain the seventh feature vector of each sample image;
[0065] S318: Multiply the seventh feature vector of each sample image with the fifth feature vector of each sample image to obtain the encoded feature vector of each sample image.
[0066] In this embodiment, since the encoder includes four encoding modules, the encoder ultimately outputs four encoded feature vectors. The first convolutional layer, average pooling layer, max pooling layer, and first perceptron network constitute the encoder's channel attention module, and the second convolutional layer constitutes the encoder's spatial attention module. Each sample image is sequentially passed through the channel attention module and the spatial attention module for feature extraction, achieving better feature extraction results.
[0067] In an optional embodiment, the connector includes a first linear fitting layer, a multi-head cross-attention network layer, a second linear fitting layer, and a second perceptron network. Step S32 includes steps S321 to S325, as follows:
[0068] S321: Input the encoded feature vector of each sample image into the first linear fitting layer to obtain the first linear feature vector of each sample image;
[0069] S322: Input the first linear feature vector of each sample image into the multi-head cross-attention network layer to obtain the attention feature vector of each sample image;
[0070] S323: Input the attention feature vector of each sample image into the second linear fitting layer to obtain the second linear feature vector of each sample image;
[0071] S324: Input the second linear feature vector of each sample image into the second perceptron network to obtain the eighth feature vector of each sample image;
[0072] S325: Add the eighth feature vector of each sample image to the attention feature vector of each sample image to obtain the connection feature vector of each sample image.
[0073] Both the first and second linear fitting layers are Linear layers.
[0074] In this embodiment, the four encoded feature vectors of each sample image are connected through a connector to calculate the weights of the four encoded feature vectors. The weights of the four encoded feature vectors are then added to the corresponding eighth feature vector to establish a global relationship, suppress invalid features, and finally the four connected feature vectors are sent to the decoder for feature fusion.
[0075] In an optional embodiment, the decoder includes a first global average pooling layer, a second global average pooling layer, a first dimensionality reduction fitting layer, a second dimensionality reduction fitting layer, and a transpose layer. Step S33 includes steps S331 to S338, as follows:
[0076] S331: Input the connection feature vector of each sample image into the first global average pooling layer to obtain the first global feature vector of each sample image;
[0077] S332: Input the first global feature vector of each sample image into the first dimension reduction fitting layer to obtain the first decoded feature vector of each sample image;
[0078] S333: Input the encoded feature vector of each sample image into the second global average pooling layer to obtain the second global feature vector of each sample image;
[0079] S334: Input the second global feature vector of each sample image into the second dimension reduction fitting layer to obtain the second decoded feature vector of each sample image;
[0080] S335: Input the second decoded feature vector of each sample image into the transpose layer to obtain the third decoded feature vector of each sample image;
[0081] S336: Multiply the third decoded feature vector of each sample image with the first decoded feature vector of each sample image to obtain the product result of each sample image;
[0082] S337: Multiply the product of each sample image with the connection feature vector of each sample image to obtain the segmentation result of each sample image.
[0083] In this embodiment, the decoder uses a global average pooling layer to compress the encoded features to obtain global spatial information. Next, global attention is obtained through the expansion operation of a linear layer. Unfolding operations are used to obtain noteworthy encoder and decoder features. Finally, the two features are multiplied to obtain a similarity matrix, which is then normalized using a softmax function to obtain weights for different features. This strengthens highly correlated features and suppresses weakly correlated features, thus eliminating the ambiguity between the encoder and decoder to some extent and improving the accuracy of the segmentation results.
[0084] In an optional embodiment, step S34 includes steps S341 to S342, as follows:
[0085] S341: Based on the segmentation results and segmentation labels of each sample image, determine the first loss function and the second loss function of the mammary duct image segmentation model;
[0086] S342: Determine the total loss function of the mammary duct image segmentation model based on the first loss function and the second loss function.
[0087] The formula for the first loss function is:
[0088]
[0089] The formula for the second loss function is:
[0090]
[0091] The formula for the total loss function is:
[0092] in, The segmentation label represents the segmentation label of each sample image. This represents the segmentation results for each sample image. express and The number of elements in the intersection between them. express The number of elements in the middle. express The number of elements in the middle. This represents the weighting coefficients of the first loss function. This represents the weighting coefficients of the second loss function.
[0093] In this embodiment of the application, the training set is input into the image segmentation model of mammary ducts to train the image segmentation model of mammary ducts. During the training process, the learning rate of the network is lr=0.01, and the optimizer adopts the adaptive moment estimation optimizer, iterating until the total loss function converges.
[0094] Example 2
[0095] The following are embodiments of the apparatus of this application, which can be used to execute the method described in Embodiment 1 of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method described in Embodiment 1 of this application.
[0096] Please see Figure 2 This document illustrates a schematic diagram of the image segmentation device for mammary ducts provided in an embodiment of this application. The image segmentation device 5 for mammary ducts provided in this embodiment includes:
[0097] Data set acquisition module 51 is used to acquire a sample image dataset of mammary ducts;
[0098] The model building module 52 is used to build an image segmentation model for mammary ducts. The image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence. The encoder is used for feature extraction, the connector is used for feature weight calculation, and the decoder is used for feature fusion.
[0099] The model training module 53 is used to train the encoder, connector, and decoder based on the sample image dataset to obtain a trained image segmentation model of mammary ducts.
[0100] The region image acquisition module 54 is used to input the mammary duct image to be segmented into the trained mammary duct image segmentation model to obtain the mammary duct region image.
[0101] It should be noted that the image segmentation device for mammary ducts provided in the above embodiments is only illustrated by the division of the above functional modules when performing the image segmentation method for mammary ducts. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image segmentation device for mammary ducts and the image segmentation method for mammary ducts provided in the above embodiments belong to the same concept, and its implementation process is detailed in the method embodiments, which will not be repeated here.
[0102] Example 3
[0103] The following are embodiments of the device described in this application, which can be used to execute the method described in Embodiment 1 of this application. For details not disclosed in the embodiments of the device described in this application, please refer to the method described in Embodiment 1 of this application.
[0104] Please see Figure 3 This application also provides an electronic device 300, which may specifically be a computer, mobile phone, tablet computer, etc. In an exemplary embodiment of this application, the electronic device 300 is a computer, which may include: at least one processor 301, at least one memory 302, at least one display, at least one network interface 303, user interface 304, and at least one communication bus 305.
[0105] The user interface 304 is primarily used to provide an input interface for the user and to acquire user input data. Optionally, the user interface may also include a standard wired interface or a wireless interface.
[0106] The network interface 303 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0107] The communication bus 305 is used to enable communication between these components.
[0108] The processor 301 may include one or more processing cores. The processor connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.
[0109] The memory 302 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor. As a computer storage medium, the memory may include an operating system, a network communication module, a user interface module, and operating applications.
[0110] The processor can be used to call the application program of the image segmentation method of mammary ducts stored in the memory, and specifically execute the method steps of Embodiment 1 shown above. For the specific execution process, please refer to the detailed description shown in Embodiment 1, which will not be repeated here.
[0111] Example 4
[0112] This application also provides a computer-readable storage medium storing a computer program thereon, the instructions of which are adapted to be loaded by a processor and executed by the method steps of Embodiment 1 shown above. The specific execution process can be found in the detailed description of the embodiments, and will not be repeated here. The device containing the storage medium can be an electronic device such as a personal computer, laptop computer, smartphone, or tablet computer.
[0113] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function selected in one or more boxes.
[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.
[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0118] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0119] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0121] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for image segmentation of mammary ducts, characterized in that, The method includes the following steps: Obtain a dataset of sample images of mammary ducts; An image segmentation model for mammary ducts is constructed; the image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence; the encoder is used for feature extraction, the connector is used for feature weight calculation, establishing global relationships, and suppressing invalid features, and the decoder is used for feature fusion. The encoder comprises several encoding modules, each including a first convolutional layer, a second convolutional layer, an average pooling layer, a max pooling layer, and a first perceptron network. The first convolutional layer, the average pooling layer, the max pooling layer, and the first perceptron network constitute the channel attention module of the encoder, and the second convolutional layer constitutes the spatial attention module of the encoder. The connector includes a first linear fitting layer, a multi-head cross-attention network layer, a second linear fitting layer, and a second perceptron network. The decoder includes a first global average pooling layer, a second global average pooling layer, a first dimensionality reduction fitting layer, a second dimensionality reduction fitting layer, and a transpose layer. Based on the sample image dataset, the encoder, the connector, and the decoder are trained to obtain a trained image segmentation model of mammary ducts; The image of the mammary duct to be segmented is input into the trained image segmentation model of the mammary duct to obtain the image of the mammary duct region.
2. The image segmentation method for mammary ducts according to claim 1, characterized in that: The sample image dataset includes several sample images and segmentation labels for each sample image; The step of training the encoder, the connector, and the decoder based on the sample image dataset to obtain a trained image segmentation model of mammary ducts includes: Each sample image in the sample image dataset is input into the encoder to obtain the encoded feature vector of each sample image; The encoded feature vectors of each sample image are input into the connector to obtain the connection feature vectors of each sample image; The connection feature vector and the encoding feature vector of each sample image are input into the decoder to obtain the segmentation result of each sample image; Based on the segmentation results and segmentation labels of each sample image, the loss function of the image segmentation model for the mammary ducts is determined; The image segmentation model of the mammary ducts is iteratively trained according to the loss function until a predetermined training termination condition is met, thereby obtaining the trained image segmentation model of the mammary ducts.
3. The image segmentation method for mammary ducts according to claim 2, characterized in that: The step of inputting each sample image in the sample image dataset into the encoder to obtain the encoded feature vector of each sample image includes: By traversing each of the encoding modules, each sample image in the sample image dataset is input into the first convolutional layer of the current encoding module to obtain the first feature vector of each sample image; The first feature vector of each of the sample images is input into the average pooling layer of the current encoding module to obtain the second feature vector of each of the sample images; The first feature vector of each sample image is input into the max pooling layer of the current encoding module to obtain the third feature vector of each sample image; The second and third feature vectors of each sample image are input into the first perceptron network of the current encoding module to obtain the fourth feature vector of each sample image; The fourth feature vector of each sample image is multiplied by the first feature vector to obtain the fifth feature vector of each sample image. The average value and the maximum value of each vector element in the fifth feature vector of each sample image are calculated, and the average value and the maximum value are concatenated to obtain the sixth feature vector of each sample image. The sixth feature vector of each of the sample images is input into the second convolutional layer of the current encoding module to obtain the seventh feature vector of each of the sample images; The seventh feature vector of each sample image is multiplied by the fifth feature vector of each sample image to obtain the encoded feature vector of each sample image.
4. The image segmentation method for mammary ducts according to claim 2, characterized in that: The step of inputting the encoded feature vectors of each sample image into the connector to obtain the connection feature vectors of each sample image includes: The encoded feature vectors of each of the sample images are input into the first linear fitting layer to obtain the first linear feature vectors of each of the sample images; The first linear feature vector of each of the sample images is input into the multi-head cross-attention network layer to obtain the attention feature vector of each of the sample images; The attention feature vector of each of the sample images is input into the second linear fitting layer to obtain the second linear feature vector of each of the sample images; The second linear feature vector of each of the sample images is input into the second perceptron network to obtain the eighth feature vector of each of the sample images; The eighth feature vector of each sample image is added to the attention feature vector of each sample image to obtain the connection feature vector of each sample image.
5. The image segmentation method for mammary ducts according to claim 2, characterized in that: The step of inputting the connection feature vector and the encoded feature vector of each sample image into the decoder to obtain the segmentation result of each sample image includes: The connection feature vectors of each of the sample images are input into the first global average pooling layer to obtain the first global feature vector of each of the sample images. The first global feature vector of each of the sample images is input into the first dimension reduction fitting layer to obtain the first decoded feature vector of each of the sample images; The encoded feature vectors of each of the sample images are input into the second global average pooling layer to obtain the second global feature vectors of each of the sample images; The second global feature vector of each of the sample images is input into the second dimension reduction fitting layer to obtain the second decoded feature vector of each of the sample images; The second decoded feature vector of each of the sample images is input into the transposed layer to obtain the third decoded feature vector of each of the sample images; Multiply the third decoded feature vector of each sample image with the first decoded feature vector of each sample image to obtain the product result of each sample image; The product of each sample image is multiplied by the connection feature vector of each sample image to obtain the segmentation result of each sample image.
6. The image segmentation method for mammary ducts according to claim 2, characterized in that: The step of determining the loss function of the image segmentation model for the mammary ducts based on the segmentation results and segmentation labels of each sample image includes: Based on the segmentation results and segmentation labels of each sample image, the first loss function and the second loss function of the image segmentation model for the mammary ducts are determined; Based on the first loss function and the second loss function, determine the total loss function of the image segmentation model for the mammary ducts; The formula for the first loss function is as follows: The formula for the second loss function is: The formula for the total loss function is: in, The segmentation label represents the segmentation label of each of the sample images. This represents the segmentation result of each of the sample images. express and The number of elements in the intersection between them. express The number of elements in the middle. express The number of elements in the middle. This represents the weighting coefficients of the first loss function. This represents the weighting coefficients of the second loss function.
7. The image segmentation method for mammary ducts according to any one of claims 1 to 6, characterized in that: Following the step of obtaining a sample image dataset of mammary ducts, the following steps are included: The sample image dataset is preprocessed; wherein the preprocessing includes: dividing the sample image dataset into datasets and performing image compression, label normalization, and image enhancement on each sample image in the sample image dataset.
8. An image segmentation device for mammary ducts, characterized in that, include: The dataset acquisition module is used to acquire a dataset of sample images of mammary ducts; The model building module is used to construct an image segmentation model for mammary ducts. The image segmentation model for mammary ducts includes an encoder, a connector, and a decoder connected in sequence. The encoder is used for feature extraction, establishing global relationships, and suppressing invalid features. The connector is used for feature weight calculation, and the decoder is used for feature fusion. The encoder comprises several encoding modules, each including a first convolutional layer, a second convolutional layer, an average pooling layer, a max pooling layer, and a first perceptron network. The first convolutional layer, the average pooling layer, the max pooling layer, and the first perceptron network constitute the channel attention module of the encoder, and the second convolutional layer constitutes the spatial attention module of the encoder. The connector includes a first linear fitting layer, a multi-head cross-attention network layer, a second linear fitting layer, and a second perceptron network. The decoder includes a first global average pooling layer, a second global average pooling layer, a first dimensionality reduction fitting layer, a second dimensionality reduction fitting layer, and a transpose layer. The model training module is used to train the encoder, the connector, and the decoder based on the sample image dataset to obtain a trained image segmentation model of mammary ducts. The region image acquisition module is used to input the image of the mammary duct to be segmented into the trained image segmentation model of the mammary duct to obtain the mammary duct region image.
9. An electronic device, comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.