Medical image segmentation method based on contrast difference review and related products

Abstract

The invention provides a medical image segmentation method based on contrast difference review and a related product, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first training image and a corresponding mask label; respectively processing the first training image through the first structure and the second structure of the target network to obtain a first edge segmentation mask and a second edge segmentation mask; determining a first inconsistent area of the first edge segmentation mask relative to the second edge segmentation mask and a second inconsistent area relative to the mask label; determining a third inconsistent area of the second edge segmentation mask relative to the first edge segmentation mask and a fourth inconsistent area relative to the mask label; determining global loss according to the first inconsistent region, the second inconsistent region, the third inconsistent region and the fourth inconsistent region; and training the target network through the global loss so as to perform image segmentation through the target network. According to the embodiment of the invention, the accuracy of image segmentation can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a medical image segmentation method, device, electronic device, computer-readable storage medium and computer program product based on contrast difference review. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Medical image segmentation is the foundation and key step in building a powerful computer-aided diagnosis (CAD) system. It can automatically segment organs and tissues in medical images, conduct detailed analysis and quantification of lesion areas, help doctors better understand the condition, and develop individualized treatment plans. The positioning of specific objects through effective segmentation models is of great significance for the early screening and accurate evaluation of related diseases.

[0004] In recent years, research on medical image segmentation based on deep learning has made significant progress. Through sufficient training and adjustment, deep learning models can adapt to various complex medical image scenes and automatically segment lesions, reducing the need for manual processing. However, the accuracy of image edge segmentation is currently low and it is difficult to meet user needs. Summary of the invention

[0005] The purpose of the present disclosure is to provide a medical image segmentation method, device, electronic device, computer-readable storage medium and computer program product based on contrast difference review, which can improve the accuracy of image edge segmentation.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] The present disclosure provides a medical image segmentation method based on contrast difference review, including: obtaining a first training image and a mask label of the first training image; performing edge prediction processing on the first training image through a first structure of a target network to obtain a first edge segmentation mask; performing edge prediction processing on the first training image through a second structure of the target network to obtain a second edge segmentation mask; determining a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; determining a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; determining a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; determining a global loss based on the first consistency loss and the second consistency loss; and training the target network through the global loss so as to perform image segmentation through the target network.

[0008] In some embodiments, the method also includes: acquiring a second training image; performing edge prediction processing on the second training image through the first structure to obtain a third edge segmentation mask; performing edge prediction processing on the second training image through the second structure to obtain a fourth edge segmentation mask; determining a fifth inconsistent area of ​​the third edge segmentation mask relative to the fourth edge segmentation mask; determining a sixth inconsistent area of ​​the fourth edge segmentation mask relative to the third edge segmentation mask; determining a third consistency loss between the fifth inconsistent area and the sixth inconsistent area; wherein, determining a global loss based on the first consistency loss and the second consistency loss includes: determining the global loss based on the first consistency loss, the second consistency loss and the third consistency loss.

[0009] In some embodiments, the first structure includes a first encoder and a first decoder; wherein, edge prediction processing is performed on the first training image through the first structure of the target network to obtain a first edge segmentation mask, including: feature encoding processing is performed on the first training image through the first encoder to obtain an encoded feature map; and the encoded feature map is decoded through the first decoder to obtain the first edge segmentation mask.

[0010] In some embodiments, the first encoder includes a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module, and the encoding feature map includes a first feature map, a second feature map, a third feature map and a fourth feature map; wherein, performing feature encoding processing on the first training image by the first encoder to obtain the encoding feature map includes: performing feature extraction processing on the first training image by the first encoding module and the second encoding module in sequence to obtain the first feature map; performing feature extraction processing on the first feature map by the third encoding module to obtain the second feature map; performing feature extraction processing on the second feature map by the fourth encoding module to obtain the third feature map; performing feature extraction processing on the third feature map by the fifth encoding module to obtain the fourth feature map.

[0011] In some embodiments, the second structure includes a first encoder, a first downsampling module, a second downsampling module, a third downsampling module and an edge-aware gating module; wherein the first structure and the second structure share an encoder; wherein edge prediction processing is performed on the first training image through the second structure of the target network to obtain a second edge segmentation mask, including: processing the first training image through the first encoder to obtain a first feature map, a second feature map, a third feature map and a fourth feature map; downsampling the first feature map through the first downsampling module to obtain a fifth feature map; downsampling the second feature map through the second downsampling module to obtain a sixth feature map; downsampling the third feature map through the third downsampling module to obtain a seventh feature map; edge-aware processing is performed on the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map through the edge-aware gating module to obtain an edge-aware feature map; and the second edge segmentation mask is determined according to the edge-aware feature map.

[0012] In some embodiments, the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map are edge-aware processed by the edge-aware gating module to obtain an edge-aware feature map, including: performing feature fusion processing on the fifth feature map, the sixth feature map and the seventh feature map to obtain a fused feature map; cascading the fused feature map with the fourth feature map to obtain a cascade feature map; activating the cascade feature and then performing weighted processing and summing processing with the fourth feature map to obtain the edge-aware feature map.

[0013] An embodiment of the present disclosure provides a medical image segmentation device based on contrast difference review, including: a first training image acquisition module, a first edge segmentation mask generation module, a second edge segmentation mask determination module, a first inconsistent region determination module, a third inconsistent region generation module, a first consistency loss generation module, a global loss generation module and a training module.

[0014] Among them, the first training image acquisition module is used to acquire the first training image and the mask label of the first training image; the first edge segmentation mask generation module can be used to perform edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask; the second edge segmentation mask determination module can be used to perform edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; the first inconsistent region determination module can be used to determine a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; the third inconsistent region generation module can be used to determine a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; the first consistency loss generation module can be used to determine a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; the global loss generation module can be used to determine a global loss based on the first consistency loss and the second consistency loss; a training module is used to train the target network through the global loss so as to perform image segmentation through the target network.

[0015] An embodiment of the present disclosure proposes an electronic device, which includes: a memory and a processor; the memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement any of the medical image segmentation methods based on contrast difference review described above.

[0016] The embodiment of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon for implementing the medical image segmentation method based on contrast difference review as described in any one of the above items.

[0017] The present disclosure provides a computer program product or a computer program, which includes computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the above-mentioned medical image segmentation method based on contrast difference review.

[0018] The medical image segmentation method, device, electronic device, computer-readable storage medium and computer program product based on contrast difference review provided by the embodiments of the present disclosure can, on the one hand, make the second edge segmentation mask and the mask label as consistent as possible through a first consistency loss; on the other hand, can also make the first edge segmentation mask and the mask label as consistent as possible through a second consistency loss; finally, through the global loss, the first edge segmentation mask and the second edge segmentation mask can be made as consistent as possible with the mask label, thereby improving the ability of the target network to perform image segmentation.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram of a scenario of a medical image segmentation method based on contrast difference review or a medical image segmentation device based on contrast difference review that can be applied to an embodiment of the present disclosure is shown.

[0022] Figure 2 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0023] Figure 3 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0024] Figure 4 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0025] Figure 5 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0026] Figure 6 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0027] Figure 7 The diagram is a schematic diagram showing the structure of a target network according to an exemplary embodiment.

[0028] Figure 8 is a structural diagram showing a first structure according to an exemplary embodiment.

[0029] Fig. 9 is a structural diagram showing a second structure according to an exemplary embodiment.

[0030] Fig.10 The figure is a schematic diagram showing a comparison difference review mechanism according to an exemplary embodiment.

[0031] Fig.11 It is a block diagram of a medical image segmentation device based on contrast difference review according to an exemplary embodiment.

[0032] Fig.12 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0034] Those skilled in the art will appreciate that the embodiments of the present disclosure may be a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0035] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0036] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0037] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0038] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0039] In the description of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a kind of association relationship describing the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different; the terms "comprising", "including" and "having" are used to express the meaning of open inclusion and mean that in addition to the listed elements / components / etc., there may be other elements / components / etc.

[0040] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0042] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0043] Figure 1 A schematic diagram of a scenario of a medical image segmentation method based on contrast difference review or a medical image segmentation device based on contrast difference review that can be applied to an embodiment of the present disclosure is shown.

[0044] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present disclosure.

[0045] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0046] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, virtual reality devices, smart homes, etc.

[0047] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received request data, and feed back the processing results to the terminal device.

[0048] The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as basic cloud computing services such as big data and artificial intelligence platforms, etc. The present disclosure does not impose any restrictions on this.

[0049] The server 105 may, for example, obtain a first training image and a mask label of the first training image; the server 105 may, for example, perform edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask; the server 105 may, for example, perform edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; the server 105 may, for example, determine a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; the server 105 may, for example, determine a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; the server 105 may, for example, determine a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; the server 105 may, for example, determine a global loss based on the first consistency loss and the second consistency loss; the target network is trained through the global loss so as to perform image segmentation through the target network.

[0050] It should be understood that Figure 1 The number of terminal devices, networks and servers in the figure is only for illustration. The server 105 may be a physical server or may be composed of multiple servers. According to actual needs, it may have any number of terminal devices, networks and servers.

[0051] Under the above system architecture, a medical image segmentation method based on contrast difference review is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0052] Figure 2 The method provided in the embodiment of the present disclosure can be executed by any electronic device with computing and processing capabilities. For example, the method can be executed by the above-mentioned Figure 1 The server or terminal device in the embodiment may be executed, or the server and the terminal device may be executed together. In the following embodiments, the server is used as an example for illustration, but the present disclosure is not limited to this.

[0053] Reference Figure 2 The medical image segmentation method based on contrast difference review provided by the embodiment of the present disclosure may include the following steps.

[0054] Step S202: Obtain a first training image and a mask label of the first training image.

[0055] In some embodiments, the first training image may be an image used for neural network training, and the first training image may have a mask label corresponding to it. The mask label may be a label representing the segmentation result through a mask. For example, the first training image may be a medical image, and the mask label may be label information representing the position of an organ or lesion in the medical image through a mask.

[0056] Step S204: performing edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask.

[0057] In some embodiments, the target network may include a first structure and a second structure. The structural types of the first structure and the second structure may be the same, different, or partially the same, and this application does not impose any limitation on this.

[0058] In some embodiments, the parameters of the first structure and the second structure may be different or partially the same, and this application does not impose any limitation on this.

[0059] In some embodiments, the first structure can be any type of network, such as a deep neural network, a machine learning network, etc. This application does not impose any restrictions on this. Any network structure that can perform image segmentation can be the first structure.

[0060] In some embodiments, the first training image may be processed by edge prediction through the first structure to predict the edge of an object (such as an organ, a lesion area, etc.) in the first training image and represent it through a mask. The mask predicted by the first structure may be referred to as a first edge segmentation mask.

[0061] Step S206: Perform edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask.

[0062] In some embodiments, the first training image may be processed for edge prediction by the second structure of the target network to predict the edge of an object (such as an organ, a lesion area, etc.) in the first training image and represent it by a mask. The mask predicted by the second structure may be referred to as a second edge segmentation mask.

[0063] Step S208 : determining a first inconsistent region between the first edge segmentation mask and the second edge segmentation mask, and a second inconsistent region between the first edge segmentation mask and the mask label.

[0064] In some embodiments, the first edge segmentation mask may be compared with the second edge segmentation mask to determine a region of the first edge segmentation mask that is inconsistent with the second edge segmentation mask as a first inconsistent region.

[0065] In some embodiments, the first inconsistent region may be used to describe a portion in the first edge segmentation mask that is inconsistent with the second edge segmentation mask.

[0066] In some embodiments, the first edge segmentation mask may be further compared with the mask label to determine a region that is inconsistent with the mask label for the first edge segmentation mask as a second inconsistent region.

[0067] In some embodiments, the second inconsistent region may be used to describe a location in the first edge segmentation mask that is inconsistent with the mask label.

[0068] Step S210 : determining a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask and a fourth inconsistent region of the second edge segmentation mask relative to the mask label.

[0069] In some embodiments, a region that is inconsistent with the first edge segmentation mask for the second edge segmentation mask may be determined as a third inconsistent region.

[0070] In some embodiments, the third inconsistent region may be used to describe a portion of the second edge segmentation mask that is inconsistent with the first edge segmentation mask.

[0071] In some embodiments, a region inconsistent with the mask label for the second edge segmentation mask may also be determined as a fourth inconsistent region.

[0072] In some embodiments, the fourth inconsistent region may be used to describe a location in the second edge segmentation mask that is inconsistent with the mask label.

[0073] Step S212: determining a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region.

[0074] In some embodiments, a consistency constraint term between the first inconsistent region and the second inconsistent region may be constructed. For example, a mean square error loss function between the first inconsistent region and the second inconsistent region may be calculated as the first consistency loss.

[0075] In some embodiments, a consistency constraint term between the third inconsistent region and the fourth inconsistent region may be constructed. For example, a mean square error loss function between the third inconsistent region and the fourth inconsistent region may be calculated as the second consistency loss.

[0076] Step S214: determining a global loss according to the first consistency loss and the second consistency loss.

[0077] In some embodiments, the first consistency loss and the second consistency loss may be added together as a global loss.

[0078] Step S216, training the target network through the global loss so as to perform image segmentation through the target network.

[0079] In the above embodiment, on the one hand, the first consistency loss can be used to make the second edge segmentation mask and the mask label as consistent as possible; on the other hand, the second consistency loss can be used to make the first edge segmentation mask and the mask label as consistent as possible; finally, the global loss can be used to make the first edge segmentation mask and the second edge segmentation mask as consistent as possible with the mask label, thereby improving the ability of the target network to perform image segmentation.

[0080] At present, deep learning-based medical image segmentation tasks usually require a large amount of labeled data to train the model. Obtaining these pixel-level labels not only requires the professional knowledge of clinicians, but also requires a lot of effort and time, which is a challenging task. At present, the use of semi-supervised methods to improve the performance of medical image segmentation models using limited labeled data and a large amount of unlabeled data has attracted more and more research attention.

[0081] Among the semi-supervised learning methods, the semi-supervised method based on consistency regularization has gradually become one of the mainstream technologies. Based on the above problems, this embodiment also proposes a semi-supervised segmentation method based on contrast difference review. This embodiment proposes a contrast difference review strategy to introduce more constraints to the supervised training of labeled data, reduce the impact of cognitive bias in the model, and improve the segmentation accuracy of the lesion edge area.

[0082] Figure 3 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0083] refer to Figure 3 , the above-mentioned medical image segmentation method based on contrast difference review may include the following steps.

[0084] In implementation Figure 3 Before the embodiment shown, it is necessary to implement Figure 2 The illustrated embodiment is used to obtain a first consistency loss and a second consistency loss.

[0085] Step S302: Acquire a second training image.

[0086] In some embodiments, the second training image may be an image used for training. The second training image may not have a mask label.

[0087] Step S304: perform edge prediction processing on the second training image through the first structure to obtain a third edge segmentation mask.

[0088] In some embodiments, edge prediction processing can be performed on the second training image through the first structure of the target network to obtain a third edge segmentation mask.

[0089] Step S306: Perform edge prediction processing on the second training image through the second structure to obtain a fourth edge segmentation mask.

[0090] In some embodiments, edge prediction processing may be performed on the second training image using the second structure of the target network to obtain a fourth edge segmentation mask.

[0091] Step S308: determining a fifth inconsistent region of the third edge segmentation mask relative to the fourth edge segmentation mask.

[0092] In some embodiments, for the third edge segmentation mask, an inconsistent region with the fourth edge segmentation mask may be determined as a fifth inconsistent region. The fifth inconsistent region may be used to describe a portion in the third edge segmentation mask that is inconsistent with the fourth edge segmentation mask.

[0093] Step S310: determining a sixth inconsistent region of the fourth edge segmentation mask relative to the third edge segmentation mask.

[0094] In some embodiments, for the fourth edge segmentation mask, an inconsistent region with the third edge segmentation mask may be determined as a sixth inconsistent region. The sixth inconsistent region may be used to describe a portion of the fourth edge segmentation mask that is inconsistent with the third edge segmentation mask.

[0095] Step S312: determining a third consistency loss between the fifth inconsistent region and the sixth inconsistent region.

[0096] Step S314: determining a global loss according to the first consistency loss, the second consistency loss, and the third consistency loss.

[0097] In some embodiments, a mean square error loss function between the fifth inconsistent region and the sixth inconsistent region may be constructed as the third consistency loss.

[0098] In some embodiments, the first consistency loss, the second consistency loss, and the third consistency loss may be added together as a global loss.

[0099] In some embodiments, the target network can be firstly trained in a supervised manner using the first consistency loss and the second consistency of supervised training, so that the target network can roughly segment the edge of the image; and then the target network can be trained in an unsupervised manner using the third consistency loss of unsupervised training to adjust the training results of the supervised training, thereby improving the accuracy of edge segmentation while using a small number of labels.

[0100] In some embodiments, the target network can be trained in batches, and each batch of data can include some labeled training data and some unlabeled training data; then the first consistency loss, the second consistency loss, and the third consistency loss of the batch data are added together as the global loss of the batch data, and then the target network is trained using the global loss of the batch data to improve the edge segmentation capability of the target network.

[0101] The above embodiment proposes a comparative difference review strategy to introduce more constraints to the supervised training of labeled data, reduce the impact of cognitive bias in the model, and improve the segmentation accuracy of the lesion edge area.

[0102] In some embodiments, the first structure of the target network may include an encoder (eg, a first encoder) and a decoder (eg, a first decoder).

[0103] In some embodiments, performing edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask may include: performing feature encoding processing on the first training image through a first encoder to obtain an encoded feature map; and decoding the encoded feature map through a first decoder to obtain a first edge segmentation mask.

[0104] Below, this application will describe the processing process of the above first encoder in detail.

[0105] Figure 4 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0106] In some embodiments, the first encoder may include a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module, and the above-mentioned encoding feature map may include a first feature map, a second feature map, a third feature map and a fourth feature map.

[0107] In some embodiments, the first encoder may include five groups of hybrid coding modules (such as the first to fifth coding modules above), and each group of hybrid coding modules may be formed by cascading a residual convolution block and a self-attention coding block.

[0108] refer to Figure 4, the above-mentioned medical image segmentation method based on contrast difference review may include the following steps.

[0109] Step S402: performing feature extraction processing on the first training image through the first encoding module and the second encoding module in sequence to obtain a first feature map.

[0110] In some embodiments, the first training image may be first subjected to feature extraction processing by a first encoding module, and then the output result of the first encoding module may be input into a second encoding module, and finally the output result of the second encoding module may be used as the first feature map.

[0111] Step S404: Perform feature extraction processing on the first feature map through a third encoding module to obtain a second feature map.

[0112] It is understandable that the low-level feature map of the image is rich in edge information, and the high-level features of the image are more friendly to texture feature extraction.

[0113] Obviously, compared with the first feature map, the second feature map has an additional third encoding module for feature extraction.

[0114] In some embodiments, the feature level of the first feature map is lower than that of the second feature map, that is, the edge / contour information in the first feature map is more than that in the second feature map.

[0115] Step S406: Perform feature extraction processing on the second feature map through a fourth encoding module to obtain a third feature map.

[0116] Step S408: Perform feature extraction processing on the third feature map through a fifth encoding module to obtain a fourth feature map.

[0117] Similarly, it can be understood that the feature levels of the first feature map, the second feature map, the third feature map, and the fourth feature map gradually increase, the edge information gradually decreases, but the texture information gradually increases.

[0118] In some embodiments, the output feature maps of the second to fifth hybrid encoding modules may be labeled as first to fourth feature maps (F1 to F4).

[0119] Step S410: decoding the encoded feature map through a first decoder to obtain a first edge segmentation mask.

[0120] In some embodiments, the first decoder may be composed of three groups of hybrid decoding modules and one group of residual convolution modules, wherein the hybrid decoding module is formed by cascading a self-attention decoding block and a residual convolution block, and is connected to the output of a hybrid encoding module at the same level.

[0121] In some embodiments, the first to fourth feature maps may be decoded correspondingly by a hybrid decoding module and a residual convolution module in the first decoder to obtain a first edge segmentation mask having the same resolution as the first training image.

[0122] The technical solution provided by the above embodiment, the feature map extracted by the first to fifth encoders is rich in both edge features and texture features, which is convenient for both edge feature extraction and global feature extraction.

[0123] Figure 5 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0124] In some embodiments, the second structure of the target network may include a first encoder, a first downsampling module, a second downsampling module, a third downsampling module, and an edge-aware gating module, wherein the first structure and the second structure may share one encoder.

[0125] Step S502: Process the first training image through the first encoder to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map.

[0126] In some embodiments, the first training image can be processed by the first to fifth encoding modules of the first encoder to obtain the first to fourth feature maps. Figure 4 The embodiment shown is not described in detail in this embodiment.

[0127] Step S504: downsample the first feature map by using a first downsampling module to obtain a fifth feature map.

[0128] Step S506: Perform a downsampling operation on the second feature map through a second downsampling module to obtain a sixth feature map.

[0129] Step S508: Perform a downsampling operation on the third feature map through a third downsampling module to obtain a seventh feature map.

[0130] In some embodiments, the first to third down-sampling modules may respectively perform down-sampling operations on the first to third feature maps (F1 to F3) to obtain the fifth to seventh feature maps (f1 to f3), respectively.

[0131] Step S510: Perform edge-aware processing on the fourth feature map, the fifth feature map, the sixth feature map, and the seventh feature map through an edge-aware gating module to obtain edge-aware feature maps.

[0132] In some embodiments, because the low-level feature map contains a large amount of edge information, the edge information can be accurately extracted by performing edge-aware processing on the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map through the edge-aware gating module.

[0133] Step S512: determining a second edge segmentation mask according to the edge-aware feature map.

[0134] In some embodiments, the edge-aware feature map may be directly used as the second edge segmentation mask; or the edge-aware feature map may be processed by a convolution module or the like to generate the second edge segmentation mask.

[0135] Through the technical solution provided by the above embodiment, edge information can be accurately extracted from each feature map of the first training image, so as to accurately generate an edge segmentation mask of the first training image.

[0136] Figure 6 The present invention is a flowchart of a medical image segmentation method based on contrast difference review according to an exemplary embodiment.

[0137] refer to Figure 6 , the above-mentioned medical image segmentation method based on contrast difference review may include the following steps.

[0138] Step S602: perform feature fusion processing on the fifth feature map, the sixth feature map, and the seventh feature map to obtain a fused feature map.

[0139] In some embodiments, the fifth feature map f1, the sixth feature map f2, and the seventh feature map f3 may be subjected to feature fusion processing to obtain a fused feature map.

[0140] Step S604, cascade processing is performed on the fused feature map and the fourth feature map to obtain a cascade feature map.

[0141] In some embodiments, the fused feature map and the fourth feature map F4 may be cascaded to obtain a cascaded feature map.

[0142] It can be understood that the fourth feature map contains rich edge features, so the cascade feature map also includes rich edge features.

[0143] Step S606, after activating the cascaded features, weighted processing and summing the cascaded features with the fourth feature map are performed to obtain an edge-aware feature map.

[0144] Through the above method, the edge-aware feature map finally obtained can have a stronger perception of edge information, so as to better depict the edge information in the first training map.

[0145] The present application also proposes a semi-supervised learning network based on contrastive difference review, which can be used for medical image segmentation and mainly includes the following steps.

[0146] (1) Data collection. In compliance with relevant laws and regulations, obtain the patient's lung imaging data containing lesions such as nodules or masses, divide all imaging data into a training set and a validation set, and divide the training set into labeled data and unlabeled data.

[0147] (2) Construct a segmented subnetwork A (i.e., the first structure of the target network) for long-range relationship modeling and obtain the prediction results of subnetwork A (i.e. the edge segmentation mask output by the first structure).

[0148] (3) Construct a segmentation subnetwork B (i.e., the second structure of the target network) for edge feature perception and obtain the prediction results of subnetwork B. (i.e. the edge segmentation mask output by the second structure).

[0149] (4) For supervised training with labeled data, a comparative difference review mechanism is constructed to obtain the difference areas of the labels. (i.e., the inconsistent area after comparing the result with the label mask) and the difference area of ​​the predicted result (i.e., inconsistent areas after comparison with the predicted edge segmentation mask).

[0150] (5) Based on the difference area of ​​the label The difference between the predicted results and the Construct the supervised training loss function.

[0151] (6) For unsupervised training of unlabeled data, an unsupervised loss function based on consistency learning and a total loss function are constructed to update the model.

[0152] Specific implementations may include the following.

[0153] 1. Data Collection Phase (1.1) In compliance with relevant laws and regulations, obtain the patient's lung imaging data containing lesions such as nodules or masses (which must comply with relevant laws and regulations), and divide all imaging data into a training set and a validation set.

[0154] (1.2) For the training set, the lesion areas in 20% of the data are annotated at the pixel level to obtain the labeled data XL and the corresponding label Label.

[0155] (1.3) The remaining 80% of the training data is defined as unlabeled data XU.

[0156] In some embodiments, a target network may be constructed first. The target network may include subnet A (i.e., a first structure, such as Figure 7 701 in) and subnet B (i.e., the second structure, such as Figure 7 702 in ). Figure 7 As shown, the first structure 701 may include an encoder 7011 and a decoder 7012; the second structure 702 may include an encoder 7011, an edge perception module 7021, and a decoder 7022. The first structure 701 and the second structure 702 may share one encoder 7011.

[0157] 2. Construct a segmented subnetwork A for long-range relationship modeling (i.e., the first structure of the target network, see Figure 7 701 in the figure), and obtain the prediction result of subnet A (i.e. the edge segmentation mask output by subnet A).

[0158] (2.1) The segmentation subnetwork A can adopt an encoder-decoder network structure. The network structure of the segmentation subnetwork A can be as follows: Figure 8 .

[0159] (2.2) If Figure 8 As shown, the encoder 801 in the segmentation subnet A can be composed of 5 groups of hybrid encoding modules, each group of hybrid modules can be cascaded by a residual convolution block and a self-attention encoder, where the output feature maps of the 2nd, 3rd, 4th, and 5th groups of hybrid blocks can be labeled as F1, F2, F3, and F4.

[0160] (2.3) If Figure 8 As shown, the decoder 802 in the segmentation subnet A can be composed of 3 groups of hybrid decoding modules and 1 group of residual convolution blocks, wherein the hybrid decoding module can be cascaded by a self-attention decoding block and a residual convolution block, and connected to the output of a hybrid encoding module at the same level.

[0161] (2.4) The output of the third group of hybrid decoding modules is input into the residual convolution block after upsampling operation, and finally the lesion segmentation mask with the same resolution as the initial input image is obtained. (i.e., the edge segmentation mask output by subnet A).

[0162] 3. Construct a segmentation subnetwork B for edge feature perception (i.e. the second structure, for details, please refer to Figure 7 702 shown in FIG. 702 ), the prediction result of subnet B is obtained (i.e. the edge segmentation mask output by subnet B).

[0163] (3.1) If Figure 7As shown, subnet B 702 shares encoder 7011 with subnet A 701, and can input feature maps F1, F2, F3, and F4 obtained in (2.2) into edge-aware modules (EMA) 7021. The edge-aware modules 7021 fully consider the network's use of low-level feature map information, so that the features can increase the attention to edge features before entering the decoder 7022 of subnet B.

[0164] (3.2) The network structure of the edge perception module can be as follows Fig. 9 As shown in 901, the input data is the feature maps F1, F2, F3, and F4 from the encoder. In the edge perception module, feature map F1 is downsampled 16 times to obtain feature map f1, feature map F2 is downsampled 8 times to obtain feature map f2, and feature map F3 is downsampled 4 times to obtain feature map f3. After feature concatenation of f1, f2, and f3, dimension reduction is performed through a convolution layer with a convolution kernel of 1×1 to obtain a fused feature Fs. Fs is feature concatenated with feature map F4, weighted by an edge perception gating function, and then input into a convolution layer to obtain the output of the edge perception module, that is, the feature map Fe with edge perception capability. The calculation process is as follows:

[0165] in, represents a convolution with a kernel of 3×3, represents the Sigmoid activation function, Represents a feature cascade.

[0166] (3.3) If Figure 7 As shown, the high-dimensional feature map containing edge information can be The 4-layer decoder 7022 of the input subnet B is gradually upsampled to the initial resolution size. The decoder can use bilinear upsampling operation to obtain the lesion segmentation mask of subnet B. (i.e. the edge segmentation mask output by subnet B).

[0167] (3.4) For the labeled data XL and the corresponding label Label, when performing supervised training, the output of subnet A is obtained (i.e. the first edge segmentation mask) and the output of subnet B (i.e., the second edge segmentation mask), and Input Comparative Differential Review (CDR) module to build constraints for supervised training. The network framework is as follows Fig.10 shown.

[0168] (4) For supervised training with labeled data, a comparative difference review mechanism is constructed to obtain the difference areas of the labels. (i.e., the inconsistent areas after comparison with the label data) and the different areas of the predicted results (i.e., areas of inconsistency after comparison with the predicted structure).

[0169] (4.1) In order to correct the influence of the model's cognitive bias, a comparative difference review module is constructed to treat the prediction differences of the sub-network as potential bias areas and guide the sub-network to correct them. The calculation process of the CDR module is as follows: Fig.10 shown.

[0170] (4.2) If Fig.10 As shown, the output of subnet A can be (Right now Fig.10 The first edge segmentation mask in ) and the output of subnet B (i.e., the second edge segmentation mask) is superimposed. Since one of the two regions with inconsistent predictions must be wrong, first obtain the region with respect to In terms of The forecast inconsistent area (Right now Fig.10 ), and secondly, obtain the first inconsistent area relative to In terms of The forecast inconsistent area (Right now Fig.10 The third inconsistent region in ).

[0171] (4.3) Subnet A’s output (i.e., the first edge segmentation mask) overlaps with the lesion label mask Label to obtain For the inconsistent area between Label (Right now Fig.10 2nd inconsistent region in ).

[0172] (4.4) Subnet B’s output (i.e., the second edge segmentation mask) overlaps with the lesion label mask Label to obtain For the inconsistent area between Label (Right now Fig.10 4 inconsistent region in ).

[0173] (4.5) Constructing the difference region of labels The difference between the predicted results and the Consistency constraints between and .

[0174]

[0175]

[0176] Among them, MSE(-) represents the mean square error (MSE) loss function.

[0177] (5) Construct the supervised training loss function Lsup.

[0178] (5.1) The supervision loss includes the common lesion segmentation loss and the correction loss from CDR , which can be expressed as:

[0179] Among them, Dice represents weighted dice loss, CE represents cross entropy loss, is a hyperparameter and Label is a mask label.

[0180] (5.2) For the correction loss , which consists of a consistency constraint between the labels and the difference regions predicted by the two subnetworks.

[0181]

[0182] (6) For unsupervised training of unlabeled data, an unsupervised loss function Lconsis based on consistency learning and a total loss function Loss are constructed to update the model.

[0183] (6.1) For the training of unlabeled data XU, a consistency constraint on the prediction results between the two subnetworks is constructed, and the unsupervised loss function is expressed as follows.

[0184]

[0185] Among them, MSE(-) represents the mean square error (MSE) loss function, Represents the output result of subnet A during unsupervised training; Represents the output of subnet B during unsupervised training.

[0186] (6.2) The total loss function Loss consists of supervised loss and unsupervised loss, which is expressed as follows.

[0187]

[0188] in, is a Gaussian slope curve.

[0189] In the above scheme, segmentation subnet A and segmentation subnet B share an encoder, which consists of a convolution module and an efficient self-attention module, and can extract local and global features of the input image. Segmentation subnet A and segmentation subnet B use different decoders. The decoder of subnet A uses conventional bilinear upsampling and adds jump connections to aggregate feature map information at different levels, thereby improving the network's attention to remote context features. For the feature map output by the encoder of the highest dimension of segmentation subnet B, it is first input into an edge-aware module (EAM) as a link between the encoder and the decoder, and the low-dimensional edge features and high-dimensional fine-grained features are fused before being input into the decoder, thereby improving the network's attention to edge features. The loss function consists of a supervised loss Lsup and an unsupervised loss Lconsis. The supervised loss consists of a cross entropy loss, a Dice loss, and a consistency correction loss based on contrast difference review. The semi-supervised loss uses a mean square error loss function based on consistency learning, which can provide the accuracy of edge segmentation.

[0190] It should be particularly pointed out that the various steps in the various embodiments of the above-mentioned medical image segmentation method based on contrast difference review can be mutually crossed, replaced, added, and deleted. Therefore, these reasonable permutations and combinations of the medical image segmentation method based on contrast difference review should also belong to the protection scope of the present disclosure, and the protection scope of the present disclosure should not be limited to the embodiments.

[0191] Based on the same inventive concept, the present disclosure also provides a medical image segmentation device based on contrast difference review, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0192] Fig.11 is a block diagram of a medical image segmentation device based on contrast difference review according to an exemplary embodiment. Fig.11 The medical image segmentation device 1100 based on contrast difference review provided by the embodiment of the present disclosure may include: a first training image acquisition module 1101, a first edge segmentation mask generation module 1102, a second edge segmentation mask determination module 1103, a first inconsistent region determination module 1104, a third inconsistent region generation module 1105, a first consistency loss generation module 1106, a global loss generation module 1107 and a training module 1108.

[0193] Among them, the first training image acquisition module 1101 can be used to obtain the first training image and the mask label of the first training image; the first edge segmentation mask generation module 1102 can be used to perform edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask; the second edge segmentation mask determination module 1103 can be used to perform edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; the first inconsistent region determination module 1104 can be used to determine a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; The third inconsistent region generation module 1105 can be used to determine the third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and the fourth inconsistent region of the second edge segmentation mask relative to the mask label; the first consistency loss generation module 1106 can be used to determine the first consistency loss between the first inconsistent region and the second inconsistent region and the second consistency loss between the third inconsistent region and the fourth inconsistent region; the global loss generation module 1107 can be used to determine the global loss based on the first consistency loss and the second consistency loss; the training module 1108 can be used to train the target network through the global loss so as to perform image segmentation through the target network.

[0194] It should be noted here that the above-mentioned first training image acquisition module 1101, first edge segmentation mask generation module 1102, second edge segmentation mask determination module 1103, first inconsistent region determination module 1104, third inconsistent region generation module 1105, first consistency loss generation module 1106, global loss generation module 1107 and training module 1108 correspond to S202 to S216 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0195] In some embodiments, the medical image segmentation device 1100 based on comparative difference review may also include: a second training image acquisition submodule, a third edge segmentation mask acquisition submodule, a fourth edge segmentation mask acquisition submodule, a fifth inconsistent region determination submodule, a sixth inconsistent region determination submodule and a third consistency loss determination submodule.

[0196] Among them, the second training image acquisition submodule can be used to acquire the second training image; the third edge segmentation mask acquisition submodule can be used to perform edge prediction processing on the second training image through the first structure to obtain a third edge segmentation mask; the fourth edge segmentation mask acquisition submodule can be used to perform edge prediction processing on the second training image through the second structure to obtain a fourth edge segmentation mask; the fifth inconsistent area determination submodule can be used to determine the fifth inconsistent area of ​​the third edge segmentation mask relative to the fourth edge segmentation mask; the sixth inconsistent area determination submodule can be used to determine the sixth inconsistent area of ​​the fourth edge segmentation mask relative to the third edge segmentation mask; the third consistency loss determination submodule can be used to determine the third consistency loss between the fifth inconsistent area and the sixth inconsistent area.

[0197] In some embodiments, the global loss generation module 1107 may include: a global loss determination submodule.

[0198] Among them, the global loss determination submodule can be used to determine the global loss according to the first consistency loss, the second consistency loss and the third consistency loss.

[0199] In some embodiments, the first structure includes a first encoder and a first decoder; wherein the first edge segmentation mask generation module 1102 includes: an encoding submodule and a decoding submodule.

[0200] Among them, the encoding submodule can be used to perform feature encoding processing on the first training image through a first encoder to obtain a coding feature map; the decoding submodule can be used to decode the coding feature map through a first decoder to obtain a first edge segmentation mask.

[0201] In some embodiments, the first encoder includes a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module, and the encoding feature map includes a first feature map, a second feature map, a third feature map and a fourth feature map; wherein the encoding submodule may include: a first feature map determination unit, a second feature map determination unit, a third feature map determination unit and a fourth feature map determination unit.

[0202] Among them, the first feature map determination unit can be used to perform feature extraction processing on the first training image through the first encoding module and the second encoding module in sequence to obtain a first feature map; the second feature map determination unit can be used to perform feature extraction processing on the first feature map through the third encoding module to obtain a second feature map; the third feature map determination unit can be used to perform feature extraction processing on the second feature map through the fourth encoding module to obtain a third feature map; the fourth feature map determination unit can be used to perform feature extraction processing on the third feature map through the fifth encoding module to obtain a fourth feature map.

[0203] In some embodiments, the second structure includes a first encoder, a first downsampling module, a second downsampling module, a third downsampling module and an edge-aware gating module; wherein the first structure and the second structure share an encoder; wherein the second edge segmentation mask determination module 1103 may include: a feature map generation submodule, a fifth feature map determination submodule, a sixth feature map determination submodule, a seventh feature map determination submodule, an edge-aware feature map determination submodule and an edge-aware submodule.

[0204] Among them, the feature map generation submodule can be used to process the first training image through the first encoder to obtain the first feature map, the second feature map, the third feature map and the fourth feature map; the fifth feature map determination submodule can be used to downsample the first feature map through the first downsampling module to obtain the fifth feature map; the sixth feature map determination submodule can be used to downsample the second feature map through the second downsampling module to obtain the sixth feature map; the seventh feature map determination submodule can be used to downsample the third feature map through the third downsampling module to obtain the seventh feature map; the edge-aware feature map determination submodule can be used to perform edge-aware processing on the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map through the edge-aware gating module to obtain the edge-aware feature map; the edge-aware submodule can be used to determine the second edge segmentation mask according to the edge-aware feature map.

[0205] In some embodiments, the edge-aware feature map determination submodule may include: a fused feature map acquisition unit, a cascade feature map determination unit, and a weighted summation unit.

[0206] Among them, the fused feature map acquisition unit can be used to perform feature fusion processing on the fifth feature map, the sixth feature map and the seventh feature map to obtain a fused feature map; the cascade feature map determination unit can be used to perform cascade processing on the fused feature map and the fourth feature map to obtain a cascade feature map; the weighted summation unit can be used to activate the cascade features and then perform weighted processing and summation processing with the fourth feature map to obtain an edge-perceived feature map.

[0207] Since the functions of the device 1100 have been described in detail in the corresponding method embodiments, the present disclosure will not elaborate on them here.

[0208] The modules and / or submodules and / or units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The modules and / or submodules and / or units described may also be arranged in a processor. The names of these modules and / or submodules and / or units do not, in certain circumstances, constitute limitations on the modules and / or submodules and / or units themselves.

[0209] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a part of a module or program segment, and a part of the above-mentioned module or program segment contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer program instructions.

[0210] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0211] Fig.12 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Fig.12 The electronic device 1200 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0212] like Fig.12 As shown, the electronic device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 to a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0213] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0214] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains computer program instructions for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1209, and / or installed from a removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the present disclosure are executed.

[0215] It should be noted that the computer-readable storage medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable computer program instructions. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The computer program instructions contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0216] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by a device, the device can realize functions including: obtaining a first training image and a mask label of the first training image; performing edge prediction processing on the first training image through the first structure of the target network to obtain a first edge segmentation mask; performing edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; determining a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; determining a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; determining a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; determining a global loss based on the first consistency loss and the second consistency loss; training the target network through the global loss so as to perform image segmentation through the target network.

[0217] According to one aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer program instructions, the computer program instructions being stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the methods provided in various optional implementations of the above embodiments.

[0218] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of computer program instructions for enabling an electronic device (which can be a server or a terminal device, etc.) to execute the method according to the embodiment of the present disclosure.

[0219] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0220] It should be understood that the present disclosure is not limited to the detailed structures, drawings or implementations shown herein, but rather the present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A medical image segmentation method based on contrast difference review, characterized in that: include: Obtaining a first training image and a mask label of the first training image; Performing edge prediction processing on the first training image through a first structure of a target network to obtain a first edge segmentation mask; Performing edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; Determine a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask and a second inconsistent region of the first edge segmentation mask relative to the mask label; Determine a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; determining a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; Determining a global loss according to the first consistency loss and the second consistency loss; The target network is trained by the global loss so as to perform image segmentation by the target network.

2. The method according to claim 1, characterized in that: The method further comprises: acquiring a second training image; Performing edge prediction processing on the second training image through the first structure to obtain a third edge segmentation mask; performing edge prediction processing on the second training image by using the second structure to obtain a fourth edge segmentation mask; determining a fifth inconsistent region of the third edge segmentation mask relative to the fourth edge segmentation mask; determining a sixth inconsistent region of the fourth edge segmentation mask relative to the third edge segmentation mask; determining a third consistency loss between the fifth inconsistent region and the sixth inconsistent region; The determining of the global loss according to the first consistency loss and the second consistency loss includes: The global loss is determined according to the first consistency loss, the second consistency loss, and the third consistency loss.

3. The method according to claim 1, characterized in that: The first structure includes a first encoder and a first decoder; wherein, edge prediction processing is performed on the first training image through the first structure of the target network to obtain a first edge segmentation mask, including: Performing feature encoding processing on the first training image by the first encoder to obtain an encoding feature map; The first decoder is used to decode the encoded feature map to obtain the first edge segmentation mask.

4. The method according to claim 3, characterized in that: The first encoder includes a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module, and the encoding feature map includes a first feature map, a second feature map, a third feature map and a fourth feature map; wherein, the first encoder performs feature encoding processing on the first training image to obtain the encoding feature map, including: performing feature extraction processing on the first training image by the first encoding module and the second encoding module in sequence to obtain the first feature map; Performing feature extraction processing on the first feature map by the third encoding module to obtain the second feature map; Performing feature extraction processing on the second feature map by the fourth encoding module to obtain the third feature map; The third feature map is subjected to feature extraction processing by the fifth encoding module to obtain the fourth feature map.

5. The method according to claim 1, characterized in that: The second structure includes a first encoder, a first downsampling module, a second downsampling module, a third downsampling module and an edge-aware gating module; wherein the first structure and the second structure share an encoder; wherein edge prediction processing is performed on the first training image through the second structure of the target network to obtain a second edge segmentation mask, including: Processing the first training image by the first encoder to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map; Performing a downsampling operation on the first feature map by the first downsampling module to obtain a fifth feature map; Performing a downsampling operation on the second feature map by the second downsampling module to obtain a sixth feature map; Performing a downsampling operation on the third feature map by the third downsampling module to obtain a seventh feature map; Performing edge-aware processing on the fourth feature map, the fifth feature map, the sixth feature map, and the seventh feature map through the edge-aware gating module to obtain an edge-aware feature map; The second edge segmentation mask is determined according to the edge-aware feature map.

6. The method according to claim 5, characterized in that: Performing edge-aware processing on the fourth feature map, the fifth feature map, the sixth feature map, and the seventh feature map through the edge-aware gating module to obtain an edge-aware feature map, including: Performing feature fusion processing on the fifth feature map, the sixth feature map, and the seventh feature map to obtain a fused feature map; Performing cascade processing on the fused feature map and the fourth feature map to obtain a cascade feature map; The cascade feature is activated and then weighted and summed with the fourth feature map to obtain the edge-aware feature map.

7. A medical image segmentation device based on contrast difference review, characterized in that: include: A first training image acquisition module, used to acquire a first training image and a mask label of the first training image; A first edge segmentation mask generating module, configured to perform edge prediction processing on the first training image through a first structure of a target network to obtain a first edge segmentation mask; A second edge segmentation mask determination module is used to perform edge prediction processing on the first training image through the second structure of the target network to obtain a second edge segmentation mask; A first inconsistent region determining module, configured to determine a first inconsistent region of the first edge segmentation mask relative to the second edge segmentation mask, and a second inconsistent region of the first edge segmentation mask relative to the mask label; a third inconsistent region generating module, configured to determine a third inconsistent region of the second edge segmentation mask relative to the first edge segmentation mask, and a fourth inconsistent region of the second edge segmentation mask relative to the mask label; A first consistency loss generating module, configured to determine a first consistency loss between the first inconsistent region and the second inconsistent region and a second consistency loss between the third inconsistent region and the fourth inconsistent region; A global loss generating module, configured to determine a global loss according to the first consistency loss and the second consistency loss; A training module is used to train the target network through the global loss so as to perform image segmentation through the target network.

8. An electronic device, characterized in that: include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the medical image segmentation method based on contrast difference review according to any one of claims 1 to 6 based on computer program instructions stored in the memory.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the medical image segmentation method based on contrast difference review as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising computer program instructions, wherein the computer program instructions are stored in a computer-readable storage medium, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images