Method, device and storage medium for generating training samples

Abstract

The embodiment of the specification provides a training sample generation method, device and equipment and a storage medium. The method comprises the following steps: receiving a medical image; the medical image comprises a coronary region and a main stem region; inputting the medical image into a first mask image generation model to obtain a first mask image; the first mask image comprises a first prediction region for indicating the position of the coronary region in the medical image; determining a second mask image; the second mask image comprises a second prediction region for indicating the position of the main stem region in the medical image; in the case that a target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, taking the target mask image as a label of the medical image to generate a training sample of the first mask image generation model. The training sample optimization model is generated by superimposing the first mask image and the second mask image, so that the accuracy of the blood vessel region identification in the medical image is improved to a certain extent.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of image processing, specifically to a method, apparatus, device, and storage medium for generating training samples. Background Technology

[0002] Coronary digital subtraction angiography (CDA) technology involves digitizing X-ray images taken before and after contrast agent injection, inputting them into an image computer, and obtaining clearer vascular images through processes such as subtraction, enhancement, and re-imaging.

[0003] Currently, deep learning methods can be used to automatically generate mask images of blood vessels, which can be used to extract regions representing blood vessels in the images to assist medical professionals in diagnosis, treatment, and other tasks.

[0004] However, common deep learning methods require a certain number of training samples to ensure the accuracy of the target region represented by the mask image to a certain extent. However, training samples for medical images often require manual annotation, which consumes a lot of time, resulting in deep learning-generated mask images failing to accurately represent blood vessels in medical images. Summary of the Invention

[0005] This specification provides a method, apparatus, device, and storage medium for generating mask images, which improves the accuracy of illustrating blood vessels in medical images using mask images generated based on deep learning.

[0006] This specification provides a method for generating training samples, comprising: receiving a medical image; wherein the medical image includes a coronary artery region representing a coronary artery and a trunk region representing the trunk of the coronary artery; inputting the medical image into a first mask image generation model to obtain a first mask image; wherein the first mask image includes a first predicted region for representing the position of the coronary artery region in the medical image; determining a second mask image including a second predicted region; wherein the second mask image includes a second predicted region for representing the position of the trunk region in the medical image; and, if a target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, using the target mask image as a label for the medical image to generate training samples for the first mask image generation model, for training the first mask image generation model.

[0007] One embodiment of this specification provides a method for generating a mask image, comprising: receiving a medical image; wherein the medical image includes a coronary artery region representing a coronary artery and a trunk region representing the trunk of the coronary artery; determining a first mask image including a first prediction region and a second mask image including a second prediction region; wherein the first prediction region is used to represent the position of the coronary artery region in the medical image; the second prediction region is used to represent the position of the trunk region in the medical image; wherein the first prediction region and the second prediction region each include a plurality of non-adjacent connected regions; removing connected regions in the plurality of connected regions included in the first mask image and the second mask image whose pixel count is lower than a specified threshold; superimposing the first mask image and the second mask image to form a target mask image; wherein the target mask image includes a target prediction region representing the coronary artery region.

[0008] One embodiment of this specification provides a method for training a mask image generation model, comprising: constructing a training sample set for an initial mask image generation model; wherein the training sample set includes training samples composed of a medical image and a target mask image of the medical image; wherein the medical image includes a coronary artery region representing a coronary artery; the target mask image is used to represent the coronary artery region of the medical image; using the medical image as input to the initial mask image generation model, and using the target mask image as the target output of the initial mask image generation model; calculating a prediction loss of the training sample set using the initial mask image generation model; wherein the prediction loss represents the difference between the mask image generated by the initial mask image generation model based on the input medical image and the corresponding target mask image; updating the parameters of the initial mask image generation model to adjust the prediction loss, such that the prediction loss, under specified conditions, allows the initial mask image generation model with the corresponding parameters to be used as the trained mask image generation model.

[0009] One embodiment of this specification provides a computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method described in the above embodiment.

[0010] One embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the above embodiments.

[0011] The various embodiments provided in this specification generate training samples for a first mask image generation model by superimposing a first mask image representing the coronary artery region and a second mask image representing the trunk region into a target mask image, thereby improving the accuracy of the first mask image generation model in identifying regions representing blood vessels in medical images. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a mask image generation system architecture provided for one embodiment of this specification.

[0013] Figure 2a This is a schematic diagram of a method for generating training samples according to one embodiment of this specification.

[0014] Figure 2b This is a schematic diagram of a method for generating training samples according to one embodiment of this specification.

[0015] Figure 2c This is a flowchart illustrating a method for generating training samples according to one embodiment of this specification.

[0016] Figure 3a This is a schematic diagram of a method for generating a mask image according to one embodiment of this specification.

[0017] Figure 3b This is a schematic flowchart illustrating a method for generating a mask image according to one embodiment of this specification.

[0018] Figure 4a This is a schematic diagram of a mask image provided for one embodiment of this specification.

[0019] Figure 4b This is a schematic diagram of a mask image provided for one embodiment of this specification.

[0020] Figure 5a A schematic diagram of a first mask image provided for one embodiment of this specification.

[0021] Figure 5b This is a schematic diagram of a second mask image provided for one embodiment of this specification.

[0022] Figure 5c This is a schematic diagram of a target mask image provided for one embodiment of this specification.

[0023] Figure 6 This is a flowchart illustrating a training method for a mask image generation model provided in one embodiment of this specification.

[0024] Figure 7This is a schematic diagram of a training sample generation apparatus provided for one embodiment of this specification.

[0025] Figure 8 This is a schematic diagram of a mask image generation apparatus provided for one embodiment of this specification.

[0026] Figure 9 This is a schematic diagram of a training apparatus for a mask image generation model provided in one embodiment of this specification.

[0027] Figure 10 A schematic diagram of a computer device provided for one embodiment of this specification. Detailed Implementation

[0028] Overview

[0029] In related technologies, deep learning methods can be used to generate mask images of medical images to represent the regions where target objects are located within the medical images. Specifically, medical images can be vascular images, and deep learning methods can be used to generate regions that represent blood vessels within the vascular images.

[0030] In related technologies, building deep learning models requires a large number of training samples, while the number of training samples for vascular images is relatively small, which is not conducive to the learning of deep learning model parameters. In addition, due to the influence of vascular image quality, deep learning models are prone to overfitting, and the regions representing the main vascular trunks in the generated mask images may be broken or have multiple discrete regions.

[0031] Therefore, it is necessary to provide a method for generating mask images. This method can generate a first mask image to represent the region of blood vessels in a medical image and a second mask image to represent the region of the main trunk of blood vessels in a medical image. The target mask image obtained by fusing the first mask image and the second mask image, which retains the largest connected component, is used as a label for the medical image to generate training samples to optimize the first mask image generation model. This method aims to solve the technical problem of low accuracy of mask images generated by deep learning technology to represent the region of blood vessels in medical images.

[0032] Scenario Example

[0033] Please see Figure 1 This specification provides an example application scenario for a mask image generation system. Users of this mask image generation system may be medical professionals. The system can generate mask images that represent coronary artery regions in medical images, further assisting medical workers in patient diagnosis and treatment. The mask image generation system may include a client and a server.

[0034] Healthcare professionals can upload medical images via a client application. The client application can then send these medical images to a server.

[0035] Please see Figure 2a and Figure 2b After receiving medical images including those of the coronary arteries, the server can preprocess the images using the image preprocessing module 20. Specifically, a bilateral filter can be used to remove some noise from the medical images while preserving information at high-frequency edges. Next, global histogram equalization enhancement is performed on the medical images processed by the bilateral filter to improve their contrast to some extent.

[0036] The medical image may include a coronary artery region representing the coronary arteries and a trunk region representing the main trunk of the coronary arteries. Of course, the medical image may also include non-coronary artery regions that do not represent the coronary arteries and non-trunk regions that do not represent the main trunk of the coronary arteries. After preprocessing the medical image, a first mask image representing the coronary artery region and the non-coronary artery region can be generated using a first mask image generation model, and a second mask image representing the trunk region and the non-trunk region can be generated using a second mask image generation model. The first mask image generation model may be a U-net model (U-shaped network model). The second mask image generation model may be an FCN model (fully convolutional network model).

[0037] Please see Figure 2b A mask image can be generated through the first mask image generation module 22. During the generation of the first mask image, the medical image can first undergo multi-level downsampling to generate multiple first feature maps. Then, based on the first feature map obtained from the last level of downsampling, multi-level upsampling can be performed to obtain multiple second feature maps corresponding to the first feature maps. During the upsampling to generate the second feature maps, the feature map obtained by processing the first feature map of the corresponding level and the second feature map of the previous level through the attention module, and the feature map obtained by upsampling based on the second feature map of the previous level, can be concatenated, and then convolutional processing can be used to generate the second feature map of that level. Additionally, during the sampling process, the feature map can be pooled after passing through the CBAM attention mechanism to obtain the first feature map. The first mask image can be obtained through the result of the last level of upsampling. Considering the intricate and complex structure of blood vessels, the accuracy of coronary artery region prediction is improved by combining the attention mechanism and the U-net network.

[0038] The first mask image may include a first predicted region representing the coronary artery region. The second mask image may include a second predicted region representing the trunk region. Both the first and second mask images can be represented by binary images. The first and second predicted regions can be represented by pixels with a value of 1. Portions in the first mask image that are not part of the first predicted region can be represented by pixels with a value of 0. Portions in the second mask image that are not part of the second predicted region can be represented by pixels with a region value of 0.

[0039] Please see Figure 2b The image post-processing module 26 in the image processing module 26, after obtaining the first and second mask images, retains the most important connected components in the first and second prediction regions while discarding scattered connected components. Specifically, if the number of pixels included in a connected component is greater than a specified threshold, the corresponding connected component is retained. Otherwise, the values ​​of all pixels in the connected component can be set to 0. By retaining the main connected components, the usability of the mask image is ensured to a certain extent. Finally, the first and second mask images, which retain the most important connected components, can be superimposed to generate the target mask image. Specifically, corresponding pixels in the first and second mask images can be ORed to obtain the target mask image.

[0040] If the generated target mask image meets the given conditions, the target mask image can be used as a label for the medical image to generate training samples that can be used to supplement the training of the U-net model, thereby further improving the accuracy of the generated mask image.

[0041] After obtaining the target mask image, the server can send it to the client. The client can then display the target mask image on a display device for reference by medical professionals.

[0042] Additionally, please see Figure 2b In the sampling module 24, during the training of the first mask image, to address the issue of a relatively small total number of data samples, batch normalization and Dropout nesting methods can be used to train the first mask image, thereby improving the robustness of the model to a certain extent.

[0043] System Architecture

[0044] Please see Figure 1This specification provides a mask image generation system. The mask image generation system may include a client and a server. The client can be used to receive user operations, display input medical images, and generate mask images. The server can deploy the mask image generation method to generate a mask image of the received medical image.

[0045] The client can be an electronic device with network access capabilities. Specifically, for example, the client can be a desktop computer, tablet computer, laptop computer, smartphone, digital assistant, smart wearable device, shopping guide terminal, television, smart speaker, microphone, etc. Among them, smart wearable devices include, but are not limited to, smart bracelets, smartwatches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client can also be software that can run on the electronic device. The server can be an electronic device with a certain computing power. It can have a network communication module, processor, and memory, etc. Of course, the server can also refer to the software running on the electronic device. The server can also be a distributed server, which can be a system with multiple processors, memory, network communication modules, etc. working together. Alternatively, the server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server can also be a new technical means that can realize the corresponding functions of the embodiments described in the specification. For example, it can be a new form of "server" based on quantum computing.

[0046] In some implementations, the mask image generation system may consist only of a server. For example, the mask image generation system may process medical images from a database on the server to obtain corresponding mask images. Alternatively, the mask image generation system may consist only of a client. Accordingly, a user can input medical images on the client, and the client can execute a mask image generation method to generate a mask image corresponding to the medical image.

[0047] Example method

[0048] Please see Figure 2c This specification provides a method for generating training samples, which may include the following steps.

[0049] Step S101: Receive medical images; wherein the medical images include a coronary artery region representing the coronary arteries and a trunk region representing the main trunk of the coronary arteries.

[0050] In some cases, after receiving a medical image, the image can be further processed to generate a mask image that can be used to represent a target object in the medical image. Specifically, for example, for a medical image including coronary arteries, the target object can be the coronary arteries in the medical image.

[0051] In this embodiment, the medical image can refer to images or pictures in the medical field. For example, the medical image can be an image obtained by imaging human organs.

[0052] In some cases, the medical image can be an image obtained from imaging the body parts where the coronary arteries (coronary arteries) are located. Specifically, the medical image may include a coronary artery region representing the coronary arteries, and a trunk region representing the main trunk of the coronary arteries. The coronary artery region may include the trunk region. Of course, the medical image may also include non-coronary artery regions. The coronary artery region may represent the area composed of pixels obtained from imaging the coronary arteries of the human body. The trunk region may represent the area composed of pixels obtained from imaging the main trunk of the coronary arteries. The coronary artery region can serve as the foreground of the medical image. Additionally, the medical image may also include non-coronary artery regions obtained from imaging non-coronary artery sites. For example, the medical image may also include non-coronary artery regions obtained from imaging cardiac tissue. The non-coronary artery regions can serve as the background of the medical image.

[0053] In this embodiment, the mask image can be used to represent target and non-target regions of a medical image. The target region can represent the area in the medical image where the object to be identified is located. Specifically, the mask image can classify pixels in the medical image. For example, pixels in the medical image can be divided into pixels of the target region and pixels of the non-target region. The target region can be the coronary artery region or the main trunk region. In some embodiments, the mask image can extract the corresponding target region from the medical image. Specifically, the mask image can be a binary image corresponding to the medical image. The binary image can clearly show the shape of blood vessels. The mask image can include pixels with a value of 1 and pixels with a value of 0. The region formed by pixels with a value of 1 can represent the target region of the medical image. Of course, there can be various implementations for the pixel values ​​and the setting scheme of the target region to be extracted. For example, there can be multiple pixel values, and the pixels of the medical image can be divided into multiple categories. Correspondingly, the medical image can be divided into multiple regions. The mask image can be used to extract at least one region from these regions. Of course, the mask image can also be an image that has not undergone binarization. The pixel value in the mask image can be a value between 0 and 1, used to represent the probability that the corresponding pixel belongs to the target region.

[0054] The method for receiving medical images can involve the server receiving the medical images sent by the client and generating a corresponding target mask image. Alternatively, the method can involve the server requesting medical images from a database, which then sends the images to the server for receipt.

[0055] Step S102: Input the medical image into the first mask image generation model to obtain the first mask image; wherein, the first mask image includes a first prediction region for representing the location of the coronary artery region in the medical image.

[0056] In some cases, a first mask image of the medical image can be generated using a machine learning model.

[0057] In this embodiment, the first mask image generation model can be used to generate a machine learning model representing the coronary artery region in a medical image. Specifically, the first mask image generation model can be a U-net model. Of course, the first mask image generation model can also be an FCN model. Correspondingly, the method of inputting the medical image into the first mask image generation model to obtain the first mask image can be generated by the first mask image generation model.

[0058] In this embodiment, the first mask image can be a mask image representing both the coronary artery region and non-coronary artery region of the medical image. The first mask image may include a first predicted region representing the coronary artery region. This first predicted region may represent the area in the medical image where the coronary arteries are located. Specifically, for example, the size of the first mask image may be the same as the medical image. Each pixel of the first mask image may correspond to a pixel in the medical image. Pixels in the first predicted region can be used to indicate that the corresponding pixel in the medical image belongs to the predicted coronary artery region. In some embodiments, the pixel values ​​of the first predicted region in the first mask image may all be 1, and the pixel values ​​of non-first predicted regions may all be 0. Accordingly, by multiplying the first mask image by each pixel value of the medical image, the coronary artery region of the medical image can be extracted to obtain the predicted coronary artery region. Please refer to [link to relevant documentation]. Figure 4a , Figure 4a This can serve as a mask image representing coronary artery regions and non-coronary artery regions. The region formed by black pixels can represent the non-coronary artery region, and the region formed by white pixels can represent the coronary artery region. Specifically, the region formed by the white pixels can be the first predicted region. Of course, in some embodiments, each pixel of the first mask image can represent the probability that its corresponding pixel in the medical image belongs to the first predicted region.

[0059] Step S103: Determine a second mask image including a second prediction region; wherein the second mask image includes a second prediction region for representing the location of the trunk region in the medical image.

[0060] In this embodiment, the second mask image can be a mask image representing a non-trunk region of the trunk region of the medical image. The second mask image may include a second predicted region representing the trunk region. This second predicted region may represent the region where the main trunk of the coronary artery is located in the medical image.

[0061] In this embodiment, the method for determining the second mask image including the second prediction region can be obtained through conventional image processing methods. For example, it can be obtained through region growing. Alternatively, the method for determining the second mask image including the second prediction region can also be determined using deep learning techniques. Specifically, the second mask image can be determined using a second mask image generation model. This second mask image generation model can be a deep learning model. Specifically, for example, the second mask image generation model can be a convolutional neural network.

[0062] Step S104: If the target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, the target mask image is used as a label for the medical image to generate training samples for the first mask image generation model, which are then used to train the first mask image generation model.

[0063] In some cases, the number of training samples for the model generating the first mask image has a significant impact on the accuracy of the first mask image generated by the model. In the medical field, the number of training samples is relatively small, requiring considerable manpower to label the regions representing the target object in medical images to generate corresponding training samples. Therefore, the target mask image obtained by superimposing the first mask image and the second mask image can be used as a label for the medical image to generate training samples. This allows for the adjustment of the machine learning model that generates the first mask image, continuously improving the accuracy of the mask image generated by the machine learning model.

[0064] In some cases, the first mask image can be used to identify both coronary artery regions and non-coronary artery regions in the medical image. Therefore, the accuracy of the prediction of important coronary artery regions by the first mask image generation model may vary. For example, the first mask image has relatively weak accuracy in identifying the main trunk region of the coronary artery. Therefore, the main trunk region of the coronary artery can be identified separately to obtain a second mask image. Furthermore, the first and second mask images can be superimposed. That is, the second mask image can enhance the representation of the main trunk region of the coronary artery in the first mask image, thereby improving the accuracy of coronary artery region identification to some extent. Furthermore, if the target mask image obtained by superimposing the first and second mask images has a good representation of the coronary artery regions in the medical image, the target mask image can be used as a label for the medical image and as a training sample for the first mask image generation model to further improve the accuracy of the first mask image generated by the first mask image generation model in predicting the coronary artery regions.

[0065] The method for obtaining a target mask image by superimposing the first mask image and the second mask image can be to fuse corresponding pixels in the first mask image and the second mask image. Specifically, pixels in the first prediction region included in the first mask image and pixels in the second prediction region included in the second mask image can both be used as the target prediction region of the target mask image. For example, the first mask image and the second mask image can have the same size, and the pixel value of pixels included in the first prediction region and the second prediction region is 1, while the pixel value of pixels not included in the first prediction region and the second prediction region is 0. The target mask image can be obtained by performing an OR operation on the pixel values ​​at corresponding positions. Please refer to [link to relevant documentation]. Figure 5a , Figure 5b and Figure 5c .in, Figure 5a The first mask image represents the coronary artery region and the non-coronary artery region. Figure 5b The second mask image represents the coronary artery region and the non-coronary artery region. Figure 5c To be Figure 5a Represents the first mask image and Figure 5b This represents the target mask image obtained by overlaying the second mask image.

[0066] When the target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, it can be a confirmation signal received from the user. Specifically, for example, if the target mask image is deemed to have high accuracy after evaluation by a professional, the confirmation signal can be input. After receiving the confirmation signal, it can be considered that the target mask image obtained by superimposing the first mask image and the second mask image can represent the coronary artery region of the medical image well. Therefore, the mask image generation system can use the mask image as a label for the medical image to form new training samples for further improving the performance of the machine learning model.

[0067] In some implementations, the first prediction region includes a plurality of non-adjacent first connected regions; the second prediction region includes a plurality of non-adjacent second connected regions; the method for generating training samples may further include: removing first connected regions in the first mask image with a pixel count lower than a specified threshold, and second connected regions in the second mask image with a pixel count lower than a specified threshold, respectively.

[0068] In some cases, the first mask image may include multiple non-adjacent first connected regions, and the second mask image may include multiple non-adjacent second connected regions. The first and second connected regions may be partially noisy regions. For example, multiple second connected regions can be used to represent the main coronary artery. Since the multiple second connected regions are non-adjacent, this may result in the identified main coronary artery being broken. Therefore, the first connected regions in the first mask image with a pixel count below a specified threshold, and the second connected regions in the second mask image with a pixel count below a specified threshold, can be removed to ensure the accuracy of the first and second mask images. Furthermore, the accuracy of obtaining the target mask image from the first and second mask images can be improved to some extent.

[0069] In this embodiment, the first mask image can be a mask image representing a non-coronary artery region of the coronary artery region in the medical image. The first mask image may include a first predicted region representing the coronary artery region. This first predicted region may represent the area in the medical image where the coronary artery is located. Specifically, for example, the size of the first mask image may be the same as the medical image. Each pixel of the first mask image may correspond to a pixel in the medical image. Pixels in the first predicted region can be used to indicate that the corresponding pixel in the medical image belongs to the predicted coronary artery region. In some embodiments, the pixel values ​​of the first predicted region in the first mask image may all be 1, and the pixel values ​​of non-first predicted regions may all be 0. Accordingly, by multiplying the first mask image by each pixel value of the medical image, the coronary artery region of the medical image can be extracted to obtain the predicted coronary artery region. Please refer to [link to relevant documentation]. Figure 4a , Figure 4a This can serve as a mask image representing coronary artery and non-coronary artery regions. The region composed of black pixels represents the non-coronary artery region, and the region composed of white pixels represents the coronary artery region. Specifically, the region composed of white pixels can be the first predicted region.

[0070] In this embodiment, the method for removing a first connected region in the first mask image with a pixel count lower than a specified threshold, and a second connected region in the second mask image with a pixel count lower than a specified threshold, can be to calculate the number of pixels included in the first connected region of the first mask image and the number of pixels included in the second connected region of the second mask image, respectively. When the pixel count is lower than the specified threshold, the value of the pixels within the corresponding connected region can be set to the value of pixels not in the first prediction region or not in the second prediction region. Specifically, for example, when the pixel value of the first and second prediction regions is 1, and the pixel value of the non-first or non-second prediction regions is 0, the pixel value of the connected region with a pixel count lower than the specified threshold can be set to 0. See also... Figure 4a Multiple smaller connected regions may represent noise, so they can be removed, retaining only the largest connected region. For details, please refer to [link to relevant documentation]. Figure 4b . Figure 4b This can represent the result obtained after removing connected regions whose pixel count is below a specified threshold from multiple connected regions included in the first mask image and the second mask image.

[0071] In some embodiments, the method for generating training samples may further include: if the target mask image is identified as an invalid mask image, obtaining the label of the medical image to generate training samples for the first mask image generation model.

[0072] In some cases, the target mask image obtained by superimposing the first and second mask images may not be able to accurately represent the coronary artery region in the medical image. Therefore, when the target mask image cannot accurately represent the coronary artery region, manually annotated labels can be obtained from the medical image to generate training samples, thereby improving the accuracy of the first mask image generation model in predicting the coronary artery region in the medical image.

[0073] The target mask image is deemed invalid if an invalid signal is received from the user. Specifically, for example, if a target mask image is deemed to have low accuracy after evaluation by professionals, it can be provided to the server with annotation information representing the corresponding medical image. This annotation information can be a mask image representing the target region in the medical image, serving as a label for the medical image and as a training sample for the first mask image generation model.

[0074] Please see Figure 3a and Figure 3b This specification provides a method for generating a mask image, one embodiment of which can be applied to a server. The method for generating the mask image may include the following steps.

[0075] Step S201: Receive medical images; wherein the medical images include a coronary artery region representing the coronary arteries and a trunk region representing the main trunk of the coronary arteries.

[0076] In some cases, after receiving the medical images, predictions can be made about them.

[0077] Step S202: Determine a first mask image including a first prediction region and a second mask image including a second prediction region; wherein, the first prediction region is used to represent the position of the coronary artery region in the medical image; the second prediction region is used to represent the position of the trunk region in the medical image; wherein, the first prediction region and the second prediction region each include a plurality of non-adjacent connected regions.

[0078] In some cases, the accuracy of the mask images obtained from medical images using related technologies may not be high. Therefore, a first mask image representing the coronary artery region and non-coronary artery regions can be generated separately, as well as a second mask image representing the main trunk region and non-main trunk regions. Furthermore, connected regions with a pixel count below a specified threshold included in the first and second mask images can be removed. Further, the first and second mask images can be fused to obtain a target mask image, thereby improving the accuracy of the generated mask image to some extent.

[0079] In this embodiment, the method for determining the first mask image including the first prediction region and the second mask image including the second prediction region can be obtained through conventional image processing methods. For example, region growing can be used. Alternatively, the method for determining the first mask image including the first prediction region and the second mask image including the second prediction region can also be achieved using deep learning techniques. Specifically, the first mask image can be determined using a first mask image generation model, and the second mask image can be determined using a second mask image generation model. Both the first and second mask image generation models can be deep learning models. Specifically, for example, both the first and second mask image generation models can be convolutional neural networks. In some embodiments, the first and second mask image generation models can be constructed using different models. Specifically, for example, the first mask image generation model can be a U-net model, and the second mask image generation model can be an FCN model.

[0080] Step S203: Remove connected regions in the first mask image and the second mask image from those regions where the number of pixels is lower than a specified threshold.

[0081] In some cases, the first and second mask images may include multiple connected regions. Some of these connected regions may contain a small number of pixels. That is, these partially connected regions represent multiple non-adjacent and small areas in the medical image. The areas represented by these connected regions in the medical image may contain noise. Therefore, by removing connected regions in the first and second mask images whose pixel count is below a specified threshold, noise information in the first and second mask images can be removed, while retaining connected regions containing more information.

[0082] Specifically, the method for removing connected regions in the first and second mask images where the number of pixels is below a specified threshold can be to calculate the number of pixels in each connected region of the first and second mask images. If the number of pixels is below the specified threshold, the corresponding connected region in the first mask image is marked as a non-first prediction region, or the corresponding connected region in the second mask image is marked as a non-second prediction region.

[0083] Step S204: Overlay the first mask image and the second mask image to form a target mask image; wherein the target mask image includes a target prediction region representing the coronary artery region. In some cases, the first mask image can be used to represent both coronary artery and non-coronary artery regions of a medical image, therefore the accuracy for identifying local coronary arteries may not be high. The second mask image can represent the main trunk region of the medical image. The second mask image can enhance the identification capability of local coronary arteries in the first mask image, especially the identification of main coronary arteries, thereby improving the accuracy of the generated target mask image to a certain extent.

[0084] The target mask image is a mask image obtained by superimposing the first mask image and the second mask image. Specifically, the method of superimposing the first mask image and the second mask image to form the target mask image can be to perform a fusion process on corresponding pixels in the first mask image and the second mask image. Specifically, pixels in the first prediction region included in the first mask image and pixels in the second prediction region included in the second mask image can both be used as the target prediction region of the target mask image.

[0085] In some implementations, the pixel values ​​of the first and second mask images can represent the probability that a corresponding pixel belongs to a corresponding target region. Accordingly, the method of superimposing the first and second mask images to form a target mask image can involve taking a weighted average of the pixel values ​​of the second predicted region in the second mask image and the corresponding pixel values ​​in the first mask image. If the pixel value is greater than 0.5, the pixel can be considered to belong to the main trunk region. Furthermore, for pixels in the first mask image that belong to the coronary artery region but not the main trunk region, the result of the first mask image can be used as the standard. Finally, the superimposed mask image can be binarized.

[0086] In some embodiments, the method for generating the mask image may further include: adjusting the pixel value of a pixel based on the spatial distance and grayscale distance between a pixel of the medical image and its neighboring pixels to reduce noise in the medical image; wherein the spatial distance represents the distance between the position of the pixel and its neighboring pixels in the medical image; and the grayscale distance represents the difference between the pixel value of the pixel and the pixel value of its neighboring pixels.

[0087] In some cases, the quality of the medical images can affect the accuracy of the target mask image during the generation process. Therefore, preprocessing of the medical images can improve their quality.

[0088] The medical images can be generated using digital subtraction angiography. The coronary artery region, representing the coronary arteries, has a relatively clear outline compared to the non-coronary artery region imaged from non-coronary artery areas. Therefore, to avoid reducing the resolution of the coronary artery outline by using neighborhood pixel values ​​for noise reduction in the medical images, a bilateral filter can be used for noise reduction. This preserves high-frequency edge information while reducing image noise, which is beneficial for protecting the vascular structures in the image.

[0089] The spatial distance can represent the distance between the position of the pixel and its neighboring pixels in the medical image. Specifically, the spatial distance can represent the spatial domain information of the pixel. The grayscale distance represents the difference between the pixel value of the pixel and the pixel value of its neighboring pixels. Specifically, the grayscale distance can represent value range information.

[0090] A method for reducing noise in a medical image by adjusting the pixel value based on the spatial and grayscale distances between pixels and their neighboring pixels can classify the neighboring pixels of the pixel to be filtered according to the pixel value, with different weights corresponding to different categories. Then, a weighted summation of the neighboring pixels can be performed to obtain the adjusted pixel value. The larger the pixel difference between the neighboring pixels of the pixel to be filtered, the smaller the corresponding weight can be. In some embodiments, a bilateral filter can be directly used for noise reduction.

[0091] In some implementations, the step of determining a first mask image including a first prediction region may include: downsampling the medical image to obtain a multidimensional feature map of the medical image; wherein the multidimensional feature map includes multiple feature sub-maps representing different characteristics of the medical image; each feature sub-map includes a plurality of feature pixels; performing global pooling on each feature sub-map to obtain pooling values ​​corresponding to each feature sub-map, which are used as first weights corresponding to each feature sub-map; performing global pooling on feature pixels corresponding to the same position in the feature sub-maps to obtain pooling values ​​corresponding to each feature pixel in the feature sub-map, which are used as second weights for the feature pixels in the feature sub-maps; adjusting the values ​​of the feature pixels in the multidimensional feature map based on the first weights and the second weights to obtain a target multidimensional feature map; and using the target multidimensional feature map to determine the first mask image of the medical image.

[0092] In some cases, downsampling can be used to generate multidimensional feature maps of the medical images. Specifically, the downsampling process may include multiple sets of convolutional kernel pooling. Convolutional processing can capture local features of neighboring pixels, which may cause global features to be ignored to some extent. Therefore, during the generation of multidimensional feature maps, an attention mechanism can be added to ensure that the generated feature maps also take into account global information.

[0093] The multidimensional feature map can be a three-dimensional tensor. The tensor may include a dimension representing the length of the feature map, a dimension representing the width of the feature map, and a channel dimension representing the number of feature maps. Each channel can be used to define a feature sub-map. Each feature sub-map can be used to represent information about different characteristics of the medical image.

[0094] The method for obtaining the multidimensional feature map of the medical image can be achieved by downsampling the medical image. Specifically, this can be done by convolving the medical image or its multidimensional feature map using multiple convolution kernels. In some embodiments, after convolution, pooling or normalization operations can be performed to obtain the multidimensional feature map. Alternatively, in some embodiments, the multidimensional feature map of the medical image can be obtained by temporarily storing the multidimensional feature map obtained at each downsampling level. Correspondingly, the method for obtaining the multidimensional feature map of the medical image can be to obtain the temporarily stored multidimensional feature map.

[0095] The first weight is generated by performing global pooling on the feature sub-images respectively, yielding pooling values ​​corresponding to each feature sub-image. This can be achieved by first performing global max pooling on the feature sub-images of different channels to obtain the first weight for each channel. Alternatively, global average pooling can be performed on the feature sub-images of different channels to obtain the first weight for each channel. In some embodiments, global max pooling and global average pooling can be performed on the feature sub-images of different channels separately, and the weight sequences obtained from the global max pooling and global average pooling processes can be fused to generate the first weight.

[0096] The feature pixels can represent pixels in the feature sub-image. A method to obtain pooled values ​​for feature pixels at the same position in the feature sub-image through global pooling, and using these pooled values ​​as the second weights for the feature pixels in the feature sub-image, can be achieved by performing global max pooling on the feature pixels at the same position in the feature sub-image to obtain the second weight for each feature pixel. Alternatively, global average pooling can be performed on the feature pixels at the same position in the feature sub-image to obtain the second weight for each pixel. In some embodiments, global max pooling and global average pooling can be performed separately on the feature pixels at the same position in the feature sub-image, and the weight sequences obtained from the global max pooling and global average pooling can be combined to generate the second weight.

[0097] The method for adjusting the values ​​of feature pixels in the multidimensional feature map based on the first weight and the second weight to obtain the target multidimensional feature map can be as follows: First, adjust the multidimensional feature map using the first weight, and then adjust the multidimensional feature map adjusted by the first weight using the second weight. Alternatively, the multidimensional feature map can be adjusted using the first weight and the second weight respectively, and then the two adjusted multidimensional feature maps can be merged as the final adjustment result. Specifically, adjusting the multidimensional feature map using the first weight can be achieved by weighting the first weight with the feature pixels in its corresponding feature sub-map. Adjusting the multidimensional feature map using the second weight can be achieved by weighting the second weight with the feature pixels at its corresponding positions.

[0098] The method for determining the first mask image of the medical image using the target multidimensional feature map may involve processing the target multidimensional feature map through convolution, upsampling, or other methods to obtain the target multidimensional feature map. Specifically, the target multidimensional feature map may be a multidimensional feature map in the U-net network model. Based on the multidimensional feature map, the first mask image can be generated according to model calculations.

[0099] Please see Figure 6 This specification provides a method for training a mask image generation model, which may include the following steps.

[0100] Step S301: Construct a training sample set for the initial mask image generation model; wherein, the training sample set includes training samples composed of medical images and label mask images of the medical images; wherein, the medical images include coronary artery regions representing coronary arteries; the label mask images are used to represent the coronary artery regions and non-coronary artery regions of the medical images.

[0101] In some cases, the initial mask image generation model can be a machine learning model. Training samples for the initial mask image generation model need to be built to adjust the parameters of the initial mask image generation model in order to improve the accuracy of the mask images generated by the initial mask image generation model.

[0102] The training sample set may include multiple training samples. These training samples may include medical images and label mask images that serve as labels for the medical images. The medical images may include coronary artery regions representing coronary arteries. The label mask images may represent both coronary artery regions and non-coronary artery regions of the medical images. Specifically, the label mask image may be a binary image. Regions with a pixel value of 1 may represent the coronary artery regions, and regions with a pixel value of 0 may represent non-coronary artery regions.

[0103] A method for constructing a training sample set for the initial mask image generation model can be to obtain the medical image and its corresponding label mask image, and bind them together to form a training sample. The labels corresponding to the medical images can be determined manually.

[0104] Step S302: Using the medical image as input to the initial mask image generation model, and the label mask image as the target output of the initial mask image generation model; using the initial mask image generation model to calculate the prediction loss of the training sample set; wherein, the prediction loss represents the difference between the mask image generated by the initial mask image generation model based on the input medical image and the corresponding label mask image.

[0105] In some cases, the mask images of medical images generated by the model from the corresponding training samples using the initial mask image may differ from the corresponding label mask images. Therefore, by calculating the difference between the two, the model parameters can be further adjusted to reduce the difference.

[0106] The prediction loss characterizes the difference between the predicted mask image corresponding to the medical image obtained by inputting the medical image of the training samples into the initial mask image generation model and the corresponding label mask image. The size of the predicted mask image can be the same as the medical image. The pixel values ​​of the predicted mask image can represent the category of the pixels in the corresponding location of the medical image. Specifically, the prediction loss can be determined by constructing a loss function for the initial mask image generation model. The output value of the loss function can be used as the prediction loss. For example, the loss function can be constructed using cross-entropy, or it can be constructed based on Euclidean distance. In some implementations, the prediction loss corresponding to the training sample set and the initial mask image generation model can be obtained by calculating the prediction loss of each training sample in the training sample set and obtaining the average value of the prediction loss.

[0107] Step S303: Update the parameters of the initial mask image generation model to adjust the prediction loss so that the prediction loss, under specified conditions, will use the initial mask image generation model with the corresponding parameters as the trained mask image generation model.

[0108] In some cases, the parameters of the initial mask image generation model can be adjusted based on the loss function to reduce the difference between the predicted value and the label.

[0109] The method for updating the parameters of the initial mask image generation model can be based on gradient descent, adjusting the parameters of the initial mask image generation model by calculating the gradient of the loss function. Alternatively, the method for updating the parameters of the initial mask image generation model can employ optimization methods such as real-time gradient descent or the Adam optimizer.

[0110] Under certain conditions, the initial mask image generation model with corresponding parameters can be used as the trained mask image generation model. Specifically, the specified condition may be that the prediction loss output by the loss function tends to be constant. Of course, in some embodiments, the specified condition may also be that the prediction loss of the training sample set decreases, and the initial mask image generation model at the point where the prediction loss of the test sample set gradually increases is determined as the trained mask image generation model.

[0111] In some implementations, the step of calculating the prediction loss of the training sample set using the initial mask image generation model includes: generating a prediction mask image of the medical image using the initial mask generation model; wherein the prediction mask image includes a plurality of prediction pixels corresponding to pixels of the medical image; determining the loss weight corresponding to the prediction pixel based on the value of the prediction pixel and the value of the corresponding pixel in the label mask image; wherein the value of the prediction pixel is used to represent the probability that the corresponding pixel of the medical image belongs to the coronary artery region; and calculating the prediction loss of the training sample set according to the prediction mask image and the loss weight corresponding to the prediction pixels of the prediction mask image.

[0112] In some cases, the model for generating mask images can be represented by classifying each pixel in the medical image. Specifically, pixels in the medical image can be divided into categories belonging to the coronary artery region and categories not belonging to the coronary artery region. Because coronary arteries are relatively long and thin, the number of pixels representing the coronary artery region in the corresponding medical image is much smaller than the number of pixels included in the non-coronary artery region. Consequently, the number of pixels representing the coronary artery region and the number of pixels representing the non-coronary artery region are unbalanced. Therefore, by setting loss weights, higher weights can be given to pixels that are difficult to classify during training, while lower weights can be given to pixels that are easily distinguishable during training, thereby quickly focusing the training effort on those difficult-to-distinguish pixels.

[0113] The predicted mask image can be a mask image generated by an initial mask generation model based on the input medical image. The predicted mask image may include a predicted region representing the target region. The predicted pixels in the predicted mask image may correspond to pixels in the medical image. The value of the predicted pixel may represent the probability that the corresponding pixel in the medical image is input into the target region. The value of the predicted pixel may be a normalized value. The target region may be the coronary artery region. Specifically, for example, the value of the predicted pixel may be 0.8. Correspondingly, this can indicate that the probability that the pixel in the medical image corresponding to the predicted pixel belongs to the coronary artery region is 80%.

[0114] Correspondingly, the pixel values ​​of the label mask image can represent the region to which the corresponding pixel of the medical image belongs. Specifically, for example, if the pixel value of the label mask image is 1, the corresponding pixel of the medical image can be considered to belong to the coronary artery region. By comparing the difference between the pixel values ​​of the label mask image and the pixel values ​​of the prediction mask image, the accuracy of the classification of each pixel in the medical image can be obtained.

[0115] Specifically, the corresponding pixel values ​​of the label mask image and the predicted mask image can be subtracted. If the difference is greater than a specified threshold, the pixel classification of the medical image is considered correct. Correspondingly, a smaller weight can be assigned to the prediction loss of correctly classified pixels, while a larger weight can be assigned to the prediction loss of misclassified pixels. The method for calculating the prediction loss of the training sample set based on the loss weights corresponding to the predicted pixels in the predicted mask image can be achieved by weighting the prediction losses of individual pixels to obtain the prediction loss of a single training sample. Thus, the prediction loss of the entire training sample set can be obtained.

[0116] In some implementations, the step of calculating the prediction loss of the training sample set using the initial mask image generation model includes: performing multi-level sampling on medical images in the training sample set to obtain a feature map representing image information of the medical images; wherein the feature map includes feature pixels representing information of at least some pixels in the medical images; and adjusting the values ​​of feature pixels corresponding to the same position in the feature maps obtained from different training samples according to a specified probability distribution model, so that the values ​​of the feature pixels satisfy the probability distribution model, thereby obtaining a target feature map;

[0117] In some cases, the initial mask image generation model can be a neural network. A neuron in the neural network can include a non-linear activation function. For example, the activation function could be the sigmoid function. During backpropagation of the neural network, the gradient of the parameters of each neuron can be calculated. In related techniques, the gradient of the activation function is relatively large in certain intervals, which is beneficial for parameter adjustment. However, the small change in the activation function in certain intervals is not conducive to gradient calculation. Therefore, the feature pixel values ​​of the feature map can be adjusted to shift them to the intervals where the gradient of the activation function changes significantly. This allows for faster adjustment of the parameters of the initial mask image generation model during backpropagation, accelerating the training speed of the model.

[0118] The specified probability distribution model can be determined according to model settings. Specifically, the specified probability distribution model can be a normal distribution model. Adjusting the values ​​of the feature pixels to a standard normal distribution allows most pixel values ​​to be adjusted to between -3 and 3, ensuring that the gradient of the activation function is not too small within the range of -3 to 3. In some implementations, corresponding parameters can be set for the probability distribution model. These parameters can be further learned during training to avoid losing certain differences after adjusting the values ​​of the feature pixels through the specified probability distribution model.

[0119] The method for sampling medical images in the training sample set to obtain feature maps representing the image information of the medical images can be through convolution and pooling processes. The feature map can be a multi-dimensional feature map, including multiple channels. In some embodiments, the downsampling method may also use only pooling processing.

[0120] The target feature map can be a feature map adjusted by a probability distribution model. The method for obtaining the target feature map by adjusting the values ​​of feature pixels at the same position in feature maps obtained from different training samples according to a specified probability distribution model, so that the values ​​of the feature pixels satisfy the probability distribution model, can be achieved by statistically analyzing the values ​​of feature pixels at the same position in feature maps obtained from different training samples in a batch of training samples, calculating the parameters of the specified probability distribution model, and thus determining the probability distribution model. Then, the feature pixels are input into the probability distribution model for adjustment. In some embodiments, after processing by the probability distribution model, the pixel values ​​of each feature pixel in the feature map can be scaled and shifted. The scaling coefficient and the shift amount can both be obtained through model learning.

[0121] The method for calculating the prediction loss of the training sample set based on the target feature map can be to further process the target feature map according to the processing flow of the image mask generation model to obtain a prediction mask image and then calculate the prediction loss.

[0122] In some implementations, the step of calculating the prediction loss of the training sample set based on the target feature map includes: upsampling the target feature map to generate a prediction mask image of the medical image; wherein, during the upsampling of the target feature map, the values ​​of some feature pixels of the target feature map are set not to participate in the upsampling operation; and the prediction loss of the training sample set is calculated using the prediction mask image.

[0123] In some cases, neural networks may encounter overfitting during training. Therefore, during the training process, some neuron parameters can be randomly discarded to mitigate the overfitting problem to some extent.

[0124] The method for upsampling the target feature map to generate the prediction mask image of the medical image can be to upsample the target feature map by transposing convolution.

[0125] During the upsampling process of the target feature map, the values ​​of some feature pixels in the target feature map can be excluded from the upsampling operation. Specifically, the output of some feature pixels can be set to zero.

[0126] In some implementations, the step of calculating the prediction loss of the training sample set using the initial mask image generation model includes: performing multi-level downsampling on the medical image to obtain multiple first feature maps of different sizes; wherein the first feature map obtained by the last level of downsampling is used as an intermediate feature map; performing multi-level upsampling on the intermediate feature map to obtain multiple second feature maps corresponding to the first feature maps; wherein the second feature maps are obtained by convolution processing on the concatenated feature maps; wherein, during the convolution processing, some feature pixels are controlled not to participate in the convolution operation; wherein the concatenated feature maps are obtained by concatenating the feature maps obtained by upsampling the previous level of second feature maps with the corresponding first feature maps; and determining the prediction loss of the training sample set based on the second feature maps.

[0127] In some cases, deep learning models are prone to overfitting. Therefore, during the training process of a neural network, some neuron parameters can be randomly discarded to mitigate the overfitting problem to some extent. Specifically, the neuron parameters can be manipulated during the channel concatenation process to achieve the desired effect.

[0128] In some implementations, an image mask generation model can be constructed using a deep learning model with an encoder and decoder structure. The encoder portion of the deep learning model can generate multiple first feature maps. Correspondingly, the decoder portion can also generate multiple second feature maps. The first feature maps generated by the encoder can be concatenated with the feature map obtained by upsampling the second feature map generated by the decoder from the previous layer. The concatenated feature map is then convolved to obtain the second feature map for the corresponding layer. Notably, after channel concatenation, some parameters of the convolution kernels can be randomly set to zero to mitigate overfitting to some extent.

[0129] The first feature map can be obtained by downsampling the medical image at multiple levels. Specifically, for example, the medical image can be downsampled five times to obtain five first feature maps. The feature map obtained by the last downsampling can be used as an intermediate feature map.

[0130] The second feature map can be a feature map obtained by upsampling an intermediate feature map. The second feature map can correspond to a first feature map. The first feature map corresponding to the second feature map can be a feature map at the same level as the first feature map. Specifically, performing five downsampling operations on a medical image yields five first feature maps, corresponding to levels one through five. The first feature map at level five can serve as an intermediate feature map. Performing one upsampling operation on the intermediate feature map yields a feature map at level four. Correspondingly, performing two upsampling operations on the intermediate feature map yields a feature map at level three.

[0131] A method for obtaining multiple second feature maps corresponding to the first feature map by performing multi-level upsampling on the intermediate feature image can be to concatenate the feature map obtained by upsampling the second feature map of the previous level with the first feature map of the corresponding level to generate a concatenated feature map. The number of channels in the concatenated feature map can be the sum of the number of channels in the first feature map and the number of channels in the feature map obtained by upsampling the second feature map of the previous level. Next, convolution processing can be performed on the concatenated feature map to reduce the number of channels by a specified number, thus obtaining the second feature map of the corresponding level. During the convolution process, some feature pixels can be excluded from the convolution operation. Specifically, the values ​​of some feature pixels can be randomly set to 0. Alternatively, the parameters of some convolution kernels can be set to 0 at any time.

[0132] The method for determining the prediction loss of the training sample set based on the second feature map can be to upsample the second feature map multiple times to generate a prediction mask image. The prediction loss for the corresponding training sample set can be generated by comparing the prediction mask image with the label mask image.

[0133] Example devices, electronic devices, storage media, and software

[0134] Please see Figure 7 One embodiment of this specification provides an apparatus for generating training samples. The apparatus may include: a receiving module, a mask generation module, a determining module, and a sample generation module.

[0135] A receiving module is used to receive medical images; wherein the medical images include a coronary artery region representing the coronary arteries, and a trunk region representing the main trunk of the coronary arteries;

[0136] A mask generation module is used to input the medical image into a first mask image generation model to obtain a first mask image; wherein, the first mask image includes a first prediction region for representing the location of the coronary artery region in the medical image;

[0137] A determining module is configured to determine a second mask image including a second prediction region; wherein the second mask image includes a second prediction region for representing the location of the backbone region in the medical image;

[0138] The sample generation module is used to generate training samples for the first mask image generation model by using the target mask image as a label for the medical image when the target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image.

[0139] Please see Figure 8 One embodiment of this specification also provides an apparatus for generating a mask image. The mask image generation apparatus may include a receiving module, a determining module, a filtering module, and an overlay module.

[0140] A receiving module is used to receive medical images; wherein the medical images include a coronary artery region representing the coronary arteries and a trunk region representing the main trunk of the coronary arteries.

[0141] A determining module is used to determine a first mask image including a first prediction region and a second mask image including a second prediction region; wherein the first prediction region is used to represent the position of the coronary artery region in the medical image; the second prediction region is used to represent the position of the trunk region in the medical image; wherein the first prediction region and the second prediction region each include a plurality of non-adjacent connected regions.

[0142] The filtering module is used to remove connected regions in the first mask image and the second mask image whose number of pixels is lower than a specified threshold.

[0143] The overlay module is used to overlay the first mask image and the second mask image to form a target mask image; wherein the target mask image includes a target prediction region representing the coronary artery region.

[0144] Please see Figure 9 One embodiment of this specification also provides a training apparatus for a mask image generation model. The mask image generation apparatus may include a construction module, a prediction module, and an update module.

[0145] A construction module is used to construct a training sample set for an initial mask image generation model; wherein the training sample set includes training samples composed of medical images and target label mask images of the medical images; wherein the medical images include coronary artery regions representing coronary arteries; and the target label mask images are used to represent the coronary artery regions and non-coronary artery regions of the medical images.

[0146] The prediction module is used to take the medical image as input to the initial mask image generation model and the label mask image as the target output of the initial mask image generation model; and to calculate the prediction loss of the training sample set using the initial mask image generation model; wherein the prediction loss represents the difference between the mask image generated by the initial mask image generation model based on the input medical image and the corresponding label mask image.

[0147] An update module is used to update the parameters of the initial mask image generation model to adjust the prediction loss so that, under specified conditions, the prediction loss uses the initial mask image generation model with the corresponding parameters as the trained mask image generation model.

[0148] Please see Figure 10 This specification provides a computer device, which may include: a processor; a memory for storing processor-executable instructions; and the processor for executing the methods described above.

[0149] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the mask image generation method in any of the above embodiments.

[0150] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the mask image generation method in any of the above embodiments.

[0151] This specification also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, causes the computer to perform the training method for the mask image generation model in any of the above embodiments.

[0152] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the training method for the mask image generation model in any of the above embodiments.

[0153] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0154] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0155] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0156] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0157] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0158] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0159] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0160] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0161] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0163] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0164] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this specification, in essence, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] The above description is merely a specific embodiment of this specification, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for generating a training sample, characterized in that, The method comprises: receiving a medical image; wherein the medical image comprises a coronary region representing a coronary artery, and a stem region representing a stem of the coronary artery; inputting the medical image into a first mask image generation model to obtain a first mask image; wherein the first mask image comprises a first predicted region for representing a position of the coronary region in the medical image; inputting the medical image into a second mask image generation model to obtain a second mask image; wherein the second mask image comprises a second predicted region for representing a position of the stem region in the medical image; in a case where a target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, taking the target mask image as a label of the medical image; in a case where the target mask image is confirmed as an invalid mask image, obtaining a label of the medical image; and based on the label of the medical image, generating a training sample of the first mask image generation model for training the first mask image generation model.

2. The method of claim 1, wherein, The first predicted region comprises a plurality of first connected regions that are not adjacent to each other; the second predicted region comprises a plurality of second connected regions that are not adjacent to each other; and the method further comprises: respectively removing first connected regions in the first mask image and second connected regions in the second mask image, which have a number of pixels lower than a specified threshold.

3. A method of generating a mask image, characterized by, The method for generating the target mask image in claim 1 comprises: receiving a medical image; wherein the medical image comprises a coronary region representing a coronary artery, and a stem region representing a stem of the coronary artery; determining a first mask image comprising a first predicted region, and a second mask image comprising a second predicted region; wherein the first predicted region is for representing a position of the coronary region in the medical image; and the second predicted region is for representing a position of the stem region in the medical image; wherein the first predicted region and the second predicted region each comprise a plurality of connected regions that are not adjacent to each other; respectively removing connected regions in the first mask image and the second mask image, which have a number of pixels lower than a specified threshold; superimposing the first mask image and the second mask image to form a target mask image; wherein the target mask image comprises a target predicted region representing the coronary region.

4. The method of claim 3, wherein, The method further comprises: based on a spatial distance and a gray distance between a pixel and its neighborhood pixels in the medical image, adjusting a pixel value of the pixel to reduce noise of the medical image; wherein the spatial distance represents a distance between the pixel and its neighborhood pixels in the medical image; and the gray distance represents a difference between the pixel value of the pixel and pixel values of its neighborhood pixels.

5. The method of claim 3, wherein, The step of determining the first mask image comprising a first predicted region comprises: down-sampling the medical image to obtain a multi-dimensional feature map of the medical image; wherein the multi-dimensional feature map comprises a plurality of feature sub-maps representing different characteristics of the medical image; and each feature sub-map comprises a plurality of feature pixels; performing global pooling on each feature sub-map to obtain a pooling value corresponding to each feature sub-map as a first weight corresponding to each feature sub-map; performing global pooling on feature pixels corresponding to a same position in each feature sub-map to obtain a pooling value corresponding to each feature pixel in each feature sub-map as a second weight corresponding to each feature pixel in each feature sub-map; adjusting values of the feature pixels in the multi-dimensional feature map based on the first weights and the second weights to obtain a target multi-dimensional feature map; using the target multi-dimensional feature map to determine a first mask image of the medical image. 6.A method for training a mask image generation model, characterized in that, The method for training the first mask image generation model in claim 1 comprises: constructing a training sample set of an initial mask image generation model; wherein the training sample set comprises training samples composed of medical images and label mask images of the medical images; wherein the medical images comprise coronary regions representing coronary arteries; and the label mask images are used to represent coronary regions and non-coronary regions of the medical images; using the medical images as inputs of the initial mask image generation model and using the label mask images as target outputs of the initial mask image generation model; calculating a prediction loss of the training sample set using the initial mask image generation model; wherein the prediction loss represents a difference between mask images generated by the initial mask image generation model based on the input medical images and corresponding label mask images; updating parameters of the initial mask image generation model to adjust the prediction loss, so that the initial mask image generation model with the corresponding parameters is used as a trained mask image generation model when the prediction loss meets a specified condition.

7. The method of claim 6, wherein, The step of calculating the prediction loss of the training sample set using the initial mask image generation model comprises: generating a prediction mask image of the medical image using the initial mask image generation model; wherein the prediction mask image comprises a plurality of prediction pixels corresponding to pixels of the medical image respectively; determining a loss weight corresponding to each prediction pixel according to a value of the prediction pixel and a value of a pixel of a corresponding label mask image; wherein the value of the prediction pixel is used to represent a probability that the corresponding pixel of the medical image belongs to the coronary region; calculating the prediction loss of the training sample set according to the prediction mask image and the loss weight corresponding to the prediction pixels of the prediction mask image.

8. The method of claim 6, wherein, The step of calculating the prediction loss of the training sample set using the initial mask image generation model comprises: performing multi-level sampling on the medical images in the training sample set to obtain a feature map representing image information of the medical images; wherein the feature map comprises feature pixels representing information of at least part of the pixels in the medical images; According to a specified probability distribution model, the values of feature pixels corresponding to the same position in feature maps obtained by different training samples are adjusted so that the values of the feature pixels satisfy the probability distribution model, to obtain a target feature map; Based on the target feature map, a prediction loss of the training sample set is calculated.

9. The method of claim 8, wherein, Based on the target feature map, a prediction loss of the training sample set is calculated. The target feature map is up-sampled to generate a predicted mask image of the medical image; wherein in the process of up-sampling the target feature map, the values of part of the feature pixels of the target feature map are set not to participate in the up-sampling operation; The prediction loss of the training sample set is calculated through the predicted mask image.

10. A training sample generation apparatus, characterized in that, Comprise: The receiving module is configured to receive a medical image; wherein the medical image comprises a coronary region representing a coronary artery, and a main stem region representing a main stem of the coronary artery; The mask generation module is configured to input the medical image into a first mask image generation model to obtain a first mask image; wherein the first mask image comprises a first predicted region for representing the position of the coronary region in the medical image; The determination module is configured to input the medical image into a second mask image generation model to obtain a second mask image; wherein the second mask image comprises a second predicted region for representing the position of the main stem region in the medical image; The sample generation module is configured to, in a case where a target mask image obtained by superimposing the first mask image and the second mask image is confirmed as a valid mask image, take the target mask image as a label of the medical image; in a case where the target mask image is confirmed as an invalid mask image, obtain a label of the medical image; and generate a training sample of the first mask image generation model based on the label of the medical image.

11. An apparatus for generating a mask image, characterized by For generating the target mask image in claim 1, comprising: The receiving module is configured to receive a medical image; wherein the medical image comprises a coronary region representing a coronary artery, and a main stem region representing a main stem of the coronary artery; The determination module is configured to determine a first mask image comprising a first predicted region, and a second mask image comprising a second predicted region; wherein the first predicted region is used to represent the position of the coronary region in the medical image; and the second predicted region is used to represent the position of the main stem region in the medical image; wherein the first predicted region and the second predicted region respectively comprise a plurality of non-adjacent connected regions; The screening module is configured to remove connected regions with a pixel number below a specified threshold from the plurality of connected regions included in the first mask image and the second mask image, respectively; The superimposition module is configured to superimpose the first mask image and the second mask image to form a target mask image; wherein the target mask image comprises a target predicted region representing the coronary region.

12. A training apparatus of a mask image generation model, characterized by, For training the first mask image generation model in claim 1, comprising: A construction module is configured to construct a training sample set of an initial mask image generation model, wherein the training sample set comprises training samples composed of medical images and label mask images of the medical images, wherein the medical images comprise coronary regions representing coronary arteries, and the label mask images are used to represent coronary regions and non-coronary regions of the medical images. A prediction module is configured to use the medical images as inputs of the initial mask image generation model, use the label mask images as target outputs of the initial mask image generation model, and calculate a prediction loss of the training sample set by using the initial mask image generation model, wherein the prediction loss represents a difference between mask images generated by the initial mask image generation model according to the input medical images and corresponding label mask images. An updating module is configured to update parameters of the initial mask image generation model, adjust the prediction loss, and use the initial mask image generation model with the corresponding parameters as a trained mask image generation model when the prediction loss meets a specified condition.

13. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method in any one of claims 1 to 9 when executing the computer program.

14. A computer readable storage medium, which stores a computer program, and the computer program implements the method in any one of claims 1 to 9 when executed by a processor.

Images