Abnormal region segmentation method of medical image, and model training method and device thereof

Abstract

The invention relates to an abnormal region segmentation method of a medical image and a model training method and device thereof, and the method comprises the steps: inputting a to-be-segmented medical image into a target adversarial discrimination model, and determining the image type of the to-be-segmented medical image; when the image type of the to-be-segmented medical image is an abnormal medical image, obtaining a relation function between the to-be-segmented medical image input into the target adversarial discrimination model and a discrimination result output by the target adversarial discrimination model; the relation function is used for representing the contribution degree of the input of the target adversarial discrimination model to the output; and segmenting the abnormal region of the medical image according to the relation function. According to the invention, the problem of inaccurate segmentation of the abnormal region of the medical image in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a method for segmenting abnormal regions of medical images, a model training method and a device thereof. Background Art

[0002] Medical imaging technologies, such as ultrasound imaging technology, are currently widely used in clinical human tissue imaging and pathological diagnosis because of their non-invasive, economical and convenient, high real-time and radiation-free characteristics.

[0003] Since the acoustic properties of diseased tissues are different from those of normal tissues, they show certain morphological differences in ultrasound images. Identifying abnormal areas in ultrasound images is the key to ultrasound diagnosis and requires the help of experienced ultrasound doctors. However, the actual ultrasound image modality is complex and the daily ultrasound scan volume is gradually increasing. These problems have led to an increase in the workload of ultrasound doctors and higher professional requirements for doctors. Therefore, more automated ultrasound image abnormal area segmentation technology is needed to improve the efficiency and accuracy of ultrasound scan diagnosis.

[0004] With the development of artificial intelligence, more and more medical image recognition and segmentation methods based on artificial intelligence have been developed, which greatly improves the efficiency and accuracy of ultrasound detection. However, in the existing solutions for abnormal area segmentation tasks, irregular abnormal areas need to be finely divided and labeled manually before model training. These processes are time-consuming and laborious, and cannot exhaust all possible abnormal patterns, resulting in inaccurate segmentation of abnormal areas in medical images. Summary of the invention

[0005] The embodiments of the present application provide a method for segmenting abnormal regions of medical images, a model training method and a device thereof to solve the problem of inaccurate segmentation of abnormal regions of medical images in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for segmenting abnormal regions of a medical image, the method comprising:

[0007] Inputting the medical image to be segmented into the target adversarial discrimination model to determine the image type of the medical image to be segmented;

[0008] When the image type of the medical image to be segmented is an abnormal medical image, obtaining a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model; the relationship function is used to represent the contribution of the input to the output of the target adversarial discriminant model;

[0009] The abnormal area of ​​the medical image is segmented according to the relationship function.

[0010] In some embodiments, segmenting the abnormal region of the medical image according to the relationship function includes:

[0011] Constructing a model residual function according to the relationship function;

[0012] The abnormal region of the medical image is segmented according to the model residual function.

[0013] In some embodiments, segmenting the abnormal region of the medical image according to the model residual function includes:

[0014] Calculating a significance score of each pixel of the medical image to be segmented according to the model residual function;

[0015] The abnormal region of the medical image is segmented according to the significance score.

[0016] In some embodiments, the relationship function is used to represent the relationship between the pixel value of the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model, and the method further includes:

[0017] The relationship function is generated according to the model parameters of the target adversarial discriminant model.

[0018] In some embodiments, inputting the medical image to be segmented into a target adversarial discriminant model to determine the image type of the medical image to be segmented includes:

[0019] Inputting the medical image to be segmented into the target adversarial discrimination model to obtain a discrimination result of the medical image to be segmented;

[0020] When the discrimination result satisfies a preset condition, determining that the image type of the medical image to be segmented is an abnormal medical image;

[0021] Among them, the preset conditions are determined according to the model parameters of the target adversarial discrimination model.

[0022] In a second aspect, an embodiment of the present application provides a method for training a model, the method comprising:

[0023] Constructing a first training set according to normal medical image data;

[0024] Construct a second training set based on random data;

[0025] An initial adversarial discriminant model is trained according to the first training set and the second training set to obtain a target adversarial discriminant model.

[0026] In some embodiments, the training of the initial adversarial discriminant model according to the first training set and the second training set to obtain the target adversarial discriminant model includes:

[0027] Obtaining a first discrimination result of the first training set;

[0028] Obtaining a second discrimination result of the second training set;

[0029] constructing an objective function according to the first discrimination result and the second discrimination result;

[0030] The initial adversarial discriminant model is trained according to the objective function to obtain a target adversarial discriminant model.

[0031] In a third aspect, an embodiment of the present application provides a device for segmenting abnormal regions of a medical image, the device comprising:

[0032] A discrimination module, used for inputting the medical image to be segmented into a target adversarial discrimination model to determine the image type of the medical image to be segmented;

[0033] A relationship function acquisition module, used for acquiring a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model when the image type of the medical image to be segmented is an abnormal medical image; the relationship function is used to represent the contribution of the input of the target adversarial discriminant model to the output;

[0034] A segmentation module is used to segment the abnormal area of ​​the medical image according to the relationship function.

[0035] In a fourth aspect, an embodiment of the present application provides a model training device, the device comprising:

[0036] A first training set construction module, used to construct a first training set according to normal medical image data;

[0037] A second training set construction module, used to construct a second training set based on random data;

[0038] The model training module is used to train the initial adversarial discriminant model according to the first training set and the second training set to obtain a target adversarial discriminant model.

[0039] In a fifth aspect, a computer-readable storage medium is provided in an embodiment of the present application, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the abnormal area segmentation method of the medical image described in the first aspect are implemented, or when the computer program is executed by a processor, the steps of the model training method described in the second aspect are implemented.

[0040] Compared with the prior art, the abnormal region segmentation method of a medical image, the training method of the model, the device thereof, and the storage medium provided in the embodiment of the present application obtain the discrimination result of the medical image to be segmented by inputting the medical image to be segmented into the target adversarial discrimination model; according to the discrimination result, the image type of the medical image to be segmented is determined; when the image type of the medical image to be segmented is an abnormal medical image, the relationship function between the medical image to be segmented input into the target adversarial discrimination model and the discrimination result output by the target adversarial discrimination model is obtained; the relationship function is used to represent the contribution of the input of the target adversarial discrimination model to the output; according to the relationship function, the abnormal region of the medical image is segmented using the contribution of the input of the target adversarial discrimination model to the output, thereby achieving accurate segmentation of the abnormal region of the medical image. This solves the problem in the prior art that the irregular abnormal region needs to be finely divided and labeled manually before model training. These processes are time-consuming and labor-intensive, and cannot exhaust all possible abnormal modes, resulting in inaccurate segmentation of the abnormal region of the medical image.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 It is a hardware structure block diagram of a terminal for executing a method for segmenting abnormal regions of medical images according to an embodiment of the present application;

[0044] Figure 2 is a flow chart of a method for segmenting abnormal regions of medical images according to an embodiment of the present application;

[0045] Figure 3 This is a flow chart of segmenting abnormal areas of a medical image in an embodiment of the present application;

[0046] Figure 4 is a flow chart of a model training method according to an embodiment of the present application;

[0047] Figure 5 This is a flowchart of an adversarial discriminant model training in an embodiment of the present application;

[0048] Figure 6 is a flow chart of another abnormal region segmentation method of a medical image according to an embodiment of the present application;

[0049] Figure 7It is a structural block diagram of a device for segmenting abnormal regions of medical images according to an embodiment of the present application;

[0050] Figure 8 It is a structural block diagram of a training device for a model in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0053] The method embodiments provided in the embodiments of the present application can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 1 is a hardware structure block diagram of a terminal for executing a method for segmenting abnormal regions of medical images according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0054] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for segmenting abnormal regions of a medical image in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0055] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0056] This embodiment provides a method for segmenting abnormal regions of medical images. Figure 2 is a flow chart of a method for segmenting abnormal regions of a medical image according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0057] Step S210: input the medical image to be segmented into a target adversarial discriminant model to determine the image type of the medical image to be segmented.

[0058] In this step, a medical image to be segmented of a target body is obtained, and the medical image to be segmented is input into a target adversarial discrimination model to obtain a discrimination result of the medical image to be segmented. The target body here can be the liver, gallbladder, pancreas, spleen, kidney, uterus, ovary, prostate, etc., which is not specifically limited here. The target adversarial discrimination model is a pre-trained adversarial discrimination model, and the adversarial discrimination model here can be an adversarial network model. The input of the target adversarial discrimination model is the medical image to be segmented, and the output is the discrimination result of the medical image to be segmented, which is used to identify the image type of the medical image to be segmented, and the image type here is that the medical image to be segmented is an abnormal medical image, or the medical image to be segmented is a normal medical image. The abnormal medical image here can be understood as a lesion in the medical image to be segmented, that is, the medical image to be segmented includes an abnormal area, and the abnormal area here can be understood as a tumor, canceration, nodule, and other lesion area of ​​the target body, which is not specifically limited here. The normal medical image here can be understood as no lesion in the medical image to be segmented. According to the discrimination result and the preset discrimination condition, the image type of the medical image to be segmented is determined. For example, when the discrimination result meets the preset discrimination condition, the medical image to be segmented is determined to be an abnormal medical image, that is, the image type of the medical image to be segmented is an abnormal medical image. When the discrimination result does not meet the preset discrimination condition, the medical image to be segmented is determined to be a normal medical image, that is, the image type of the medical image to be segmented is a normal medical image. For another example, when the discrimination result meets the preset discrimination condition, the medical image to be segmented is determined to be a normal medical image, that is, the image type of the medical image to be segmented is a normal medical image. When the discrimination result does not meet the preset discrimination condition, the medical image to be segmented is determined to be an abnormal medical image, that is, the image type of the medical image to be segmented is an abnormal medical image. The medical image here can be an ultrasound image, a magnetic resonance image, etc., which is not specifically limited here.

[0059] Step S220, when the image type of the medical image to be segmented is an abnormal medical image, obtain the relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model; the relationship function is used to represent the contribution of the input of the target adversarial discriminant model to the output.

[0060] In this step, when it is determined that the image type of the medical image to be segmented is an abnormal medical image, the relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model is obtained. Suppose the discrimination result output by the target adversarial discriminant model is y, and the medical image to be segmented input into the target adversarial discriminant model is x, and the relationship function between the two is f(x), where y=f(x). The relationship function f(x) is used to represent the contribution of the input x of the target adversarial discriminant model to the output y. The adversarial discriminant model is a nonlinear model. For a specific model input, the adversarial discriminant model can be locally approximated as a linear model by performing a first-order Taylor expansion on the adversarial discriminant model, and the relationship function f(x) is determined based on the first-order Taylor expansion of the adversarial discriminant model.

[0061] Step S230, segmenting the abnormal area of ​​the medical image according to the relationship function.

[0062] In this step, the abnormal area in the medical image is identified according to the relationship function, so that the abnormal area of ​​the medical image is segmented. The relationship function is used to represent the contribution of the input medical image x to be segmented of the target adversarial discriminant model to the output discrimination result y. As a non-limiting example, the contribution of each pixel point in the medical image x to be segmented to the discrimination result y is calculated according to the relationship function, and the image area corresponding to the pixel points in the medical image x to be segmented whose contribution is greater than the threshold is taken as the abnormal area of ​​the medical image, so as to segment the abnormal area of ​​the medical image. As another non-limiting example, the contribution of each pixel point in the medical image x to be segmented to the discrimination result y is calculated according to the relationship function, and the image area corresponding to the pixel points in the medical image x to be segmented whose contribution is less than or equal to the threshold is taken as the abnormal area of ​​the medical image, so as to segment the abnormal area of ​​the medical image.

[0063] This embodiment obtains the discrimination result of the medical image to be segmented by inputting the medical image to be segmented into the target adversarial discriminant model; according to the discrimination result, the image type of the medical image to be segmented is determined; when the image type of the medical image to be segmented is an abnormal medical image, the relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model is obtained; the relationship function is used to represent the contribution of the input of the target adversarial discriminant model to the output; according to the relationship function, the abnormal area of ​​the medical image is segmented, thereby achieving accurate segmentation of the abnormal area of ​​the medical image. This solves the problem in the prior art that the irregular abnormal area needs to be finely divided and labeled manually before model training. These processes are time-consuming and labor-intensive, and cannot exhaust all possible abnormal patterns, resulting in inaccurate segmentation of the abnormal area of ​​the medical image.

[0064] In some of the embodiments, Figure 3As shown, step S230, segmenting the abnormal area of ​​the medical image according to the relationship function, includes the following steps:

[0065] Step S310, constructing a model residual function according to the relationship function.

[0066] Step S320, segmenting the abnormal area of ​​the medical image according to the model residual function.

[0067] In this step, the residual function of the model is constructed according to the relationship function. The relationship function is used to represent the contribution of the input medical image x to be segmented of the target adversarial discriminant model to the output discrimination result y. When constructing the residual function of the model, the contribution of the model input to the output is taken into account, thereby improving the accuracy of the model.

[0068] In some of the embodiments, the abnormal region of the medical image is segmented according to the model residual function, including: calculating the significance score of each pixel of the medical image to be segmented according to the model residual function; and segmenting the abnormal region of the medical image according to the significance score.

[0069] In this step, the derivative of the model residual function with respect to each pixel of the medical image x to be segmented input by the target adversarial discriminant model is calculated, and the obtained derivative is defined as the significance score of each pixel of the image, where the significance of the abnormal area is high (i.e., the change of a specific pixel makes the model output change greatly), and the significance of the normal area is low (i.e., the change of the pixel has no effect on the model output). Finally, the medical image x to be segmented is segmented by the significance score, and the abnormal area segmentation of the abnormal medical image at the pixel level is realized, thereby improving the accuracy of the abnormal area segmentation of the abnormal medical image.

[0070] In some of the embodiments, the abnormal region segmentation method of a medical image further includes: visually displaying the significance score of each pixel point of the image, thereby improving the efficiency of information acquisition.

[0071] In some embodiments, the abnormal region segmentation method of the medical image further includes: generating a relationship function according to the model parameters of the target adversarial discriminant model. The relationship function is used to represent the relationship between the pixel value of the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model. The neural network model can be regarded as a nonlinear model in nature. For a specific model input, the model can be locally approximated as a linear model by performing a first-order Taylor expansion on the model, and its first-order derivative can represent the degree of influence of each pixel point in the image on the current discrimination result. The model parameters of the target adversarial discriminant model here can be the first-order derivative of the target adversarial discriminant model, and the first-order derivative is determined as a relationship function, that is, the relationship between the pixel value of the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model. In this embodiment, the relationship function is generated according to the model parameters of the target adversarial discriminant model, so that the generated relationship function can better reflect the characteristics of the model, making the segmentation of the abnormal region of the medical image more accurate.

[0072] In some embodiments, the image type of the medical image to be segmented is determined by inputting the medical image to be segmented into the target adversarial discriminant model, including: inputting the medical image to be segmented into the target adversarial discriminant model to obtain a discrimination result of the medical image to be segmented; when the discrimination result meets a preset condition, determining that the image type of the medical image to be segmented is an abnormal medical image; wherein the preset condition is determined according to the model parameters of the target adversarial discriminant model. For example, if the medical image x to be segmented is input into the adversarial discriminant model to obtain a discrimination result y, the discrimination result y can be quantified with a value of 0 to 1, and the preset condition is set as whether the discrimination result is greater than a discrimination threshold, where the discrimination threshold depends on the adjustment parameters of the model, that is, the preset condition is determined according to the model parameters of the target adversarial discriminant model, and the discrimination threshold is set to 0.5. If the value of the discrimination result y is greater than 0.5, the discrimination result meets the preset condition, and the medical image to be segmented is considered to be an abnormal medical image. This embodiment determines a preset condition according to the parameters of the model. When the discrimination result meets the preset condition, the image type of the medical image to be segmented is determined to be an abnormal medical image, thereby improving the determination result of the image type of the medical image.

[0073] This embodiment also provides a model training method. Figure 4 is a flow chart of a model training method of an embodiment of the present application, such as Figure 4 As shown, the process includes the following steps:

[0074] Step S410: construct a first training set based on normal medical image data.

[0075] In this step, normal medical image data is acquired, the normal medical image data is cleaned, and a first training set is constructed based on the cleaned normal medical image data.

[0076] Step S420: construct a second training set based on the random data.

[0077] In this step, random data is obtained through random sampling, and a second training set is constructed based on the random data.

[0078] Step S430: training the initial adversarial discriminant model according to the first training set and the second training set to obtain a target adversarial discriminant model.

[0079] In this step, the initial adversarial discriminant model is trained according to the first training set and the second training set to obtain a target adversarial discriminant model. The initial adversarial discriminant model includes a generator (G) and a discriminator (D). The second training set is input into the generator (G) to generate an artificial image, and the artificial image and the normal medical image in the first training set are input into the discriminator (D) to obtain the discrimination results of the two images respectively. The initial adversarial discriminant model is trained according to the discrimination results of the two images to obtain the target adversarial discriminant model.

[0080] In this embodiment, the initial adversarial discriminant model is trained by normal medical image data to obtain a target adversarial discriminant model. The training of the target adversarial discriminant model does not rely on negative samples (i.e., abnormal medical images), but only needs to use more easily obtained positive samples (i.e., normal medical images) for training. At the same time, the training process does not rely on manual segmentation and labeling in the early stage, which greatly reduces the training difficulty of the adversarial discriminant model and improves the accuracy of the segmentation of abnormal areas of medical images.

[0081] In some of the embodiments, Figure 5 As shown, step S430, training the initial adversarial discriminant model according to the first training set and the second training set to obtain the target adversarial discriminant model, includes the following steps:

[0082] Step S510, obtaining a first discrimination result of a first training set.

[0083] In this step, the initial adversarial discriminant model includes a generator (G) and a discriminator (D). The normal medical images in the first training set are input into the discriminator (D) to obtain the first discrimination result of the first training set.

[0084] Step S520, obtaining a second discrimination result of a second training set.

[0085] In this step, the second training set is input into the generator (G) to generate an artificial image, and the artificial image is input into the discriminator (D) to obtain a discrimination result of the artificial image, that is, a second discrimination result of the second training set.

[0086] Step S530: construct an objective function according to the first discrimination result and the second discrimination result.

[0087] In this step, an objective function is constructed according to the first discrimination result and the second discrimination result. The first discrimination result and the second discrimination result can be represented by numerical values. For example, the output value of the discriminator result is a numerical value between 0 and 1. A discrimination threshold is set. If it is greater than the discrimination threshold, it is judged as an abnormal sample, and if it is less than the discrimination threshold, it is judged as a normal sample. The objective function can be the difference between the first discrimination result and the second discrimination result.

[0088] Step S540, training the initial adversarial discriminant model according to the objective function to obtain a target adversarial discriminant model.

[0089] In this step, the initial adversarial discriminant model is trained according to the objective function. When the objective function reaches a preset value, the training of the initial adversarial discriminant model is completed, and the target adversarial discriminant model is obtained.

[0090] In this embodiment, the initial adversarial discrimination model is trained according to the difference between the discrimination results of artificial images and normal medical image data, thereby improving the distance between artificial images and normal medical image data in the feature space, thereby making the trained target adversarial discrimination model more accurate.

[0091] This embodiment also provides a method for segmenting abnormal regions of medical images. Figure 6 is a flowchart of another abnormal region segmentation method of a medical image according to an embodiment of the present application. Figure 6 As shown, the process includes the following steps:

[0092] Step S610, constructing a training data set.

[0093] In this step, normal medical image data is collected as a training data set. Medical images of specific parts of normal patients or target organs (such as thyroid gland, liver, etc.) are collected to prepare a normal sample set. The normal sample set is cleaned. The purpose of cleaning is to remove abnormal samples and ensure that there are only normal medical images in the normal data set. The cleaning method can be manual cleaning or cleaning using a supervised binary classification discriminant model.

[0094] Step S620, constructing an adversarial discriminant model.

[0095] In this step, the adversarial discriminant model is an adversarial discriminant neural network model, and the adversarial discriminant model includes a feature extraction module, a generator (G), and a discriminator (D). Among them, normal medical images are mapped to image feature space through the feature extraction module; the generator (G) generates artificial samples in the feature space; the discriminator (D) delineates the discrimination boundary in the feature space, wherein the samples within the boundary are identified as normal samples, and the samples outside the boundary are identified as abnormal samples. The feature extraction module, the generator (G), and the discriminator (D) are all composed of multi-layer convolution, pooling, full connection and other neural network structures, and the model parameters are obtained by initialization. The generator (G) and the discriminator (D) both have feature extraction modules, and the feature extraction modules of the two can be shared or set separately. The feature extraction module is composed of a multi-layer convolution layer and a pooling layer structure, and its function is to map the medical image to the feature space; the generator (G) is composed of a multi-layer convolution layer and a deconvolution layer, and its function is to map the random features to the medical image space, thereby generating an artificial image; the discriminator (D) is composed of a convolution layer, a pooling layer, a fully connected layer, a nonlinear mapping layer, etc. The feature extraction module inputs the obtained features into the discriminator (D), and the discriminator (D) performs a binary classification. All parameters in the adversarial discriminant model are randomly initialized. Random feature z is obtained by random sampling from a normal distribution, and the random feature z is input into the generator (G) to obtain an artificial sample G(z). A normal sample is extracted from the normal sample set obtained in step S510, and the extracted normal sample and the artificial sample G(z) are input into the discriminator (D) to obtain a discrimination result. After normalization, the output range of the discriminator (D) falls in the range of 0 to 1. It is defined that the larger the output of the discriminator (D), the more abnormal the sample is, and vice versa, the more normal the sample is. The artificial sample G(z) and the collected normal sample x are simultaneously input into the discriminator (D) to obtain the discrimination results. The output value of the discriminator (D) is between 0 and 1. The discrimination threshold is set to a (which depends on the parameter adjustment of the adversarial discrimination model). Samples greater than or equal to the discrimination threshold are considered abnormal samples, and samples less than the discrimination threshold are considered normal samples. The greater the difference between the discrimination result of the artificial sample G(z) and the discrimination result of the normal sample x, the better. That is, the greater the distance in the feature space, the better. The parameters of all modules are initially obtained by random initialization. Because they are initialized by random methods, it can be considered that in the feature space, the discrimination boundary constructed by the discriminator (D) is loose enough and includes both normal and abnormal samples. The generator (G) is also initialized by random methods. It can be considered that the generated samples are quite different from normal samples. The model is completed through continuous iterative training. During the training process, the training goal of the generator (G) is to generate samples that are close to the distribution of normal samples, and the training goal of the discriminator (D) is to define a sufficiently fine discrimination boundary to distinguish normal samples from artificial samples generated by the generator (G).During the training process, the initially initialized loose discriminant boundary is continuously compressed by artificial samples, and finally a discriminant boundary is constructed that tightly surrounds the distribution of normal samples (because it is tight enough, so in an ideal state, the inside of the boundary only contains normal samples, and abnormal samples fall outside the boundary), thereby achieving the distinction between normal samples and abnormal samples and obtaining an adversarial discriminant model.

[0096] Step S630, training an adversarial discriminant model.

[0097] In this step, the training of the adversarial discriminant model can be carried out in batches. A batch of random features z, N in total, are constructed by random sampling from a normal distribution and input into the generator (G) to obtain a batch of artificial samples. Normal samples of the same number as the artificial samples obtained above are randomly selected from the normal sample set, and the obtained artificial samples and normal samples are input into the discriminator (D) to obtain the discrimination results. The generator (G) and the discriminator (D) are trained by gradient descent, as shown in formula (1) and formula (2). The optimization target of the generator (G) is L G , the optimization target of the discriminator (D) is L D . Where D(G(z)) is the discrimination result of the artificial sample G(z), and D(x) is the discrimination result of the normal sample x.

[0098] L G = max(D(G(z))) (1)

[0099] L D = max(D(x)+(1-D(G(z)))) (2)

[0100] During the training process, as the training progresses, the learning rates of the generator (G) and the discriminator (D) need to be weighted and adjusted to control the learning speed of the generator (G) and the discriminator (D). The generator (G) and the discriminator are trained alternately, and the discrimination effect of the discriminator is recorded, and the accuracy, misjudgment rate, and overall accuracy of the discriminator are recorded respectively. During the training process, as the discriminator converges, the accuracy gradually increases, the error rate gradually decreases, and the overall accuracy increases. This process indicates that the generator (G) gradually fits the feature distribution of normal samples, and the discriminant boundary fitted by the discriminator gradually shrinks. Finally, a discriminant boundary that tightly surrounds the feature distribution of normal samples is formed in the feature space, thereby realizing the training of the abnormal medical image discrimination model that only relies on normal samples. In the early stage of training, the learning rate of the generator (G) is relatively high, the purpose is to make the generator (G) learn the normal sample data distribution characteristics as soon as possible. As the training progresses, the learning rate of the generator (G) is gradually reduced, while the learning rate of the discriminator (D) is increased, aiming to depict a fine discrimination boundary in the later stage of training. The generator (G) and the discriminator (D) are optimized according to the discrimination results, wherein the optimization goal of the generator (G) is to reduce the discrimination accuracy of the discriminator (D), that is, the artificial sample G(z) obtained by the generator (G) falls within the discrimination boundary constructed by the discriminator (D); the optimization goal of the discriminator (D) is to increase the discrimination accuracy, that is, to make the discrimination boundary more able to distinguish normal samples from artificial samples G(z). Model convergence visualization: During the training process, the convergence process of the generator (G) and the discriminator (D) is statistically analyzed. When the model completes convergence, the discriminator (D) is obtained as the final abnormal ultrasonic adversarial discrimination model. Optionally, the discriminator (D) is fine-tuned. During the fine-tuning process, the parameters of the generator (G) are fixed, and only the discriminator (D) is optimized. A small number of artificial samples G(z) can be used in training, or a certain number of real abnormal samples can be added to the artificial samples, and the discriminator (D) is optimized with a small learning rate alone.

[0101] Step S640: segment the abnormal medical image using a significant factor analysis method.

[0102] In this step, after completing the training, the adversarial discriminant model is obtained, and the image segmentation at the pixel level is performed by the significance analysis method. The adversarial discriminant model obtained by the above training is a nonlinear model. For the nonlinear model, the nonlinear model can be approximated as a linear model locally by Taylor's first-order expansion, wherein the first-order derivative of the adversarial discriminant model locally can represent the contribution rate of each pixel point in the input image to the current discrimination result, that is, the influence degree of each pixel point in the input image on the current discrimination result, that is, significance. The model parameters of the adversarial discriminant model are fixed, the sample image is input into the network to obtain the discrimination result, and the contribution degree of each pixel point in the sample image to the discrimination result is analyzed, that is, significance. Pixel points with high significance are considered to be abnormal points, and vice versa as normal points. The significance is used as the basis for distinguishing abnormal pixels, thereby realizing the division of abnormal areas. The medical image x to be segmented is input into the adversarial discriminant model to obtain the discrimination result y, and the discrimination threshold is set to 0.5. If the value of the discrimination result y is greater than 0.5, the input sample is considered to be an abnormal sample. For abnormal samples, construct the model residual function Res = ‖1-y‖, calculate the derivative of the model residual function for each pixel of the model input x, and obtain the significance of each pixel in the input image to the output of the adversarial discriminant model. Visualize the final saliency image, and segment the image by saliency. The significance of the abnormal area is high (that is, the change of a specific pixel makes the model output larger), and the significance of the normal area is low (that is, the change of the pixel has no effect on the model output). Finally, the image is segmented by the significance score to achieve pixel-level abnormal image area segmentation.

[0103] In the prior art, it is difficult to train medical image recognition models based on artificial intelligence. On the one hand, model training requires the collection of a large number of training samples, including normal medical image samples and abnormal ultrasound samples of abnormal areas reflecting tissue lesions. Secondly, for the abnormal area segmentation task, the irregular abnormal areas need to be finely divided and labeled manually before model training. These processes are time-consuming and laborious, and cannot exhaust all possible abnormal patterns, which has become the main problem restricting the training of artificial intelligence medical image segmentation models. The abnormal area segmentation method of medical images provided in the embodiment of the present application enables the adversarial discriminant model to be trained without the help of negative samples, only using more easily available positive samples, and the training process does not rely on the previous manual segmentation and annotation, which greatly reduces the difficulty of training the adversarial discriminant model, improves the accuracy of abnormal area segmentation of medical images, and improves the application of artificial intelligence models in multi-scene ultrasound abnormal area segmentation. The embodiment of the present application constructs an adversarial discriminant neural network, maps the medical image to the feature space, and constructs a discriminant boundary that compactly surrounds the distribution of normal medical images in the feature space through training, thereby distinguishing normal medical images from abnormal medical images, and based on the significant factor analysis of the obtained discriminant model, realizes pixel-level abnormal medical image area segmentation.

[0104] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0105] The present application also provides an abnormal region segmentation device for medical images, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0106] Figure 7 is a structural block diagram of a device for segmenting abnormal regions of medical images according to an embodiment of the present application. Figure 7 As shown, the device comprises:

[0107] A discrimination module 710, for inputting the medical image to be segmented into a target adversarial discrimination model to determine the image type of the medical image to be segmented;

[0108] The relationship function acquisition module 720 is used to acquire the relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model when the image type of the medical image to be segmented is an abnormal medical image; the relationship function is used to represent the contribution of the input to the output of the target adversarial discriminant model;

[0109] The segmentation module 730 is used to segment the abnormal area of ​​the medical image according to the relationship function.

[0110] Figure 8 is a structural block diagram of a training device for a model in an embodiment of the present application, such as Figure 8 As shown, the device comprises:

[0111] A first training set construction module 810, configured to construct a first training set according to normal medical image data;

[0112] A second training set construction module 820, configured to construct a second training set based on random data;

[0113] The model training module 830 is used to train the initial adversarial discriminant model according to the first training set and the second training set to obtain a target adversarial discriminant model.

[0114] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0115] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0116] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0117] Optionally, in the embodiment of the present application, the processor may be configured to perform the following steps through a computer program:

[0118] S1, inputting the medical image to be segmented into the target adversarial discriminant model to determine the image type of the medical image to be segmented;

[0119] S2, when the image type of the medical image to be segmented is an abnormal medical image, obtaining a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model; the relationship function is used to represent the contribution of the input to the output of the target adversarial discriminant model;

[0120] S3, segmenting the abnormal area of ​​the medical image according to the relationship function.

[0121] Optionally, in the embodiment of the present application, the processor may also be configured to perform the following steps through a computer program:

[0122] Q1, construct the first training set based on normal medical image data;

[0123] Q2, construct the second training set based on random data;

[0124] Q3, train the initial adversarial discriminant model based on the first training set and the second training set to obtain the target adversarial discriminant model;

[0125] Q4, inputting the medical image to be segmented into the target adversarial discriminant model to determine the image type of the medical image to be segmented;

[0126] Q6, when the image type of the medical image to be segmented is an abnormal medical image, obtaining a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model; the relationship function is used to represent the contribution of the input to the output of the target adversarial discriminant model;

[0127] Q7, segment the abnormal area of ​​the medical image based on the relationship function.

[0128] It should be noted that the specific examples in the embodiments of the present application can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in the embodiments of the present application.

[0129] In addition, in combination with the abnormal region segmentation method of a medical image provided in the above embodiment, a storage medium can also be provided in the embodiment of the present application to implement the abnormal region segmentation method of a medical image. The storage medium stores a computer program; when the computer program is executed by a processor, any abnormal region segmentation method of a medical image in the above embodiment is implemented.

[0130] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0131] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0132] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0133] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for segmenting abnormal regions of medical images, characterized in that: The method comprises: Inputting the medical image to be segmented into the target adversarial discrimination model to determine the image type of the medical image to be segmented; When the image type of the medical image to be segmented is an abnormal medical image, obtaining a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model; the relationship function is used to represent the contribution of the input to the output of the target adversarial discriminant model; The abnormal area of ​​the medical image is segmented according to the relationship function.

2. The abnormal region segmentation method of medical images according to claim 1, characterized in that: The step of segmenting the abnormal region of the medical image according to the relationship function includes: Constructing a model residual function according to the relationship function; The abnormal region of the medical image is segmented according to the model residual function.

3. The abnormal region segmentation method of medical images according to claim 2, characterized in that: The step of segmenting the abnormal region of the medical image according to the model residual function comprises: Calculating a significance score of each pixel of the medical image to be segmented according to the model residual function; The abnormal region of the medical image is segmented according to the significance score.

4. The abnormal region segmentation method of medical images according to claim 1, characterized in that: The relationship function is used to represent the relationship between the pixel values ​​of the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model, and the method further includes: The relationship function is generated according to the model parameters of the target adversarial discriminant model.

5. The abnormal region segmentation method of medical images according to claim 1, characterized in that: The step of inputting the medical image to be segmented into the target adversarial discrimination model to determine the image type of the medical image to be segmented includes: Inputting the medical image to be segmented into the target adversarial discrimination model to obtain a discrimination result of the medical image to be segmented; When the discrimination result satisfies a preset condition, determining that the image type of the medical image to be segmented is an abnormal medical image; Among them, the preset conditions are determined according to the model parameters of the target adversarial discrimination model.

6. A model training method, characterized in that: The method comprises: Constructing a first training set according to normal medical image data; Construct a second training set based on random data; An initial adversarial discriminant model is trained according to the first training set and the second training set to obtain a target adversarial discriminant model.

7. The training method of the model according to claim 6, characterized in that: The step of training the initial adversarial discriminant model according to the first training set and the second training set to obtain a target adversarial discriminant model includes: Obtaining a first discrimination result of the first training set; Obtaining a second discrimination result of the second training set; constructing an objective function according to the first discrimination result and the second discrimination result; The initial adversarial discriminant model is trained according to the objective function to obtain a target adversarial discriminant model.

8. A device for segmenting abnormal regions of medical images, characterized in that: The device comprises: A discrimination module, used for inputting the medical image to be segmented into a target adversarial discrimination model to determine the image type of the medical image to be segmented; A relationship function acquisition module, used for acquiring a relationship function between the medical image to be segmented input into the target adversarial discriminant model and the discrimination result output by the target adversarial discriminant model when the image type of the medical image to be segmented is an abnormal medical image; the relationship function is used to represent the contribution of the input of the target adversarial discriminant model to the output; A segmentation module is used to segment the abnormal area of ​​the medical image according to the relationship function.

9. A model training device, characterized in that: The device comprises: A first training set construction module, used to construct a first training set according to normal medical image data; A second training set construction module, used to construct a second training set based on random data; The model training module is used to train the initial adversarial discriminant model according to the first training set and the second training set to obtain a target adversarial discriminant model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the abnormal region segmentation method of a medical image described in any one of claims 1 to 5 are implemented, or when the computer program is executed by a processor, the steps of the model training method described in any one of claims 6 to 7 are implemented.

Images