Segmentation model training method, image segmentation method, device, equipment and medium

Abstract

The present invention discloses a segmentation model training method, an image segmentation method, a device, an equipment and a medium. The method includes obtaining a sample image for model training and a pixel label of each pixel in the sample image; the pixel includes a target pixel and a background pixel; iteratively executing the following training process until the training completion condition is met to obtain a target segmentation model: inputting the sample image into the segmentation model to be trained to obtain a sample segmentation result of each pixel in the sample image; calculating a model loss function based on each pixel label and a sample prediction result; wherein the model loss function includes a first loss function generated based on the label of the target pixel and a second loss function generated based on the label of the background pixel; adjusting the model parameters of the segmentation model based on the model loss function. Through the technical solution disclosed by the present invention, the problem of inaccurate image segmentation in the prior art is solved, and the accuracy of the image segmentation result is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a segmentation model training method, an image segmentation method, a device, equipment and a medium. Background Art

[0002] With the advancement of non-invasive imaging diagnostic technologies such as computed tomography (CT) and magnetic resonance imaging (MRI), higher and higher requirements are placed on clinicians. In order to reduce the workload of doctors and avoid the influence of personal subjectivity, computer-aided diagnosis technology based on image processing and artificial intelligence has become a hot topic of research around the world.

[0003] The existing technologies generally extract a series of features such as statistical features and texture features of the image first, and then use artificial neural networks, random forests and support vector machines to segment. However, in the implementation process of the existing technologies, the following problems are found: low computational efficiency, high labor costs and poor robustness, and it is very easy to over-segment the target. Summary of the invention

[0004] The present invention provides a segmentation model training method, an image segmentation method, a device, an equipment and a medium to solve the problem of inaccurate image segmentation in the prior art and improve the accuracy of image segmentation results.

[0005] In a first aspect, an embodiment of the present invention provides a segmentation model training method, the method comprising:

[0006] Acquire a sample image for model training and a pixel label of each pixel in the sample image; wherein the pixel includes a target pixel and a background pixel;

[0007] The following training process is iteratively performed until the training completion condition is met to obtain the target segmentation model: the sample image is input into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image;

[0008] Calculating a model loss function based on the pixel labels and the sample prediction results; wherein the model loss function includes a first loss function generated based on the target pixel label and a second loss function generated based on the background pixel label;

[0009] The model parameters of the segmentation model are adjusted based on the model loss function.

[0010] Optionally, the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component;

[0011] Accordingly, the step of inputting the sample image into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image includes:

[0012] For any level coding module, a convolution operation is performed on the sample image / the coding features output after downsampling processing of the previous level coding module based on the first preset convolution parameter to obtain the coding features after feature extraction of the current level coding module;

[0013] For any level decoding module, a convolution operation is performed on the code output after downsampling processing of the previous level encoding module based on the second preset convolution parameters, or on the spliced ​​features after splicing the code features output by the current level encoding module and the image features output after upsampling processing of the previous level decoding module, and feature extraction is performed on the image features after the convolution operation based on the residual attention component to obtain the image features after feature extraction of the current level decoding module.

[0014] Optionally, the calculating the model loss function based on each of the pixel point labels and the sample prediction results includes:

[0015] Obtaining a first sample prediction result corresponding to a target pixel in the sample prediction results, and generating a first loss function of the segmentation model based on the first sample prediction result and a label of the target pixel;

[0016] Obtaining a second sample prediction result corresponding to a background pixel in the sample prediction result, and generating a second loss function of the segmentation model based on the second sample prediction result and a label of the background pixel;

[0017] A model loss function of the segmentation model is determined based on the first loss function, the second loss function, a preset first loss function weight, and a preset second loss function weight.

[0018] Optionally, the generating a second loss function of the segmentation model based on the second sample prediction result and the label of the background pixel point includes:

[0019] Determine candidate background pixels among the background pixels based on a comparison result between the label of the background pixel and the second sample prediction result, and determine target background pixels among the candidate background pixels based on a preset screening condition;

[0020] A second loss function of the segmentation model is generated based on the label of the target background point and the sample prediction result corresponding to the target background point in the second sample prediction result.

[0021] Optionally, before inputting the sample image into the segmentation model to be trained, the method further includes:

[0022] Cropping the sample into a preset size to obtain at least two cropped sample images;

[0023] Correspondingly, obtaining the sample prediction result of each pixel in the sample image also includes:

[0024] Outputting the initial sample prediction result of the cropped sample image based on an overlapping sliding window method;

[0025] The initial sample prediction results are averaged based on the number of cropped sample images to obtain sample prediction results of the sample images.

[0026] In a second aspect, an embodiment of the present invention further provides an image segmentation method, the method comprising:

[0027] Obtain the image to be segmented;

[0028] The image to be segmented is input into a pre-trained segmentation model to obtain a segmentation result output by the segmentation model; wherein the segmentation model is trained based on the segmentation model training method described in any of the above embodiments.

[0029] In a third aspect, an embodiment of the present invention further provides a segmentation model training device, the device comprising:

[0030] A sample image acquisition module, used to acquire sample images for model training and pixel labels of each pixel in the sample images; wherein the pixel includes target pixel and background pixel;

[0031] The target segmentation model acquisition module is used to iteratively perform the following training process until the training completion condition is met to obtain the target segmentation model: input the sample image into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image;

[0032] Calculating a model loss function based on the pixel labels and the sample prediction results; wherein the model loss function includes a first loss function generated based on the target pixel label and a second loss function generated based on the background pixel label;

[0033] The model parameters of the segmentation model are adjusted based on the model loss function.

[0034] In a fourth aspect, an embodiment of the present invention further provides an image segmentation device, the device comprising:

[0035] An image acquisition module, used for acquiring an image to be segmented;

[0036] The segmentation result acquisition module is used to input the image to be segmented into a pre-trained segmentation model to obtain the segmentation result output by the segmentation model; wherein the segmentation model is trained based on the segmentation model training method described in any of the above embodiments.

[0037] In a fifth aspect, an embodiment of the present invention further provides an electronic device, including:

[0038] at least one processor; and

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the segmentation model training method and / or image segmentation method described in any embodiment of the present invention.

[0041] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the segmentation model training method and / or image segmentation method described in any embodiment of the present invention when executed.

[0042] The technical solution of the embodiment of the present invention specifically includes: obtaining a sample image for model training and a pixel label of each pixel in the sample image; wherein the pixel includes a target pixel and a background pixel; iteratively executing the following training process until the training completion condition is met to obtain a target segmentation model: inputting the sample image into the segmentation model to be trained to obtain a sample prediction result of each pixel in the sample image; calculating a model loss function based on each pixel label and the sample prediction result; wherein the model loss function includes a first loss function generated based on the label of the target pixel and a second loss function generated based on the label of the background pixel; adjusting the model parameters of the segmentation model based on the model loss function. The above scheme adjusts the model parameters based on the first loss function generated by the positive sample in the training sample and the second loss function generated based on the negative sample to obtain a trained segmentation model, so that the model can pay part of its attention to the extraction of negative sample features in the process of learning to identify positive samples, thereby further improving the accuracy of model segmentation.

[0043] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a flow chart of a segmentation model training method provided by an embodiment of the present invention;

[0046] Figure 2 is a model structure diagram of a segmentation model proposed in an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a principle of a loss function provided by an embodiment of the present invention;

[0048] Figure 4 is a flow chart of an image segmentation method provided by an embodiment of the present invention;

[0049] Figure 5 is a structural schematic diagram of a segmentation model training device provided in Embodiment 3 of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of an image segmentation device provided by Embodiment 3 of the present invention.

[0051] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0054] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0055] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0056] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0057] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0058] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0059] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0060] Figure 1 A flow chart of a segmentation model training method is provided for an embodiment of the present invention. This embodiment is applicable to the case of training a model. The method can be performed by a segmentation model training device. The segmentation model training device can be implemented in the form of hardware and / or software. The segmentation model training device can be configured in an intelligent terminal and a cloud server. Figure 1 As shown, the method includes:

[0061] S110, obtaining a sample image for model training and a pixel label of each pixel in the sample image; wherein the pixel includes a target pixel and a background pixel.

[0062] S120, iteratively execute the following training process until the training completion conditions are met to obtain the target segmentation model: input the sample image into the segmentation model to be trained to obtain the sample prediction results of each pixel in the sample image; calculate the model loss function based on the label of each pixel and the sample prediction result; wherein the model loss function includes a first loss function generated based on the label of the target pixel and a second loss function generated based on the label of the background pixel; adjust the model parameters of the segmentation model based on the model loss function.

[0063] In order to improve the accuracy of the segmentation results in the process of segmenting images based on the segmentation model, the embodiment of the present invention adds a residual attention component to the existing segmentation model to improve the ability to extract image feature details of the image to be segmented, thereby improving the accuracy of the segmentation results obtained based on the segmentation model. Furthermore, the technical solution of this embodiment also adjusts the model parameters by adjusting the first loss function generated based on the positive samples in the training samples and the second loss function generated based on the negative samples during the training of the segmentation model with the residual attention component, so as to obtain a trained segmentation model, so that the model can pay part of its attention to the extraction of negative sample features during the process of learning to identify positive samples, thereby further improving the accuracy of model segmentation.

[0064] The technical solution of the embodiment of the present invention is to obtain a sample image for model training in advance before training the segmentation model. Optionally, if the sample image is a two-dimensional image, each pixel constituting the sample image is obtained, and each pixel is labeled to obtain a pixel label of each pixel in the sample image. Among them, the pixel includes a target pixel and a background pixel. In the process of labeling the sample image, exemplarily, if the sample image includes a target area where a portrait is located and a background area where other content is located, the pixel constituting the target area is labeled as a target pixel, i.e., a positive sample pixel, and the pixel constituting the background area is labeled as a background pixel, i.e., a background pixel.

[0065] It should be noted that the sample image may also include a three-dimensional image; accordingly, each voxel point constituting the sample image is obtained, and based on labeling each voxel point, a voxel point label of each voxel point in the sample image is obtained. This embodiment does not limit the image dimension of the sample image.

[0066] When a sample image is obtained, a segmentation model to be trained is obtained, and the model is iteratively trained based on the sample image and the pixel labels of each pixel in the sample image until the training completion condition is met to obtain a target segmentation model.

[0067] On the basis of the above implementation, the technical solution of this embodiment also includes normalizing the image processing of each sample image after acquiring the sample image, and performing data enhancement processing on the sample image to enhance the generalization ability of the model so that the model can process different image data. Optionally, the data enhancement processing method includes but is not limited to: random rotation, random translation, random brightness change, local brightness transformation and other processing methods.

[0068] Specifically, the training process for any iterative round includes: inputting the sample image into the segmentation model to be trained to obtain the sample prediction results of each pixel in the sample image; calculating the model loss function based on the label of each pixel and the sample prediction results; wherein the model loss function includes a first loss function generated based on the label of the target pixel and a second loss function generated based on the label of the background pixel; and adjusting the model parameters of the segmentation model based on the model loss function.

[0069] In this embodiment, the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component; accordingly, the sample image is input into the segmentation model to be trained to obtain a sample prediction result for each pixel in the sample image, including: for any level of encoding modules, a convolution operation is performed on the sample image / the encoding features output after downsampling processing of the encoding module of the previous level based on a first preset convolution parameter to obtain the encoding features after feature extraction by the encoding module of the current level; for any level of decoding modules, a convolution operation is performed on the encoding output after downsampling processing of the encoding module of the previous level based on a second preset convolution parameter, or on the splicing features after splicing the encoding features output by the encoding module of the current level with the image features output after upsampling processing of the decoding module of the previous level, and feature extraction is performed on the image features after the convolution operation based on the residual attention component to obtain the image features after feature extraction by the decoding module of the current level.

[0070] Specific participation Figure 2 , obtain a sample image, perform convolution processing on the input sample image based on the first-level encoding module in the segmentation model, and downsample the image features after convolution to obtain the encoding features output by the encoding module of the current level; then input the encoding features to the encoding modules of the subsequent levels until the encoding features output by the encoding module of the last level after downsampling are obtained. The encoding features are used as the input features of the decoding module of the last level, and the encoding features are adaptively refined based on the residual attention component in the decoding module of the current level, and the image features after adaptive feature refinement are upsampled to obtain the image features output by the decoding module of the current level

[0071] Specifically, the image feature is used as the first input feature of the decoding module of the previous level, the encoding module of the same level of the decoding module of the previous level before downsampling is used as the second input feature of the decoding module of the previous level, the first input feature and the second input feature are feature spliced, the image feature after feature splicing is adaptively refined based on the residual attention component in the decoding module of the current level, and the image feature after adaptive feature refinement is upsampled to obtain the output image feature of the decoding module of the current level; the image feature is then used as the first input feature of the decoding module of the next higher level, until the image feature output by the decoding module of the last level is obtained, and the sample prediction result corresponding to the sample image is obtained based on the image feature.

[0072] It should be noted that the sample prediction result includes the sample prediction result of each pixel in the sample image, specifically including the first sample prediction result corresponding to the target pixel and the second sample prediction result corresponding to the background pixel.

[0073] Optionally, obtain a first sample prediction result corresponding to a target pixel in the sample prediction result, and generate a first loss function of the segmentation model based on the first sample prediction result and the label of the target pixel; obtain a second sample prediction result corresponding to a background pixel in the sample prediction result, and generate a second loss function of the segmentation model based on the second sample prediction result and the label of the background pixel; determine the model loss function of the segmentation model based on the first loss function, the second loss function, a preset first loss function weight, and a preset second loss function weight.

[0074] Optionally, the first loss function may be generated based on the following expression. Exemplarily, the expression includes:

[0075]

[0076] Among them, P-Dice represents the first loss function; X + Indicates the pixel label of the target pixel; Y + Represents the prediction result of the first sample.

[0077] Optionally, a second loss function of the segmentation model is generated based on the second sample prediction result and the label of the background pixel point, including: determining candidate background pixel points among the background pixel points based on the comparison result of the label of the background pixel point and the second sample prediction result, and determining target background pixel points among the candidate background pixel points based on preset screening conditions; generating the second loss function of the segmentation model based on the label of the target background point and the sample prediction result corresponding to the target background point in the second sample prediction result.

[0078] For more information, see Figure 3, the method of determining the second loss function may include: obtaining the pixel label of each pixel in the sample image, that is, label; and obtaining the sample prediction result of each pixel in the sample image, that is, logits; respectively subtracting the predicted logits and label from 1, and exchanging the labels of the target pixel and the background pixel to obtain logits- and label-. Since the number of target pixels is generally much smaller than the number of background pixels, it is necessary to filter the loss part to filter out a part of difficult-to-predict sample pixels, that is, sample pixels that are easy to predict wrongly. The second loss function is calculated based on the filtered logits- and label-.

[0079] Optionally, the second loss function may be generated based on the following expression. Exemplarily, the expression includes:

[0080]

[0081] Among them, N-Dice represents the second loss function; X - Y represents the pixel label of the background pixel; - Represents the second sample prediction result.

[0082] Optionally, a first loss function weight of the first loss function is obtained, a second loss function weight of the second loss function is obtained, and a model loss function of the segmentation model is determined based on the first loss function, the second loss function, the first loss function weight, and the second loss function weight.

[0083] Specifically, the model loss function can be generated based on the following expression. Exemplarily, the expression includes:

[0084]

[0085] Among them, PNDice represents the model loss function; α represents the weight of the first loss function; 1-α represents the weight of the second loss function.

[0086] On the basis of the above-mentioned implementation manner, the technical solution of the embodiment of the present invention also includes: before inputting the sample image into the segmentation model to be trained, it also includes: cropping the sample to a preset size to obtain at least two cropped sample images; accordingly, obtaining the sample prediction result of each pixel point in the sample image, and also includes: outputting the initial sample prediction result of the cropped sample image based on the overlapping sliding window method; performing mean processing on each initial sample prediction result based on the number of cropped sample images to obtain the sample prediction result of the sample image.

[0087] Specifically, the entire sample image is predicted in turn using a 1 / 2 overlapping sliding window, and finally the average of each predicted pixel is taken, and then a fixed threshold is selected to obtain the sample prediction result; the 1 / 2 overlapping slider is used in the above implementation to solve the edge effect caused by the block prediction. Of course, overlapping sliding windows of other lengths can also be used, and this embodiment is not limited to this.

[0088] Optionally, the technical solution of this embodiment also includes: using connected domain post-processing to remove false positives in the sample prediction results whose pixel values ​​are less than a certain threshold, resampling the results to the original size and mapping them to the original sample image coordinates to obtain the final prediction results, further improving the learning ability of the model, and improving the accuracy of the results of the segmentation model in subsequent segmentation processing.

[0089] The technical solution of the embodiment of the present invention specifically includes: obtaining a sample image for model training and a pixel label of each pixel in the sample image; wherein the pixel includes a target pixel and a background pixel; iteratively executing the following training process until the training completion condition is met to obtain a target segmentation model: inputting the sample image into the segmentation model to be trained to obtain a sample prediction result of each pixel in the sample image; calculating a model loss function based on each pixel label and the sample prediction result; wherein the model loss function includes a first loss function generated based on the label of the target pixel and a second loss function generated based on the label of the background pixel; adjusting the model parameters of the segmentation model based on the model loss function. The above scheme adjusts the model parameters based on the first loss function generated by the positive sample in the training sample and the second loss function generated based on the negative sample to obtain a trained segmentation model, so that the model can pay part of its attention to the extraction of negative sample features in the process of learning to identify positive samples, thereby further improving the accuracy of model segmentation.

[0090] Figure 4 This is a flowchart of an image segmentation method provided by an embodiment of the present invention. This embodiment is applicable to the case of image segmentation. The method can be executed by an image segmentation device. The image segmentation device can be implemented in the form of hardware and / or software. The image segmentation device can be configured in a smart terminal and a cloud server. Figure 4 As shown, the method includes:

[0091] S210: Obtain an image to be segmented.

[0092] S220. Input the image to be segmented into a pre-trained segmentation model to obtain a segmentation result output by the segmentation model; wherein the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component.

[0093] The technical solution of the embodiment of the present invention specifically includes: obtaining an image to be segmented and inputting the image to be segmented into a pre-trained segmentation model to obtain a segmentation result output by the segmentation model; wherein the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component. The above technical solution improves the ability to extract image feature details of the image to be segmented by adding a residual attention component to the existing segmentation model, thereby improving the accuracy of the segmentation result obtained based on the segmentation model.

[0094] Figure 5 Schematic diagram of the structure of a segmentation model training device provided by an embodiment of the present invention. Figure 5 As shown, the device includes: a sample image acquisition module 310 and a target segmentation model acquisition module 320; wherein,

[0095] The sample image acquisition module 310 is used to acquire sample images for model training and pixel labels of each pixel in the sample images; wherein the pixel includes target pixel and background pixel;

[0096] The target segmentation model acquisition module 320 is used to iteratively execute the training process until the training completion condition is met to obtain the target segmentation model.

[0097] Optionally, the target segmentation model acquisition module 320 includes:

[0098] A sample prediction result obtaining unit is used to input the sample image into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image;

[0099] A model loss function generating unit, used to calculate the model loss function based on the labels of each pixel point and the sample prediction result; wherein the model loss function includes a first loss function generated based on the label of the target pixel point and a second loss function generated based on the label of the background pixel point;

[0100] The target segmentation model obtaining unit is used to adjust the model parameters of the segmentation model based on the model loss function.

[0101] Based on the above implementation, optionally, the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component;

[0102] Correspondingly, the sample prediction result obtaining unit includes:

[0103] The coding feature acquisition subunit is used to perform a convolution operation on the sample image / the coding features output after downsampling processing of the previous level coding module based on the first preset convolution parameter for any level coding module, so as to obtain the coding features after feature extraction of the current level coding module;

[0104] The image feature acquisition subunit is used to perform a convolution operation on the code output after downsampling processing of the previous level encoding module based on the second preset convolution parameters for any level decoding module, or on the spliced ​​features after splicing the code features output by the current level encoding module and the image features output after upsampling processing of the previous level decoding module, and to extract features of the image features after the convolution operation based on the residual attention component to obtain the image features after feature extraction of the current level decoding module.

[0105] Based on the above implementation, optionally, the model loss function generating unit includes:

[0106] A first loss function generating subunit is used to obtain a first sample prediction result corresponding to a target pixel point in the sample prediction result, and to generate a first loss function of a segmentation model based on the first sample prediction result and a label of the target pixel point;

[0107] A second loss function generating subunit is used to obtain a second sample prediction result corresponding to a background pixel in the sample prediction result, and generate a second loss function of the segmentation model based on the second sample prediction result and a label of the background pixel;

[0108] The model loss function generating subunit is used to determine the model loss function of the segmentation model based on the first loss function, the second loss function, the preset first loss function weight and the preset second loss function weight.

[0109] Based on the above implementation, optionally, the second loss function generating subunit includes:

[0110] A target background pixel point generating component is used to determine candidate background pixel points among the background pixel points based on a comparison result between the label of the background pixel point and the prediction result of the second sample, and to determine target background pixel points among the candidate background pixel points based on a preset screening condition;

[0111] The second loss function generating component is used to generate a second loss function of the segmentation model based on the label of the target background point and the sample prediction result corresponding to the target background point in the second sample prediction result.

[0112] Based on the above implementation, optionally, the target segmentation model obtaining module 320 further includes:

[0113] A sample image cropping unit, used to crop the sample into a preset size before inputting the sample image into the segmentation model to be trained, so as to obtain at least two cropped sample images;

[0114] Correspondingly, the sample prediction result obtaining unit further includes:

[0115] An initial sample prediction result obtaining subunit, used for outputting the initial sample prediction result of the cropped sample image based on an overlapping sliding window method;

[0116] The sample prediction result obtaining unit is used to perform mean processing on each initial sample prediction result based on the number of cropped sample images to obtain a sample prediction result of the sample image.

[0117] The segmentation model training device provided in the embodiment of the present invention can execute the segmentation model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0118] Figure 6 FIG. 1 is a schematic diagram of the structure of an image segmentation device provided by an embodiment of the present invention. Figure 6 As shown, the device includes: an image acquisition module 410 and a segmentation result acquisition module 420; wherein,

[0119] An image acquisition module 410 is used to acquire an image to be segmented;

[0120] The segmentation result acquisition module 420 is used to input the image to be segmented into a pre-trained segmentation model to obtain the segmentation result output by the segmentation model; wherein the segmentation model includes at least one layer of cascaded encoding modules and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component.

[0121] The image segmentation device provided in the embodiment of the present invention can execute the image segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0122] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0123] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as a segmentation model training method and / or an image segmentation method.

[0126] In some embodiments, the segmentation model training method and / or the image segmentation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the segmentation model training method and / or the image segmentation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the segmentation model training method and / or the image segmentation method in any other appropriate manner (e.g., by means of firmware).

[0127] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0129] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0132] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0134] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A segmentation model training method, characterized in that: include: Acquire a sample image for model training and a pixel label of each pixel in the sample image; wherein the pixel includes a target pixel and a background pixel; Normalizing the sample image and performing data enhancement processing on the sample image; wherein the data enhancement processing includes random rotation, random translation, random brightness change and local brightness transformation; The following training process is iteratively performed until the training completion condition is met to obtain the target segmentation model: the sample image is input into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image; Obtaining a first sample prediction result corresponding to a target pixel in the sample prediction results, and generating a first loss function of the segmentation model based on the first sample prediction result and a label of the target pixel; The first loss function of the segmentation model is generated based on the following expression: Among them, P-Dice represents the first loss function; X + Indicates the label of the target pixel; Y + Indicates the prediction result of the first sample; Obtaining a second sample prediction result corresponding to a background pixel in the sample prediction result, determining a candidate background pixel among the background pixels based on a comparison result between a label of the background pixel and the second sample prediction result, and determining a target background pixel among the candidate background pixels based on a preset screening condition; generating a second loss function of the segmentation model based on the label of the target background point and the sample prediction result corresponding to the target background point in the second sample prediction result; The second loss function of the segmentation model is generated based on the following expression: Among them, N-Dice represents the second loss function; X - Indicates the label of the background pixel; Y - Represents the second sample prediction result; Determine a model loss function of the segmentation model based on the first loss function, the second loss function, a preset first loss function weight, and a preset second loss function weight; The model loss function of the segmentation model is generated based on the following expression: Wherein, PNDiceLoss represents the model loss function; α represents the preset weight of the first loss function; 1-α represents the preset weight of the second loss function; Adjusting model parameters of the segmentation model based on the model loss function; Wherein, before inputting the sample image into the segmentation model to be trained, the method further includes: Crop the sample to a preset size to obtain at least two cropped sample images; Wherein, obtaining the sample prediction result of each pixel in the sample image includes: Outputting the initial sample prediction result of the cropped sample image based on an overlapping sliding window method; Performing mean processing on each of the initial sample prediction results based on the number of cropped sample images to obtain a sample prediction result of the sample image; The connected domain post-processing method is adopted to remove the false positives whose pixel values ​​in the sample prediction results are less than the preset threshold, and the results are resampled to the original size and mapped to the original sample image coordinates to obtain the final sample prediction results.

2. The method according to claim 1, characterized in that The segmentation model includes at least one cascaded encoding module and a decoding module at the same level as the encoding module; the decoding module includes a residual attention component; Accordingly, the step of inputting the sample image into the segmentation model to be trained to obtain the sample prediction result of each pixel in the sample image further includes: For any level coding module, a convolution operation is performed on the sample image / the coding features output after downsampling processing of the previous level coding module based on the first preset convolution parameter to obtain the coding features after feature extraction by the current level coding module; For any level decoding module, a convolution operation is performed on the code output after downsampling processing of the previous level encoding module based on the second preset convolution parameters, or on the spliced ​​features after splicing the code features output by the current level encoding module and the image features output after upsampling processing of the previous level decoding module, and feature extraction is performed on the image features after the convolution operation based on the residual attention component to obtain the image features after feature extraction of the current level decoding module.

3. An image segmentation method, characterized in that: include: Obtain the image to be segmented; The image to be segmented is input into a pre-trained segmentation model to obtain a segmentation result output by the segmentation model; wherein the segmentation model is trained based on the segmentation model training method described in any one of claims 1-2.

4. A segmentation model training device, characterized in that: include: A sample image acquisition module, used to acquire sample images for model training and pixel labels of each pixel in the sample images; wherein the pixel includes target pixel and background pixel; The target segmentation model acquisition module includes: a sample prediction result acquisition unit, a model loss function generation unit and a target segmentation model acquisition unit; The sample prediction result obtaining unit is used to perform normalization processing on the sample image and data enhancement processing on the sample image; wherein the data enhancement processing method includes random rotation, random translation, random brightness change and local brightness transformation; iteratively perform the following training process until the training completion condition is met to obtain the target segmentation model: input the sample image into the segmentation model to be trained, and obtain the sample prediction result of each pixel in the sample image; The model loss function generating unit comprises: a first loss function generating subunit, a second loss function generating subunit and a model loss function generating subunit; The first loss function generating subunit is used to obtain a first sample prediction result corresponding to a target pixel in the sample prediction result, and generate a first loss function of the segmentation model based on the first sample prediction result and a label of the target pixel; The first loss function of the segmentation model is generated based on the following expression: Among them, P-Dice represents the first loss function; X + Indicates the label of the target pixel; Y + Indicates the prediction result of the first sample; The second loss function generating subunit includes: a target background pixel generating component and a second loss function generating component; The target background pixel point generation component is used to obtain a second sample prediction result corresponding to the background pixel point in the sample prediction result, determine the candidate background pixel points in the background pixel points based on the comparison result between the label of the background pixel point and the second sample prediction result, and determine the target background pixel points in the candidate background pixel points based on a preset screening condition; The second loss function generating component is used to generate a second loss function of the segmentation model based on the label of the target background point and the sample prediction result corresponding to the target background point in the second sample prediction result; The second loss function of the segmentation model is generated based on the following expression: Among them, N-Dice represents the second loss function; X - Indicates the label of the background pixel; Y - Represents the second sample prediction result; The model loss function generating subunit is used to determine the model loss function of the segmentation model based on the first loss function, the second loss function, a preset first loss function weight, and a preset second loss function weight; The model loss function of the segmentation model is generated based on the following expression: Wherein, PNDiceLoss represents the model loss function; α represents the preset weight of the first loss function; 1-α represents the preset weight of the second loss function; The target segmentation model obtaining unit is used to adjust the model parameters of the segmentation model based on the model loss function; The target segmentation model acquisition module also includes a sample image cropping unit and a sample prediction result acquisition unit; The sample image cropping unit is used to crop the sample into a preset size before inputting the sample image into the segmentation model to be trained, so as to obtain at least two cropped sample images; The sample prediction result obtaining unit comprises: The initial sample prediction result obtaining subunit is used to output the initial sample prediction result of the cropped sample image based on an overlapping sliding window method; perform mean processing on each of the initial sample prediction results based on the number of cropped sample images to obtain the sample prediction result of the sample image; use a connected domain post-processing method to remove false positives in the sample prediction result whose pixel value is less than a preset threshold, resample the result to the original size and map it to the original sample image coordinates to obtain the final sample prediction result.

5. An image segmentation device, characterized in that: include: An image acquisition module, used for acquiring an image to be segmented; A segmentation result acquisition module is used to input the image to be segmented into a pre-trained segmentation model to obtain a segmentation result output by the segmentation model; wherein the segmentation model is trained based on the segmentation model training method described in any one of claims 1-2.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the segmentation model training method described in any one of claims 1-2 and / or the image segmentation method described in claim 3.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the segmentation model training method described in any one of claims 1-2 and / or the image segmentation method described in claim 3 when executed.

Images