Image segmentation method and device, computer equipment and storage medium

Abstract

The invention relates to an image segmentation method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring an original scanning image; the original scanning image comprises a non-interested area and an interested area; inputting the original scanning image into a preset segmentation model, and performing image segmentation on a non-interested area in the original scanning image to obtain a segmentation result of the non-interested area; the preset segmentation model is obtained by training based on an initial bilateral segmentation network, an attention mechanism and a self-distillation mechanism; and based on the segmentation result of the non-interested area, segmenting the interested area in the original scanning image to obtain a segmentation result of the interested area. Since the information of the non-interested area is relatively single, the segmentation of the non-interested area is easier, and the segmentation result is more accurate, so that the segmentation result of the interested area is more accurate, and the segmentation accuracy of the interested area can be improved.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to an image segmentation method, apparatus, computer equipment and storage medium. Background Art

[0002] In the process of performing medical scanning on the object to be measured by using a medical imaging scanning device, in order to reduce the radiation dose received by the object to be measured, the position of the beam limiter in the medical imaging scanning device is usually adjusted according to the part to be measured of the object to be measured, so as to ensure that the rays emitted by the medical imaging scanning device can irradiate the region of interest corresponding to the part to be measured, thereby obtaining a medical scan image including the region of interest.

[0003] Traditionally, when analyzing and processing a medical scan image, it is necessary to crop an image of a region of interest from the medical scan image to analyze and process the region of interest.

[0004] However, the traditional method of obtaining the image of the region of interest is prone to the situation that the cropped image still includes part of the beam limiter area, or part of the region of interest is cropped, that is, the situation of mis-segmentation occurs. Therefore, the accuracy of segmenting medical scan images using traditional methods is low. Summary of the invention

[0005] Based on this, it is necessary to provide an image segmentation method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of medical scan image segmentation in order to address the above technical problems.

[0006] In a first aspect, the present application provides an image segmentation method. The method comprises:

[0007] Acquire an original scanned image; the original scanned image includes a non-interested area and an interested area;

[0008] The original scanned image is input into a preset segmentation model, and the non-interested region in the original scanned image is segmented to obtain the segmentation result of the non-interested region; the preset segmentation model is trained based on the initial bilateral segmentation network, the attention mechanism and the self-distillation mechanism;

[0009] Based on the segmentation result of the non-interest region, the region of interest in the original scanned image is segmented to obtain the segmentation result of the region of interest.

[0010] In one embodiment, the preset segmentation model includes a semantic branch, a spatial branch, a bilateral guided aggregation layer and an output layer; the semantic branch adopts an attention mechanism; the original scanned image is input into the preset segmentation model, and the non-interested area in the original scanned image is segmented to obtain the segmentation result of the non-interested area, including:

[0011] Input the original scanned image into the semantic branch, combine the attention mechanism to extract features, and generate the semantic features of the original scanned image;

[0012] The original scanned image is input into the space branch for feature extraction to generate the spatial features of the original scanned image;

[0013] The semantic features and spatial features of the original scanned image are input into the bilateral guided aggregation layer for feature fusion processing to generate feature fusion results;

[0014] The feature fusion result is input to the output layer, and the non-interested area in the original scanned image is segmented through the output layer to obtain the segmentation result of the non-interested area.

[0015] In one embodiment, the semantic branch includes a first feature extraction module and a second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; the original scanned image is input into the semantic branch, and feature extraction is performed in combination with the attention mechanism to generate semantic features of the original scanned image, including:

[0016] Inputting the original scanned image into a first feature extraction module to extract features, thereby obtaining a first feature map of the original scanned image;

[0017] The first feature map is input into the context embedding module to extract attention features, and a second feature map corresponding to the first feature map is obtained; the second feature map is used to characterize the attention weights of the non-interest area and the interest area;

[0018] The first feature map is multiplied by the second feature map to generate the semantic features of the original scanned image.

[0019] In one embodiment, the method further comprises:

[0020] Obtain the sample scanned image and the labeled segmentation results of the non-interested area corresponding to the sample scanned image;

[0021] The sample scanned image is input into the initial bilateral segmentation network, and the non-interested region in the sample scanned image is segmented to obtain the predicted segmentation result of the non-interested region in the sample scanned image;

[0022] Based on the predicted segmentation results and the labeled segmentation results, the self-distillation mechanism is used to train the initial bilateral segmentation network to obtain the preset segmentation model.

[0023] In one embodiment, the initial bilateral segmentation network includes an initial semantic branch, the initial semantic branch includes at least one first feature extraction module and at least one second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; based on the predicted segmentation result and the labeled segmentation result, the initial bilateral segmentation network is trained using a self-distillation mechanism to obtain a preset segmentation model, including:

[0024] Inputting the predicted segmentation result and the labeled segmentation result into a first preset loss function to obtain a value of the first preset loss function;

[0025] Based on the value of the first preset loss function and the preset training conditions, the initial bilateral segmentation network is trained to obtain an intermediate segmentation model that meets the preset training conditions;

[0026] The intermediate segmentation model is trained according to the intermediate feature maps output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model to obtain a preset segmentation model.

[0027] In one embodiment, the intermediate segmentation model is trained according to the intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model to obtain a preset segmentation model, including:

[0028] Acquire each intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model;

[0029] Select any two target intermediate feature maps from each intermediate feature map, input the any two target intermediate feature maps into a second preset loss function, obtain multiple values ​​of the second preset loss function, and update the value of the first preset loss function according to the predicted segmentation result and the labeled segmentation result of the intermediate segmentation model;

[0030] Based on the values ​​of the plurality of second preset loss functions and the updated value of the first preset loss function, the intermediate segmentation model is trained to obtain a preset segmentation model.

[0031] In one embodiment, if the segmentation result of the non-interest region is a target probability map corresponding to the non-interest region, then based on the segmentation result of the non-interest region, the region of interest in the original scanned image is segmented to obtain the segmentation result of the region of interest, including:

[0032] Based on the target probability map corresponding to the non-interest region, determine the binary segmentation map corresponding to the non-interest region;

[0033] Based on the binary segmentation map, the region of interest in the original scanned image is segmented to obtain the segmentation result of the region of interest.

[0034] In a second aspect, the present application also provides an image segmentation device. The device comprises:

[0035] A first acquisition module is used to acquire an original scanned image; the original scanned image includes a non-interest area and an interest area;

[0036] The first segmentation module is used to input the original scanned image into a preset segmentation model, perform image segmentation on the non-interested region in the original scanned image, and obtain the segmentation result of the non-interested region; the preset segmentation model is trained based on the initial bilateral segmentation network, the attention mechanism and the self-distillation mechanism;

[0037] The second segmentation module is used to segment the region of interest in the original scanned image based on the segmentation result of the non-region of interest to obtain the segmentation result of the region of interest.

[0038] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image segmentation method in the first aspect when executing the computer program.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the image segmentation method in the first aspect when executed by a processor.

[0040] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the image segmentation method in the first aspect are implemented.

[0041] The above-mentioned image segmentation method, device, computer device, storage medium and computer program product, the computer device obtains an original scanned image including a non-interest region and an interest region; and inputs the original scanned image into a preset segmentation model, performs image segmentation on the non-interest region in the original scanned image, and obtains a segmentation result of the non-interest region; then, based on the segmentation result of the non-interest region, the interest region in the original scanned image is segmented to obtain a segmentation result of the interest region; wherein the preset segmentation model is obtained by training based on an initial bilateral segmentation network, an attention mechanism and a self-distillation mechanism; that is, the image segmentation method in this embodiment, for an original scanned image of a non-interest region with single information and an interest region with diversified information, when segmenting and cropping the interest region in the original scanned image, first performs image segmentation on the non-interest region in the original scanned image, and then, based on the image segmentation result of the non-interest region, performs image segmentation on the interest region in the original scanned image, thereby obtaining a segmentation result of the interest region; since the information of the non-interest region is relatively single, the segmentation of the non-interest region is easier, and the segmentation result is more accurate, thereby making the segmentation result of the interest region more accurate, which can improve the segmentation accuracy of the interest region. In addition, the improved bilateral segmentation model is used in the present application to perform image segmentation processing on the non-interested regions in the original scanned image. Compared with the traditional image segmentation algorithm, it can improve the image segmentation accuracy and further improve the segmentation accuracy of the region of interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A diagram showing an application environment of an image segmentation method in an embodiment;

[0043] Figure 2 is a schematic flow chart of an image segmentation method in one embodiment;

[0044] Figure 3 is a flow chart of an image segmentation method in another embodiment;

[0045] Figure 4 A schematic diagram of a network structure of a space branch in an embodiment;

[0046] Figure 5 A schematic diagram of a network structure of a bilaterally guided aggregation layer in an embodiment;

[0047] Figure 6 is a flow chart of an image segmentation method in another embodiment;

[0048] Figure 7 A schematic diagram of a network structure of a semantic branch in one embodiment;

[0049] Figure 8A schematic diagram of a network structure of a context embedding module using a self-attention mechanism in one embodiment;

[0050] Fig. 9 is a flow chart of an image segmentation method in another embodiment;

[0051] FIG10( a ) is a schematic diagram of a sample scanned image and a corresponding annotated segmentation result in one embodiment;

[0052] FIG10( b ) is a schematic diagram of a sample scanned image and a corresponding annotated segmentation result in another embodiment;

[0053] FIG10( c ) is a schematic diagram of a sample scanned image and a corresponding annotated segmentation result in another embodiment;

[0054] FIG. 11( a ) is a schematic diagram of test results of a preset segmentation model in one embodiment;

[0055] FIG11( b ) is a schematic diagram of test results of a preset segmentation model in another embodiment;

[0056] Fig.12 is a flow chart of an image segmentation method in another embodiment;

[0057] Fig.13 is a flow chart of an image segmentation method in another embodiment;

[0058] Fig.14 A schematic diagram of a network structure of a preset segmentation model in an embodiment;

[0059] Fig.15 is a structural block diagram of an image segmentation device in one embodiment;

[0060] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] Usually, when performing medical imaging scans, doctors will appropriately adjust the position of the beam limiter (also called a collimator) according to the target area to be scanned, such as the part to be tested, the area of ​​interest on the part to be tested, etc., to ensure that the X-rays can just irradiate the target area of ​​interest to the doctor, while reducing the radiation dose to the non-interested areas outside the area of ​​interest received by the object to be tested. Therefore, the medical scan image obtained by scanning from the flat-panel detector usually includes two parts, namely the beam limiter area and the area of ​​interest, wherein the beam limiter, as a metal lead leaf, will block the X-rays irradiated to the object to be tested. In the medical scan image, the beam limiter area is usually shown as a surrounding black area, and the middle part is the area of ​​interest obtained by the shooting.

[0063] For medical scan images, it is usually necessary to crop the region of interest from the medical scan image to obtain the region of interest image, so as to perform image analysis and processing on the region of interest image later. In the related art, an image segmentation algorithm is used to segment the region of interest of the medical scan image to obtain the region of interest image; however, since the objects photographed in the region of interest are relatively complex, they may include human bones or tissues, other surgical instruments, or even nothing. For example, when testing equipment, sometimes nothing can be placed, which can easily affect the stability of the image segmentation algorithm and even cause mis-segmentation, such as cropping out part of the region of interest, or including part of the beam limiter area, etc., resulting in low image segmentation accuracy.

[0064] If part of the ROI is cropped out of the cropped ROI image, the part of interest or tissue will be lost, resulting in inaccurate image analysis and processing results. If part of the beam limiter area is included in the cropped ROI image, the image contrast will be affected, and the beam limiter area will be brought into the subsequent image analysis and processing process, which may easily lead to poor post-processing image quality, such as white bars appearing in the image or even being unreadable.

[0065] Based on this, the embodiment of the present application proposes an image segmentation method. By pre-segmenting the beam limiter area, and then based on the segmented beam limiter area, the region of interest is cropped out from the medical scan image; because the area occluded by the beam limiter is relatively simple and will not be interfered by other objects, therefore, the beam limiter area is pre-segmented, and then the region of interest is cropped out excluding the beam limiter area. Compared with the traditional image segmentation algorithm, the interference will be relatively small, and the image segmentation result will be more stable and accurate.

[0066] The image segmentation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the medical image scanning device 102 communicates with the computer device 104 through the network. The medical image scanning device 102 includes but is not limited to electronic computer tomography (Computed Tomography, CT) equipment, digital radiography (digital radiography, DR) equipment, etc. The computer device 104 can be a terminal or a server; the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers, etc., and the server can be implemented by an independent server or a server cluster composed of multiple servers.

[0067] The image segmentation method can be applied to a medical imaging scanning device, that is, the original medical scanning image acquired is segmented into regions of interest by the medical imaging scanning device to obtain a region of interest image, and the obtained region of interest image is sent to a computer device; the image segmentation method can also be applied to a computer device that is communicatively connected to the medical imaging scanning device, and the computer device can obtain the original medical scanning image from the medical imaging scanning device, and perform image segmentation on the region of interest in the original medical scanning image to obtain a region of interest image.

[0068] In one embodiment, Figure 2 As shown, an image segmentation method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0069] Step 201, obtaining an original scanned image.

[0070] The original scanned image includes a non-interest region and an interest region; illustratively, the non-interest region may include a beam limiter region.

[0071] Exemplarily, the computer device can receive the original scanned image sent by the medical imaging scanning device, such as receiving the original scanned image sent by the medical imaging scanning device after the scanning is completed, or the computer device sends an image acquisition request to the medical imaging scanning device in response to a trigger operation of the user, and the medical imaging scanning device responds to the image acquisition request and sends the original scanned image corresponding to the image acquisition request to the computer device, so that the computer device acquires the original scanned image.

[0072] Exemplarily, the computer device may also obtain the original scanned image from a storage device that is communicatively connected to the medical imaging scanning device, and the storage device may include a local storage device or a cloud storage device equipped with a picture archiving and communication system (PACS), etc.; the storage device may also include a local storage device or a cloud storage device that is not equipped with a PACS system, etc.; the computer device may obtain the original scanned image from the storage device that is communicatively connected to the medical imaging scanning device based on a user's trigger operation.

[0073] Exemplarily, the computer device may also obtain the original scanned image from an internal storage component of the computer device; the embodiment of the present application does not specifically limit the method for obtaining the original scanned image.

[0074] Step 202: Input the original scanned image into a preset segmentation model, perform image segmentation on the non-interested region in the original scanned image, and obtain a segmentation result of the non-interested region.

[0075] Among them, the preset segmentation model is trained based on the initial bilateral segmentation network, attention mechanism and self-distillation mechanism.

[0076] Exemplarily, the initial bilateral segmentation network can be a BiSeNet network, or an improved network based on the BiSeNet network, such as a BiSeNet V2 network, etc.; the initial bilateral segmentation network includes two feature extraction branches, one is a detail branch (Detail Branch), which can also be called a spatial branch, and the other is a semantic branch (Semantic Branch); wherein the detail branch is used to extract the microscopic features of the image, focusing on the image details, and the semantic branch is used to extract the macroscopic features of the image, focusing on the high-level semantic context features. In addition, for the image features extracted by the detail branch and the image features extracted by the semantic branch, a guided aggregation layer (Guided Aggregation Layer) is also provided, which can also be called a bilateral guided aggregation layer, that is, the image features extracted by the detail branch and the image features extracted by the semantic branch are feature fused; finally, the image segmentation result is output after operations such as upsampling.

[0077] In this embodiment, an attention mechanism and a self-distillation mechanism are added to the traditional initial bilateral segmentation network to improve the segmentation accuracy of the preset segmentation model obtained by training and improve the accuracy of the image segmentation results. Exemplarily, for the semantic branch in the initial bilateral segmentation network, it can include multiple different types of feature extraction modules, such as: stem block, gather expansion block (GE Block), context embedding block (CE Block), etc.; the number of feature extraction modules of each type can be one or more, and feature extraction modules of the same type can be cascaded or cross-cascaded with feature extraction modules of other types.

[0078] For the attention mechanism, an attention mechanism can be added to each feature extraction module in the semantic branch; for example, an attention mechanism can be added to the context embedding module CE Block in the semantic branch; an attention mechanism can also be added to other types of feature extraction modules; in addition, an attention mechanism can be added to one feature extraction module, or to multiple feature extraction modules; the embodiments of the present application do not make specific limitations on this.

[0079] For the self-distillation mechanism, for each feature extraction module in the semantic branch, the loss value between each feature extraction module can be calculated, and the loss value between each feature extraction module can be added to the loss function of the initial bilateral segmentation network to assist network training; illustratively, for each feature extraction module in the semantic branch, the loss value between two adjacent feature extraction modules can be calculated, or the loss value between multiple adjacent feature extraction modules can be calculated, or the loss value between multiple non-adjacent feature extraction modules can be calculated, etc.; the embodiment of the present application does not make specific limitations on this.

[0080] That is, in this embodiment, the network model of the initial bilateral segmentation network is improved based on the attention mechanism and the self-distillation mechanism to obtain an improved target bilateral segmentation network, and the improved target bilateral segmentation network is trained based on the sample scan image and the gold standard mask image corresponding to the sample scan image to obtain a preset segmentation model; wherein the gold standard mask image corresponding to the sample scan image is used to characterize the non-interest area in the sample scan image.

[0081] Furthermore, when a preset segmentation model is obtained through training, image segmentation processing can be performed on the non-interested region in the original scanned image based on the preset segmentation model to obtain a segmentation result of the non-interested region. The segmentation result of the non-interested region can be a mask image of the non-interested region corresponding to the original scanned image, such as 0 for the non-interested region and 1 for the interested region.

[0082] Optionally, after obtaining the segmentation results of the non-interest area output by the preset segmentation model, the segmentation results of the non-interest area output by the preset segmentation model can also be post-processed to obtain high-precision and high-quality segmentation results of the non-interest area; the post-processing operations here may include basic image processing operations, such as smoothing, denoising, corrosion, opening and closing operations, etc.

[0083] Step 203 , based on the segmentation result of the non-interested region, segment the interested region in the original scanned image to obtain the segmentation result of the interested region.

[0084] After obtaining the segmentation result of the non-interested region, the non-interested region can be removed from the original scanned image based on the segmentation result of the non-interested region to obtain the segmentation result of the region of interest. Exemplarily, the original scanned image can be subtracted from the segmentation result of the non-interested region to obtain the segmentation result of the region of interest after removing the non-interested region.

[0085] In the above-mentioned image segmentation method, a computer device obtains an original scanned image including a non-interest region and an interest region; and inputs the original scanned image into a preset segmentation model, performs image segmentation on the non-interest region in the original scanned image, and obtains a segmentation result of the non-interest region; then, based on the segmentation result of the non-interest region, the interest region in the original scanned image is segmented to obtain a segmentation result of the interest region; wherein the preset segmentation model is obtained by training based on an initial bilateral segmentation network, an attention mechanism, and a self-distillation mechanism; that is, in the image segmentation method in this embodiment, for an original scanned image having a non-interest region with single information and an interest region with diversified information, when segmenting and cropping the interest region in the original scanned image, the image segmentation is first performed on the non-interest region in the original scanned image, and then, based on the image segmentation result of the non-interest region, the interest region in the original scanned image is segmented to obtain a segmentation result of the interest region; since the information of the non-interest region is relatively single, it is easier to segment the non-interest region, and the segmentation result is more accurate, thereby making the segmentation result of the interest region more accurate, and the segmentation accuracy of the interest region can be improved. In addition, the improved bilateral segmentation model is used in the present application to perform image segmentation processing on the non-interested regions in the original scanned image. Compared with the traditional image segmentation algorithm, it can improve the image segmentation accuracy and further improve the segmentation accuracy of the region of interest.

[0086] In one embodiment, the preset segmentation model includes a semantic branch, a spatial branch, a bilateral guided aggregation layer and an output layer; wherein the semantic branch adopts an attention mechanism; illustratively, the attention mechanism can be adopted for at least one feature extraction module in each feature extraction module in the semantic branch; based on this, on the basis of the above embodiment, Figure 3 As shown, the above step 202 includes:

[0087] Step 301, input the original scanned image into the semantic branch, combine the attention mechanism to perform feature extraction, and generate the semantic features of the original scanned image.

[0088] Among them, the semantic branch may include multiple different types of feature extraction modules, the number of feature extraction modules of each type is one or more, and the multiple feature extraction modules are cascaded, wherein at least one feature extraction module adopts an attention mechanism.

[0089] When the original scanned image is input into the semantic branch for semantic feature extraction, the feature extraction of the original scanned image can be performed in turn through the multiple feature extraction modules cascaded in the semantic branch, so as to generate the semantic features of the original scanned image; for example: the original scanned image is input into the first feature extraction module of the semantic branch for feature extraction to obtain the first feature information; then, the first feature information is input into the second feature extraction module for feature extraction to obtain the second feature information; then, the second feature information is input into the third feature extraction module for feature extraction to obtain the third feature information; and so on, until the last feature extraction module in the semantic branch outputs the feature information, and the feature information output by the last feature extraction module can be used as the semantic feature of the original scanned image.

[0090] Step 302: Input the original scanned image into the space branch for feature extraction to generate the spatial features of the original scanned image.

[0091] The spatial branch can also be called a detail branch. Figure 4 As shown, the spatial branch may also include multiple feature extraction modules, each of which may include a convolution layer (conv), a batch normalization layer (bn), and an activation function layer (such as a relu activation function or other activation functions). The original scanned image is input into the spatial branch, and the spatial detail information of the original scanned image is extracted by each feature extraction module in the spatial branch to generate the spatial features of the original scanned image.

[0092] Step 303: input the semantic features and spatial features of the original scanned image into the bilateral guided aggregation layer for feature fusion processing to generate a feature fusion result.

[0093] Among them, the implementation method of the bilateral guided aggregation layer can be: for the semantic features output by the semantic branch, multiple processing branches are used to process the semantic features respectively, and multiple first processing results corresponding to the semantic branches are obtained; for the spatial features output by the spatial branch, multiple processing branches are used to process the spatial features respectively, and multiple second processing results corresponding to the spatial features are obtained; then, each first processing result is cross-fused with each second processing result to obtain multiple intermediate fusion results, and finally, the multiple intermediate fusion results are fused to obtain the final feature fusion result.

[0094] refer to Figure 5 As shown, the semantic branch may correspond to two processing branches, such as a first processing branch and a second processing branch, wherein the first processing branch may include a convolution + batch normalization layer (conv+bn), an upsampling layer (upsampling), and an activation function layer (sigmoid); exemplarily, the upsampling layer (upsampling) may be an upsampling layer implemented based on bilinear interpolation; the second processing branch may include a depth-separable convolution + batch normalization layer (dwconv+bn), a convolution + batch normalization layer (conv+bn), a convolution layer (conv), and an activation function layer (sigmoid); the spatial branch may correspond to two processing branches, such as a third processing branch and a fourth processing branch, wherein the third processing branch may include a depth-separable convolution + batch normalization layer (dwconv+bn) and a convolution layer (conv); the fourth processing branch may include a convolution + batch normalization layer (conv+bn) and an average pooling layer (avg pool).

[0095] Exemplarily, the result of processing the semantic features of the original scanned image output by the semantic branch by the first processing branch and the result of processing the spatial features of the original scanned image output by the spatial branch by the third processing branch can be multiplied (mul) to obtain a first intermediate fusion result; and the result of processing the semantic features of the original scanned image output by the semantic branch by the second processing branch and the result of processing the spatial features of the original scanned image output by the spatial branch by the fourth processing branch can be multiplied (mul) to obtain a second intermediate fusion result; then, the first intermediate fusion result and the second intermediate fusion result are added (sum) to obtain the final feature fusion result.

[0096] It should be noted that Figure 5 The bilateral guided aggregation layer shown is only used as an example for illustration and is not used to limit the structure of the bilateral guided aggregation layer, nor is it used to limit the structures of the processing branches corresponding to the semantic branches and the spatial branches respectively.

[0097] Step 304, input the feature fusion result to the output layer, and perform image segmentation on the non-interested region in the original scanned image through the output layer to obtain the segmentation result of the non-interested region.

[0098] The output layer is used to convert the feature information into image information to obtain the final image segmentation result. The output layer may include processing operations such as upsampling and normalization, so as to decode and restore the feature information to obtain image information.

[0099] Exemplarily, after the feature fusion result is input to the output layer for processing, a classification probability map of the image pixels can be obtained; the classification probability map includes the classification probability corresponding to each pixel, such as the probability of the pixel belonging to a pixel in a non-interest region. When the probability that the pixel belongs to a non-interest region is greater than or equal to a preset probability threshold, the pixel can be considered to belong to a pixel in the non-interest region. At this time, the pixel value of the pixel can be modified to the pixel value corresponding to the pixel in the non-interest region, such as 0; when the probability that the pixel belongs to a non-interest region is less than the preset probability threshold, the pixel can be considered to belong to a pixel in the interest region. At this time, the pixel value of the pixel can be modified to the pixel value corresponding to the pixel in the interest region, such as 1; based on this, by comparing the classification probability of each pixel in the classification probability map with the preset probability threshold, the segmentation result of the non-interest region can be obtained, and the segmentation result can be a mask image of the non-interest region corresponding to the original scanned image.

[0100] In this embodiment, the preset segmentation model includes a semantic branch, a spatial branch, a bilateral guided aggregation layer and an output layer; wherein the semantic branch adopts an attention mechanism; the computer device generates semantic features of the original scanned image by inputting the original scanned image into the semantic branch and extracting features in combination with the attention mechanism; and the original scanned image is input into the spatial branch for feature extraction to generate spatial features of the original scanned image; then, the semantic features and spatial features of the original scanned image are input into the bilateral guided aggregation layer for feature fusion processing to generate feature fusion results; further, the feature fusion results are input into the output layer, and image segmentation is performed on the non-interested areas in the original scanned image through the output layer to obtain the segmentation results of the non-interested areas. Using the preset segmentation model structure provided in this embodiment and performing image segmentation on the original scanned image based on the preset segmentation model structure can improve the accuracy of the image segmentation results.

[0101] In one embodiment, the above-mentioned semantic branch may include a first feature extraction module and a second feature extraction module, wherein the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism. Optionally, the first feature extraction module may include at least one type of feature extraction module, such as at least one of StemBlock, GE Block and CE Block, and the number of different types of feature extraction modules may include one or more; the structure and connection order of the first feature extraction module are not specifically limited in the embodiment of the present application. In addition, for the second feature extraction module, it may include a context embedding module, i.e., CE Block, and may also include a context embedding module and other types of feature extraction modules. The structure and connection order of the second feature extraction module are not specifically limited in the embodiment of the present application.

[0102] Based on this, on the basis of the above embodiments, Figure 6 As shown, the above step 301 may include:

[0103] Step 601: Input the original scanned image into a first feature extraction module to perform feature extraction to obtain a first feature map of the original scanned image.

[0104] Exemplarily, the computer device may input the original scanned image into the first feature extraction module, and perform feature extraction on the original scanned image through each feature extraction module in the first feature extraction module to obtain a first feature map of the original scanned image.

[0105] Step 602: Input the first feature map into a context embedding module to extract attention features, and obtain a second feature map corresponding to the first feature map.

[0106] Among them, the second feature map is used to characterize the attention weights of non-interest regions and interest regions.

[0107] Exemplarily, if the context embedding module is not the first feature extraction module in the second feature extraction module, the first feature map can be first input into each feature extraction module before the context embedding module for feature extraction to obtain an intermediate feature map, and then the intermediate feature map is input into the context embedding module for attention feature extraction to obtain a second feature map corresponding to the first feature map.

[0108] Exemplarily, if the context embedding module is the first feature extraction module in the second feature extraction module, the first feature map can be directly input into the context embedding module for attention feature extraction to obtain a second feature map corresponding to the first feature map.

[0109] Step 603: multiply the first feature map and the second feature map to generate semantic features of the original scanned image.

[0110] Exemplarily, if the context embedding module is the last feature extraction module in the second feature extraction module, the first feature map and the second feature map may be multiplied to directly generate the semantic features of the original scanned image.

[0111] Exemplarily, if the context embedding module is not the last feature extraction module in the second feature extraction module, the first feature map can be multiplied by the second feature map to obtain a multiplied feature map, and then the multiplied feature map can be input into other feature extraction modules after the context embedding module in the second feature extraction module for feature extraction, thereby generating semantic features of the original scanned image.

[0112] For example, reference Figure 7 As shown, the semantic branch may include a stem module (StemBlock), an aggregation expansion module 1 (GE Block1), an aggregation expansion module 2 (GE Block2), and a context embedding module (CE Block) which are cascaded in sequence; wherein, the Stem Block is used to initialize the downsampling layer, pre-extract the underlying semantic features, and extract features by superimposing multiple features. The GE Block is used to extract high-level semantic features, and can use deep separable convolution (dwconv) to reduce network parameters and improve the network's reasoning speed; at the same time, features can be extracted by adding multiple features. The CE Block is used to obtain global information, distinguish the weights of different features, and add the global information to the features of the semantic branch to further improve the semantic branch's extraction of feature information. The attention mechanism is used in the CE Block, refer to Figure 8 As shown, the CE Block using the attention mechanism may include a global pooling layer, a convolution layer, and an activation function layer. After further feature extraction of the feature information output by GE Block2 based on the attention mechanism, it is multiplied with the feature information output by GE Block2 to generate the semantic features of the original scanned image.

[0113] In this embodiment, the semantic branch may include a first feature extraction module and a second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; based on this, the computer device obtains a first feature map of the original scanned image by inputting the original scanned image into the first feature extraction module for feature extraction; and inputs the first feature map into the context embedding module for attention feature extraction to obtain a second feature map corresponding to the first feature map; then, the first feature map is multiplied by the second feature map to generate the semantic features of the original scanned image; wherein the second feature map is used to characterize the attention weights of non-interest areas and interest areas; by adopting the semantic feature extraction method provided in this embodiment, by adopting the attention mechanism in the context embedding module, it is possible to improve to higher-dimensional semantic features and improve the extraction accuracy of semantic features.

[0114] Fig. 9 FIG. 1 is a flow chart of an image segmentation method in another embodiment. This embodiment involves an optional implementation process of training a preset segmentation model. Based on the above embodiment, Fig. 9 As shown, the above method also includes:

[0115] Step 901: Obtain a sample scan image and a labeled segmentation result of a non-interested region corresponding to the sample scan image.

[0116] The annotated segmentation result of the non-interest region corresponding to the sample scanned image may be a mask image or a binary image of the non-interest region manually annotated by a user.

[0117] Exemplarily, a computer device may obtain historical scan images from a medical imaging scanning device as sample scan images for training a preset segmentation model; the sample scan images include non-interest areas blocked by a beam limiter and areas of interest corresponding to the scanning position.

[0118] refer to Figure 10(a)-10(c) As shown, the left picture is a sample scan image, including the surrounding non-interest area, i.e., the beam limiter area, and the middle area of ​​interest. The area of ​​interest may not contain any object to be tested, but may also include the object to be tested and other objects, such as surgical instruments, accessories on the object to be tested, etc. The right picture is the labeled segmentation result of the non-interest area corresponding to the sample scan image, and the labeled segmentation result can be a binary template image, in which the white area is the marked non-interest area, and the black area is the area of ​​interest.

[0119] Step 902: Input the sample scanned image into the initial bilateral segmentation network, perform image segmentation on the non-interested region in the sample scanned image, and obtain a predicted segmentation result of the non-interested region in the sample scanned image.

[0120] Exemplarily, the initial bilateral segmentation network may include an initial semantic branch and an initial spatial branch, and the initial semantic branch may include at least one first feature extraction module and at least one second feature extraction module, wherein the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; optionally, the first feature extraction module may include at least one of the feature extraction modules of the types Stem Block, GE Block, CE Block, etc., and the number of each type of feature extraction module may be one or more, and each type of feature extraction module may be cascaded. In addition, in addition to the context embedding module, the second feature extraction module may also include other types of feature extraction modules, which are not specifically limited in the embodiments of the present application.

[0121] When training the model, each sample scanned image is input into the initial bilateral segmentation network respectively, and the feature of each sample scanned image is extracted through the initial semantic branch in the initial bilateral segmentation network to obtain the semantic feature of each sample scanned image; and the feature of each sample scanned image is extracted through the initial spatial branch in the initial bilateral segmentation network to obtain the spatial feature of each sample scanned image; then, the semantic feature and spatial feature of each sample scanned image are fused through the initial bilateral guided aggregation layer, and the fused feature is input into the output layer for processing to obtain the predicted segmentation result of the non-interest area in each sample scanned image.

[0122] Step 903: Based on the predicted segmentation results and the labeled segmentation results, the initial bilateral segmentation network is trained using a self-distillation mechanism to obtain a preset segmentation model.

[0123] Among them, the self-distillation mechanism refers to the use of a preset loss function to calculate the loss value between the feature maps output by different feature extraction modules in the semantic branch, and the training of the entire network based on the loss value of each loss value and the loss value of the main loss function of the network; the self-distillation mechanism enables the underlying semantics and high-level semantics to learn from each other, enriching the learning effect of the overall image semantics.

[0124] Exemplarily, the predicted segmentation results and the labeled segmentation results can be input into the main loss function of the network to calculate the loss value of the main loss function; and the feature maps output by each feature extraction module in the semantic branch are input into the auxiliary loss function in pairs to obtain the loss value between the two feature extraction modules; then, based on the loss value of the main loss function and the loss value between the two feature extraction modules in the semantic branch, the initial bilateral segmentation network is trained until the preset training cutoff condition is met, thereby obtaining the preset segmentation model.

[0125] For the preset segmentation model after training, the trained preset segmentation model can be tested by using a test image, and the image segmentation result of the non-interested area corresponding to the test image is output; for example, refer to Figure 11(a) and 11(b) As shown, FIG11(a) is the test result corresponding to the normal case, that is, the beam limiter area is a regular shape, and the non-interest area wraps the interest area, and FIG11(b) is the test result corresponding to the abnormal case, that is, the beam limiter area is an irregular shape. It can be seen that the preset segmentation model trained in this embodiment can accurately segment the non-interest area in the scanned image, and will not segment out the pixels in the interest area that are close to the pixel values ​​of the non-sensory area.

[0126] In this embodiment, by obtaining the sample scan image and the annotated segmentation results of the non-interested area corresponding to the sample scan image; and inputting the sample scan image into the initial bilateral segmentation network, the non-interested area in the sample scan image is segmented to obtain the predicted segmentation results of the non-interested area in the sample scan image; then, based on the predicted segmentation results and the annotated segmentation results, the initial bilateral segmentation network is trained using a self-distillation mechanism to obtain a preset segmentation model. In this embodiment, the self-distillation mechanism is used in the process of training the preset segmentation model. Since the self-distillation mechanism can focus on the mutual learning between the underlying semantics and the high-level semantics, it can enrich the learning of the overall image semantics, thereby improving the training effect, improving the precision and robustness of the preset segmentation model, and improving the accuracy of the preset segmentation model.

[0127] In one embodiment, the self-distillation mechanism can also be used after a certain stage of network training, in order to further improve the effect of feature extraction. Because after a certain stage of training, most of the semantic information of the network has been learned, and continuing training at this time will not be very helpful in improving the effect. At this time, adding the self-distillation mechanism can further improve the training effect.

[0128] Fig.12 is a flow chart of an image segmentation method in another embodiment. This embodiment involves an optional implementation process of training the initial bilateral segmentation network based on the predicted segmentation results and the labeled segmentation results using a self-distillation mechanism to obtain a preset segmentation model. On the basis of the above embodiment, Fig.12 As shown, the above step 903 may include:

[0129] Step 1201: input the predicted segmentation result and the labeled segmentation result into a first preset loss function to obtain the value of the first preset loss function.

[0130] Among them, the first preset loss function can be the main loss function of the network, that is, by comparing the predicted segmentation result with the labeled segmentation result of the gold standard, the loss value between the predicted segmentation result and the labeled segmentation result is obtained.

[0131] Step 1202: Based on the value of the first preset loss function and the preset training conditions, the initial bilateral segmentation network is trained to obtain an intermediate segmentation model that meets the preset training conditions.

[0132] Among them, the preset training condition can be the training cutoff condition of the first stage of network training, such as: the value of the first preset loss function is less than or equal to a certain loss function threshold, the number of training iterations, the segmentation effect of the predicted segmentation result output by the network, etc.

[0133] Exemplarily, based on the value of the first preset loss function, the network parameters in the initial bilateral segmentation network are optimized and adjusted, and based on the intermediate bilateral segmentation network after parameter optimization, the sample scanned image is re-segmented to obtain a new predicted segmentation result corresponding to the sample scanned image; and the new predicted segmentation result and the labeled segmentation result are input into the first preset loss function to obtain the value of the first preset loss function; and based on the value of the first preset loss function, the intermediate bilateral segmentation network is continuously optimized, and this is iterated until the preset training conditions are met; and the intermediate segmentation model after the first stage of training is obtained.

[0134] Step 1203: Train the intermediate segmentation model according to the intermediate feature maps output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model to obtain a preset segmentation model.

[0135] For example, after the first stage of network training is completed, the self-distillation mechanism is introduced to enter the second stage of network training. That is, in the second stage of network training, the self-distillation mechanism is used to continue model training on the intermediate segmentation model obtained by the first stage training, and finally the preset segmentation model is trained.

[0136] In this embodiment, the self-distillation mechanism refers to the semantic feature learning between the feature extraction models of the semantic branches in the intermediate segmentation model, so that the underlying semantics and the high-level semantics can learn from each other, enrich the semantic feature learning, and improve the model training effect.

[0137] Exemplarily, the process of using the self-distillation mechanism to perform the second-stage model training may include:

[0138] Step a: obtaining intermediate feature maps output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model.

[0139] Step b: select any two target intermediate feature maps from each intermediate feature map, input any two target intermediate feature maps into the second preset loss function, obtain multiple values ​​of the second preset loss function, and update the value of the first preset loss function according to the predicted segmentation result and the labeled segmentation result of the intermediate segmentation model.

[0140] Step c: training the intermediate segmentation model based on the values ​​of the multiple second preset loss functions and the updated value of the first preset loss function to obtain the preset segmentation model.

[0141] That is to say, in each iterative training process of the second stage, it is necessary not only to determine the value of the main loss function of the network based on the predicted segmentation result and the labeled segmentation result of the intermediate segmentation model, that is, to obtain the loss value between the predicted segmentation result and the labeled segmentation result, but also to determine the loss value between the feature maps output by each feature extraction module in the semantic branch of the intermediate segmentation model; based on the loss value between the predicted segmentation result and the labeled segmentation result, as well as the loss value between the feature maps output by each feature extraction module, the intermediate segmentation model is iteratively trained to obtain the preset segmentation model.

[0142] Exemplarily, when calculating the loss values ​​between the intermediate feature maps corresponding to each feature extraction module of the semantic branch in step b, the loss values ​​between the intermediate feature maps corresponding to two feature extraction modules can be arbitrarily calculated, or the loss values ​​between the intermediate feature maps corresponding to two adjacent feature extraction modules can be calculated, thereby obtaining the loss values ​​between multiple intermediate feature maps; the embodiment of the present application does not make specific limitations on this.

[0143] In this embodiment, the predicted segmentation result and the labeled segmentation result are input into the first preset loss function to obtain the value of the first preset loss function; based on the value of the first preset loss function and the preset training conditions, the initial bilateral segmentation network is trained to obtain an intermediate segmentation model that meets the preset training conditions; then, according to the intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model, the intermediate segmentation model is trained to obtain the preset segmentation model. In this embodiment, the network is first trained and learned in the first stage by using the main loss function of the network. After the training reaches a certain stage, since most of the semantic information has been learned, if the training continues, it will not be of much help to improve the segmentation effect of the model; at this time, the self-distillation mechanism is used to perform the second stage of training and learning. By learning the features between different feature extraction module levels in the semantic branch, the underlying semantic information and the high-level semantic information can learn from each other, thereby improving the overall semantic learning effect, and then improving the training effect of the network model, so as to improve the segmentation accuracy of the network model.

[0144] In one embodiment, for the segmentation result of the non-interest region obtained by the preset segmentation model in the above embodiments, it can be a mask image of the non-interest region corresponding to the original scanned image, or it can be a target probability map of the non-interest region corresponding to the original scanned image; if the segmentation result of the non-interest region is a target probability map corresponding to the non-interest region, then on the basis of the above embodiments, Fig.13 As shown, the above step 203 may also include:

[0145] Step 1301, based on the target probability map corresponding to the non-interest region, determine the binary segmentation map corresponding to the non-interest region.

[0146] Exemplarily, the computer device may first obtain a preset probability threshold, which may be used to characterize the probability that the pixel value belongs to a non-interested region; based on this, for the probability value corresponding to each pixel point in the target probability map, when the probability value corresponding to the pixel point is greater than or equal to the preset probability threshold, the pixel point may be considered to belong to a non-interested region; when the probability value corresponding to the pixel point is less than the preset probability threshold, the pixel point may be considered to belong to a region of interest; further, when it is determined that the pixel point belongs to a non-interested region, the pixel value corresponding to the pixel point may be set to the pixel value corresponding to the non-interested region; when it is determined that the pixel point belongs to a region of interest, the pixel value corresponding to the pixel point may be set to the pixel value corresponding to the region of interest; optionally, when the pixel value corresponding to the non-interested region is 0, the pixel value corresponding to the region of interest may be 1; when the pixel value corresponding to the non-interested region is 1, the pixel value corresponding to the region of interest may be 0. Thus, a binary segmentation map corresponding to the non-interested region may be obtained.

[0147] Step 1301 , based on the binary segmentation map, segment the region of interest in the original scanned image to obtain a segmentation result of the region of interest.

[0148] The binary segmentation map is used to represent the non-interest region and the interest region in the original scanned image, and the pixels of the non-interest region and the pixels of the interest region are represented by two different pixel values.

[0149] Then, when a binary segmentation map is obtained, pixels whose pixel values ​​are consistent with the pixel values ​​corresponding to the region of interest can be retained, and pixels whose pixel values ​​are inconsistent with the pixel values ​​corresponding to the region of interest can be removed, so as to segment the region of interest from the original scanned image and obtain the segmentation result of the region of interest.

[0150] In this embodiment, the computer device can determine the binary segmentation map corresponding to the non-interested region based on the target probability map corresponding to the non-interested region; then, the region of interest in the original scanned image can be segmented based on the binary segmentation map to obtain the segmentation result of the region of interest. The image segmentation method in this application can improve the segmentation efficiency and segmentation accuracy of the region of interest.

[0151] In one embodiment, a specific implementation process of an image segmentation method is provided. In this embodiment, the network structure of the preset segmentation model is as follows: Fig.14 As shown, it includes three parts; the first part is feature extraction, including semantic branch and spatial branch; the second part is feature fusion, which adopts bilateral guided aggregation network; the third part is result output, which decodes and restores the features and outputs the classification probability map of image pixels.

[0152] The following is a detailed description of the three-part network structure of the preset segmentation model.

[0153] Part I: Using a bilateral network

[0154] (1) Semantic branch

[0155] Stem Block: Initializes the downsampling layer, pre-extracts the underlying semantic features, and extracts features by superimposing multiple features.

[0156] GE Block: extracts high-level semantic features, uses deep separable convolution (dwconv), reduces network parameters, and improves the network's reasoning speed. At the same time, it extracts features by adding multiple features.

[0157] Context Embedding Module (CE Block): Uses the attention mechanism to obtain global information, distinguish the weights of different features, and add the global information to the features of the semantic branch to further improve the semantic branch's extraction of feature information. Figure 8 shown.

[0158] (2) Space branch

[0159] It is composed of different convolution modules to extract spatial information. Each convolution module can include a convolution layer (conv), a batch normalization layer (bn), and an activation function layer (relu).

[0160] In addition, it should be noted that the dotted area in the first part uses a self-distillation mechanism to perform mutual learning between semantic features of different layers. For example, the mean square error loss function MSELoss can be used to calculate the MSELoss of the feature maps of two adjacent layers to obtain the loss value between the feature maps of the two adjacent layers to train the bilateral segmentation network.

[0161] In addition, the self-distillation mechanism can be used after the first stage of training is completed, in order to further improve the effect of feature extraction. After a certain stage of training, the network has learned most of the semantic information, and further training at this time will not be very helpful in improving the effect. After adding the self-distillation mechanism, the effect can be further improved.

[0162] Part 2: Feature Fusion

[0163] A bilateral guided aggregation network is used to fuse the features of different branches through conventional convolution (conv), depthwise separable convolution (dwconv), and upsampling operations to output the complete features extracted by the final network.

[0164] Part 3: Result Output

[0165] It can be a decoding layer, which decodes and restores the extracted features and outputs the classification probability map of the image pixels.

[0166] Based on the preset segmentation model of the above network structure, the non-interested region of the original scanned image can be segmented to obtain the image of the interested region corresponding to the original scanned image. The image segmentation process may include:

[0167] Step 1: Input the original scanned image into the preset segmentation model to obtain the target probability map of the non-interested area.

[0168] Step 2: Based on the target probability map corresponding to the non-interest region, determine the binary segmentation map corresponding to the non-interest region.

[0169] Step 3: Based on the binary segmentation map corresponding to the non-interest region, the original scanned image is segmented to obtain a non-interest region map.

[0170] Step 4: based on the non-interest region map, the region of interest is cropped from the original scanned image to obtain a region of interest map.

[0171] Step 5, performing advanced post-processing operations on the region of interest map, that is, performing image analysis processing on the region of interest map to obtain a target processing result corresponding to the region of interest map.

[0172] The target processing result may include an image obtained by performing advanced post-processing on the region of interest map.

[0173] The image segmentation method in this embodiment uses a deep neural network method to accurately segment the beam limiter area in advance, and then cut out the region of interest based on the segmented beam limiter area. This method has high accuracy and strong robustness, and can handle image segmentation processing of most complex scenes, such as non-human area shooting, rotating beam limiter or flat plate position shooting, etc. In addition, in this embodiment, an improved bilateral segmentation network is used for image segmentation processing. Compared with the traditional grayscale-based segmentation, although the beam limiter area is usually a low-grayscale area, it is subject to some X-ray radiation and there will also be some high-grayscale areas. The traditional grayscale-based segmentation cannot completely segment the beam limiter area, while the improved bilateral segmentation network in this application can completely segment the beam limiter area, thereby improving the segmentation effect of the non-interested area.

[0174] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0175] Based on the same inventive concept, the embodiment of the present application also provides an image segmentation device for implementing the above-mentioned image segmentation method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image segmentation device embodiments provided below can refer to the limitations on the image segmentation method above, and will not be repeated here.

[0176] In one embodiment, Fig.15 As shown, an image segmentation device is provided, comprising: a first acquisition module 1501, a first segmentation module 1502 and a second segmentation module 1503, wherein:

[0177] The first acquisition module 1501 is used to acquire an original scanned image; the original scanned image includes a non-interest region and an interest region.

[0178] The first segmentation module 1502 is used to input the original scanned image into a preset segmentation model, perform image segmentation on the non-interest area in the original scanned image, and obtain the segmentation result of the non-interest area; the preset segmentation model is trained based on the initial bilateral segmentation network, the attention mechanism and the self-distillation mechanism.

[0179] The second segmentation module 1503 is used to segment the region of interest in the original scanned image based on the segmentation result of the non-region of interest to obtain the segmentation result of the region of interest.

[0180] In one embodiment, the preset segmentation model includes a semantic branch, a spatial branch, a bilateral guided aggregation layer and an output layer; the semantic branch adopts an attention mechanism; the first segmentation module 1502 is used to input the original scanned image into the semantic branch, perform feature extraction in combination with the attention mechanism, and generate the semantic features of the original scanned image; input the original scanned image into the spatial branch for feature extraction to generate the spatial features of the original scanned image; input the semantic features and spatial features of the original scanned image into the bilateral guided aggregation layer for feature fusion processing to generate feature fusion results; input the feature fusion results into the output layer, and perform image segmentation on the non-interested areas in the original scanned image through the output layer to obtain the segmentation results of the non-interested areas.

[0181] In one embodiment, the semantic branch includes a first feature extraction module and a second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; the first segmentation module 1502 is specifically used to input the original scanned image into the first feature extraction module for feature extraction to obtain a first feature map of the original scanned image; the first feature map is input into the context embedding module for attention feature extraction to obtain a second feature map corresponding to the first feature map; the second feature map is used to characterize the attention weights of non-interest areas and interest areas; the first feature map is multiplied by the second feature map to generate the semantic features of the original scanned image.

[0182] In one embodiment, the device also includes: a second acquisition module 1501, a third segmentation module and a training module; wherein the second acquisition module 1501 is used to obtain the sample scan image and the annotated segmentation results of the non-interest area corresponding to the sample scan image; the third segmentation module is used to input the sample scan image into the initial bilateral segmentation network, perform image segmentation on the non-interest area in the sample scan image, and obtain the predicted segmentation results of the non-interest area in the sample scan image; the training module is used to train the initial bilateral segmentation network based on the predicted segmentation results and the annotated segmentation results, using a self-distillation mechanism to obtain a preset segmentation model.

[0183] In one of the embodiments, the initial bilateral segmentation network includes an initial semantic branch, the initial semantic branch includes at least one first feature extraction module and at least one second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts an attention mechanism; the above-mentioned training module is used to input the predicted segmentation result and the labeled segmentation result into a first preset loss function to obtain the value of the first preset loss function; based on the value of the first preset loss function and the preset training conditions, the initial bilateral segmentation network is trained to obtain an intermediate segmentation model that meets the preset training conditions; according to the intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model, the intermediate segmentation model is trained to obtain a preset segmentation model.

[0184] In one of the embodiments, the above-mentioned training module is specifically used to obtain each intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model; select any two target intermediate feature maps from each intermediate feature map, input any two target intermediate feature maps into the second preset loss function, obtain multiple values ​​of the second preset loss function, and update the value of the first preset loss function according to the predicted segmentation result and the labeled segmentation result of the intermediate segmentation model; based on the values ​​of the multiple second preset loss functions and the updated value of the first preset loss function, the intermediate segmentation model is trained to obtain the preset segmentation model.

[0185] In one of the embodiments, if the segmentation result of the non-interest region is a target probability map corresponding to the non-interest region, the second segmentation module 1503 is used to determine a binary segmentation map corresponding to the non-interest region based on the target probability map corresponding to the non-interest region; based on the binary segmentation map, the region of interest in the original scanned image is segmented to obtain a segmentation result of the region of interest.

[0186] Each module in the above-mentioned image segmentation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0187] In one embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.16As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as original scanned images and historical scanned images. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image segmentation method is implemented. In the case where the computer device is a terminal, it can also include a display screen and an input device (not shown in the figure). Among them, the display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0188] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0189] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the image segmentation method in each of the above embodiments when executing the computer program.

[0190] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image segmentation method in each of the above embodiments are implemented.

[0191] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the image segmentation method in each of the above embodiments when executed by a processor.

[0192] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0193] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image segmentation method, characterized in that: The method comprises: Acquire an original scanned image; the original scanned image includes a non-interest region and an interest region; Inputting the original scanned image into a preset segmentation model, performing image segmentation on the non-interested region in the original scanned image, and obtaining a segmentation result of the non-interested region; the preset segmentation model is trained based on an initial bilateral segmentation network, an attention mechanism, and a self-distillation mechanism; Based on the segmentation result of the non-interest region, the region of interest in the original scanned image is segmented to obtain a segmentation result of the region of interest.

2. The method according to claim 1, characterized in that The preset segmentation model includes a semantic branch, a spatial branch, a bilateral guided aggregation layer and an output layer; the semantic branch adopts the attention mechanism; the original scanned image is input into the preset segmentation model, the non-interested region in the original scanned image is segmented, and the segmentation result of the non-interested region is obtained, including: Inputting the original scanned image into the semantic branch, performing feature extraction in combination with the attention mechanism, and generating semantic features of the original scanned image; Inputting the original scanned image into the space branch to extract features and generate spatial features of the original scanned image; Inputting the semantic features and the spatial features of the original scanned image into the bilateral guided aggregation layer for feature fusion processing to generate a feature fusion result; The feature fusion result is input to the output layer, and the non-interest region in the original scanned image is segmented through the output layer to obtain a segmentation result of the non-interest region.

3. The method according to claim 2, characterized in that The semantic branch includes a first feature extraction module and a second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts the attention mechanism; the original scanned image is input into the semantic branch, and feature extraction is performed in combination with the attention mechanism to generate semantic features of the original scanned image, including: Inputting the original scanned image into the first feature extraction module to perform feature extraction, thereby obtaining a first feature map of the original scanned image; Inputting the first feature map into the context embedding module to extract attention features, and obtaining a second feature map corresponding to the first feature map; the second feature map is used to characterize the attention weights of the non-interest region and the interest region; The first feature map is multiplied by the second feature map to generate a semantic feature of the original scanned image.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Obtaining a sample scan image and a labeled segmentation result of a non-interested region corresponding to the sample scan image; Inputting the sample scanned image into the initial bilateral segmentation network, performing image segmentation on the non-interested region in the sample scanned image, and obtaining a predicted segmentation result of the non-interested region in the sample scanned image; Based on the predicted segmentation result and the labeled segmentation result, the initial bilateral segmentation network is trained using a self-distillation mechanism to obtain the preset segmentation model.

5. The method according to claim 4, characterized in that The initial bilateral segmentation network includes an initial semantic branch, the initial semantic branch includes at least one first feature extraction module and at least one second feature extraction module, the second feature extraction module includes a context embedding module, and the context embedding module adopts the attention mechanism; based on the predicted segmentation result and the labeled segmentation result, the initial bilateral segmentation network is trained by a self-distillation mechanism to obtain the preset segmentation model, including: Inputting the predicted segmentation result and the labeled segmentation result into a first preset loss function to obtain a value of the first preset loss function; Based on the value of the first preset loss function and the preset training condition, the initial bilateral segmentation network is trained to obtain an intermediate segmentation model that meets the preset training condition; The intermediate segmentation model is trained according to the intermediate feature maps output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model to obtain the preset segmentation model.

6. The method according to claim 5, characterized in that The step of training the intermediate segmentation model according to the intermediate feature graph output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model to obtain the preset segmentation model comprises: Acquire each intermediate feature map output by at least one first feature extraction module and at least one second feature extraction module in the intermediate segmentation model; Selecting any two target intermediate feature maps from each of the intermediate feature maps, inputting the any two target intermediate feature maps into a second preset loss function, obtaining a plurality of values ​​of the second preset loss function, and updating the value of the first preset loss function according to the predicted segmentation result of the intermediate segmentation model and the labeled segmentation result; Based on the values ​​of the plurality of second preset loss functions and the updated value of the first preset loss function, the intermediate segmentation model is trained to obtain the preset segmentation model.

7. The method according to claim 1, characterized in that If the segmentation result of the non-interest region is a target probability map corresponding to the non-interest region, segmenting the region of interest in the original scanned image based on the segmentation result of the non-interest region to obtain the segmentation result of the region of interest includes: Determine a binary segmentation map corresponding to the non-interest region based on the target probability map corresponding to the non-interest region; Based on the binary segmentation map, the region of interest in the original scanned image is segmented to obtain a segmentation result of the region of interest.

8. An image segmentation device, characterized in that: The device comprises: A first acquisition module is used to acquire an original scanned image; the original scanned image includes a non-interest region and an interest region; A first segmentation module is used to input the original scanned image into a preset segmentation model, perform image segmentation on the non-interested region in the original scanned image, and obtain a segmentation result of the non-interested region; the preset segmentation model is trained based on an initial bilateral segmentation network, an attention mechanism, and a self-distillation mechanism; The second segmentation module is used to segment the region of interest in the original scanned image based on the segmentation result of the non-region of interest to obtain the segmentation result of the region of interest.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images